2023#

Chapter 1: The Magical Forest

It was a beautiful day in the magical forest. The sun was shining and the birds were singing. The trees were tall and green, and the flowers were blooming.

In the center of the forest was a small cottage, where a young girl named Lily lived with her parents. Lily was six years old, and she loved to explore the forest.

One day, while out on a walk, Lily stumbled upon a strange and magical place. It was a clearing filled with glittering diamonds and sparkling jewels.

"Wow!" exclaimed Lily. "I've never seen anything like this before!"
As she reached out to touch one of the diamonds, a bright light shone and a magical fairy appeared before her.

"Hello, Lily," said the fairy. "I am the guardian of the forest, and I have been waiting for you. You have a very special task to complete."

"Me?" asked Lily. "What can I do?"

"There is a wicked witch who has been causing trouble in the forest," explained the fairy. "She has been turning all of the animals into stone, and we need your help to stop her."

"I'll do it!" said Lily, determined to help the animals.

Chapter 2: The Wicked Witch

Lily set out on her journey to find the wicked witch. She followed the fairy's instructions and travelled deep into the forest.
As she walked, she saw that the trees were no longer green and the flowers had withered away. It was clear that the witch had been causing destruction wherever she went.

Finally, Lily came to a dark and gloomy castle. She knew this was where the witch lived.

She tiptoed through the castle, trying not to make a sound. She could hear the witch cackling in the next room.

Lily peeked through the door and saw the witch stirring a bubbling pot. She had a long, crooked nose and sharp, pointed teeth.

"Ah-ha!" shouted the witch, turning to face Lily. "I've been expecting you, little girl. You won't be able to stop me from turning all of the animals into stone!"

But Lily was not afraid. She remembered the fairy's words and knew she had to be brave.

"I won't let you harm any more animals!" said Lily, standing up to the witch.

The witch cackled and raised her wand, but Lily was quick. She grabbed a nearby broom and swung it at the witch, knocking the witches wand out of her hand.

The witch let out a cry of rage and fled the castle, leaving Lily to save the animals.

Chapter 3: The Great Celebration

With the witch gone, Lily was able to use her magic to turn the animals back to normal. The forest was filled with joy and celebration.

The fairy thanked Lily for her bravery and offered to grant her one wish.

"I wish for the forest to always be a happy and safe place," said Lily.

The fairy nodded and waved her wand, granting Lily's wish. From then on, the magical forest was filled with happiness and peace.
Lily returned home to her parents, who were overjoyed to see her safe and sound. They threw a great celebration in her honour, and the whole forest came to join in the fun.

Lily knew that she would always treasure her adventure in the magical forest, and she hoped to have many more adventures in the future.

Chapter 1: The Science Fair

It was a beautiful day in the town of Maplewood, and Lily was excited for the annual science fair at her school. She had been working on her project for weeks, and she couldn't wait to show it off.

Lily's project was a model of the solar system, complete with miniature planets made out of clay and a sun made out of papier-mâché. She had worked hard to make sure everything was accurate and to explain the orbits of the planets in a way that other kids could understand.

As the science fair began, Lily set up her project at her table and waited for the judges to come by. She was a little nervous, but she knew she had done her best.

The judges were impressed by Lily's project and asked her lots of questions about the solar system. Lily was happy to answer and explain what she had learned.

In the end, Lily's project won first place in the science fair! She was so proud of herself and couldn't wait to show her parents.

Chapter 2: The Technology Club

After the science fair, Lily's teacher, Mrs. Kim, asked her if she would like to join the Technology Club at school. Lily was excited at the chance to learn more about science and technology, and she eagerly accepted.

In the Technology Club, Lily and her friends learned about all sorts of different topics, from coding and robotics to engineering and renewable energy. They built and programmed robots, designed and tested bridges, and learned about how solar panels work.

Lily loved the Technology Club and couldn't wait to go each week. She learned so much and had a lot of fun doing it.

Chapter 3: The Engineering Competition

One day, Mrs. Kim announced that the Technology Club was going to participate in an engineering competition. The students would have to design and build a Rube Goldberg machine, a complicated contraption that could perform a simple task in a creative and convoluted way.

Lily and her friends were up for the challenge and set to work planning and building their machine. They worked together as a team, using their knowledge of physics, engineering, and problem-solving to make their machine work.

Finally, the day of the competition arrived. Lily and her friends were nervous but excited to see how their machine would perform.

As they presented their machine to the judges, it worked perfectly, performing the simple task with a series of intricate steps. Lily and her friends were thrilled when they were announced as the winners of the competition!

Chapter 4: The Future Scientists

After the engineering competition, Lily and her friends in the Technology Club knew that they wanted to continue exploring science and technology in the future. They were already thinking about new projects and competitions they could participate in.

Lily knew that she wanted to be a scientist when she grew up, and she was grateful for all of the opportunities she had had to learn and explore in the Technology Club.

She knew that no matter what she did in the future, she would always have a passion for science and technology and the desire to learn more.

Cassie and the Apple Tree

Cassie was a curious little girl who loved to learn about the world around her. One day, she was walking through her backyard when she saw an apple fall from a tree.

"Hmm, I wonder why that happened?" Cassie thought to herself. She climbed up the tree to see if she could find out more.

As she reached the top, she saw a man standing under the tree. It was Sir Isaac Newton! Cassie had heard of him before. He was a very smart man who had made many important discoveries.
"Hello, Cassie," Sir Isaac said. "I see you're curious about the apple."

"Yes, I am," Cassie replied. "Why did it fall to the ground?"
"Ah, that's a good question," Sir Isaac said. "I figured out that there is a force called gravity that pulls objects towards the ground. That's why the apple fell."

Cassie was amazed. "Wow, that's so cool!" she exclaimed.
Sir Isaac smiled. "I also made many other discoveries," he told Cassie. "Like the laws of motion, which explain how objects move and change speed. And I invented calculus, which helps us understand how things change over time."

Cassie's head was spinning with all this new information. She couldn't wait to learn more about Sir Isaac's discoveries.

"Thank you for telling me all about this, Sir Isaac," Cassie said. "I can't wait to learn more!"

"You're welcome, Cassie," Sir Isaac replied. "I'm glad you're interested in learning. Remember, the more you learn, the more you'll discover about the world around you."

Cassie smiled and climbed back down the tree, excited to keep learning and discovering new things.

Chapter 1: Cassie and the Apple Tree

[Illustration: Cassie is standing in her backyard, looking up at an apple falling from a tree.]

Cassie was a curious little girl who loved to learn about the world around her. One day, she was walking through her backyard when she saw an apple fall from a tree.

"Hmm, I wonder why that happened?" Cassie thought to herself. She climbed up the tree to see if she could find out more.

[Illustration: Cassie is climbing up the tree, with a determined look on her face.]

As she reached the top, she saw a man standing under the tree. It was Sir Isaac Newton! Cassie had heard of him before. He was a very smart man who had made many important discoveries.

"Hello, Cassie," Sir Isaac said. "I see you're curious about the apple."

"Yes, I am," Cassie replied. "Why did it fall to the ground?"

"Ah, that's a good question," Sir Isaac said. "I figured out that there is a force called gravity that pulls objects towards the ground. That's why the apple fell."

[Illustration: Sir Isaac is pointing to a diagram of the earth and the apple, with an arrow labeled "gravity" pointing from the earth to the apple.]

Cassie was amazed. "Wow, that's so cool!" she exclaimed.

Chapter 2: The Laws of Motion

After learning about gravity, Cassie couldn't wait to find out more about Sir Isaac's discoveries. She asked him to tell her more.

"I also made many other discoveries," Sir Isaac said. "Like the laws of motion, which explain how objects move and change speed."

[Illustration: Sir Isaac is holding a book labeled "Laws of Motion," with diagrams of moving objects in the background.]

Cassie was fascinated. Sir Isaac explained that there are three laws of motion:

    Objects at rest stay at rest, and objects in motion stay in motion with the same speed and direction, unless acted upon by a force.
    The force needed to change the motion of an object is equal to the mass of the object times its acceleration.
    For every action, there is an equal and opposite reaction.

[Illustration: Sir Isaac is pointing to a chalkboard with the three laws of motion written on it, with illustrations of objects in motion on either side.]

"These laws help us understand how objects move and behave in the world around us," Sir Isaac said.

Chapter 3: Calculus

Cassie couldn't believe how much she was learning from Sir Isaac. She asked him if there was anything else he had discovered.

"Yes, I also invented calculus," Sir Isaac said. "It helps us understand how things change over time."

[Illustration: Sir Isaac is holding a book labeled "Calculus," with a diagram of a curve on the cover.]

Cassie was a little confused. "What do you mean?" she asked.

Sir Isaac explained that calculus is a way of studying how things change. "For example, we can use calculus to understand how fast a ball is rolling down a hill, or how fast a plant is growing," he said.
[Illustration: Sir Isaac is pointing to a graph with a curve on it, with a ball rolling down a hill in the background.]
Cassie was amazed. "That's so cool!" she exclaimed.
Chapter 4: The End
After learning all about Sir Isaac's discoveries, Cassie was even more excited to keep learning and discovering new things. She thanked Sir Isaac for all of his help and said goodbye.
"Remember, Cassie," Sir Isaac said as she walked away. "The more you learn, the more you'll discover about the world around you."
[Illustration: Cassie is waving goodbye to Sir Isaac, with a big smile on her face.]
Cassie smiled and waved, excited to keep exploring and learning. She knew that there were so many more mysteries to solve and discoveries to be made.

import logging
from dataclasses import dataclass
from typing import List, Optional

import hydra
from hydra.core.config_store import ConfigStore
from hydra.core.hydra_config import HydraConfig
from omegaconf import MISSING, DictConfig, OmegaConf

log = logging.getLogger(__name__)


Hydra configuration framework. URL: [https://hydra.cc/](https://hydra.cc/)..main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    log.info("Type of cfg is: %s", type(cfg))
    cfg_dict = OmegaConf.to_container(cfg, throw_on_missing=False, resolve=True)

    log.info("Merged Yaml:\n%s", OmegaConf.to_yaml(cfg))

    log.info(cfg.db)
    log.info(cfg.db.driver)

    log.info(HydraConfig.get().job.name)


if __name__ == "__main__":
    my_app()

python my_app_non_structured.py --config-dir conf_custom_hydra
python my_app_non_structured.py --config-name config_hydra

db:
  driver: mysql
  user: suneeta
  pwd: secret
ui:
  create_db: true
  view: true
schema:
  database: school
  tables:
  - name: students
  - name: exams

python my_app_non_structured.py --config-dir=outputs/2022-03-16/13-45-29/.hydra

defaults:
  - _self_  # Note: to source default of hydra config 
  - db: mysql
  - ui: full
  - schema: school

hydra:
  job:
    env_set: 
      HDF5_USE_FILE_LOCKING: "false"
      ## Useful for distributed train setup
      # RANK: ${hydra.job.num}
    env_copy: ## Local environment copy
      - API_KEY  

  job_logging:
    version: 1
    formatters:
      simple:
        format: "[%(levelname)s] - %(message)s"
    handlers:
      console:
        class: logging.StreamHandler
        formatter: simple
        stream: ext://sys.stdout
    root:
      handlers: [console]

    disable_existing_loggers: false

  run:
    dir: outputs/${hydra.job.name}/${now:%Y-%m-%d_%H-%M-%S}

python my_app_non_structured.py db.user=suneeta schema=school,support,warehouse  --config-dir conf_custom_hydra --multirun

hydra:
  job:
    env_set: 
      RANK: ${hydra.job.num}

Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class DBConfig:
    user: str = MISSING
    pwd: str = MISSING
    driver: str = MISSING
    timeout: int = 100


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class MySQLConfig(DBConfig):
    driver: str = "mysql"


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class PostGreSQLConfig(DBConfig):
    driver: str = "postgresql"


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class View:
    create_db: bool = False
    view: bool = MISSING


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class Table:
    name: str = MISSING


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class Schema:
    database: str = "school"
    tables: List[Table] = MISSING


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class Config:
    # We can now annotate db as DBConfig which
    # improves both static and dynamic type safety.
    db: DBConfig
    ui: View
    schema: Schema

cs = ConfigStore.instance()
cs.store(name="config", node=Config)
cs.store(group="db", name="mysql", node=MySQLConfig)
cs.store(group="db", name="postgresql", node=PostGreSQLConfig)
cs.store(name="ui", node=View)
cs.store(name="schema", node=Schema)

Hydra configuration framework. URL: [https://hydra.cc/](https://hydra.cc/)..main(config_path="conf", config_name="config")
def my_app(cfg: Config) -> None:    
    log.info("Type of cfg is: %s", type(cfg))
    cfg_dict = OmegaConf.to_container(cfg, throw_on_missing=False, resolve=True)

    cfg_obj = OmegaConf.to_object(cfg)

python my_app.py
python my_app.py db.user=suneeta    
python my_app.py db.user=suneeta --config-dir conf_custom_hydra
# python my_app.py db.user=suneeta --config-name config_hydra
# Multi-run
python my_app.py db.user=suneeta schema=school,support,warehouse  --config-dir conf_custom_hydra --multirun

import logging
import sys
from pathlib import Path
from typing import Optional, Union

from pydantic import BaseModel, conint, validator
from pydantic.dataclasses import dataclass


log = logging.getLogger(__name__)


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class Dataset:
    name: str
    path: str

    Python pydantic validator. URL: [https://docs.pydantic.dev/usage/validators/](https://docs.pydantic.dev/usage/validators/).("path")
    def validate_path(cls, path: str) -> Path:
        if Path(path).exists():
            print("exist")
        return Path(path)


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class Model:
    type: str
    num_layers: conint(ge=3)
    width: Optional[conint(ge=0)]

    Python pydantic validator. URL: [https://docs.pydantic.dev/usage/validators/](https://docs.pydantic.dev/usage/validators/).("type")
    def validate_supported_type(cls, type: str) -> str:
        if type not in ["alex", "le"]:
            raise ValueError(f"'type' canonly be [alex, le] got: {type}")
        return type


Python dataclass. URL: [https://docs.python.org/3/library/dataclasses.html](https://docs.python.org/3/library/dataclasses.html).
class Config:
    dataset: Dataset
    model: Model

class Dataset(BaseModel):
    name: str
    path: str

    Python pydantic validator. URL: [https://docs.pydantic.dev/usage/validators/](https://docs.pydantic.dev/usage/validators/).("path")
    def validate_path(cls, path: str) -> Path:
        if Path(path).exists():
            print("exist")
        return Path(path)

    class Config:
        title = "Dataset"
        max_anystr_length = 10
        allow_mutation = False
        validate_assignment = True
        anystr_lower = True
        validate_all = True
        use_enum_values = True 

class Secrets(BaseSettings):
    api_key: str = Field("", env="api_key")
    token: str

    class Config:
        # case_sensitive = True
        env_nested_delimiter = "_"

        Python class methods. URL: [https://docs.python.org/3/library/functions.html\#classmethod](https://docs.python.org/3/library/functions.html#classmethod).
        def customise_sources(
            cls,
            init_settings: SettingsSourceCallable,
            env_settings: SettingsSourceCallable,
            file_secret_settings: SettingsSourceCallable,
        ) -> Tuple[SettingsSourceCallable, ...]:
            return env_settings, init_settings, file_secret_settings

from typing import List, Union
from pydantic import BaseModel, conint, validator

class Model(BaseModel):
    total_classes: Union[conint(ge=0), List[conint(ge=0)]]

    Python pydantic validator. URL: [https://docs.pydantic.dev/usage/validators/](https://docs.pydantic.dev/usage/validators/).("total_classes")
    def validate_classes(cls, total_classes: Union[conint(ge=0), List[conint(ge=0)]]) -> conint(ge=0):
        if isinstance(total_classes, List):
            return len(total_classes)
        return total_classes

    OmegaConf.resolve(cfg)
    r_model = Config(**cfg)

class VersionModel(BaseModel, abc.ABC):
    v: Literal["inf"] = Field(float("inf"), const=True)

    class Config:
        title = "Model"
        allow_mutation = False
        max_anystr_length = 10
        validate_assignment = True
        anystr_lower = True
        validate_all = True
        use_enum_values = True

    def version(self) -> float:
        return self.v

    def _next_version_model(self) -> float:
        versions = list(fake_registry.keys())
        versions.sort()
        idx = versions.index(self.v) + 1
        if idx == len(versions):
            return None
        return versions[idx]

    def next_version(self) -> BaseModel:
        next_class = fake_registry.get(self._next_version_model())
        return next_class

    Python abstract base classes. URL: [https://docs.python.org/3/library/abc.html](https://docs.python.org/3/library/abc.html)..abstractmethod
    def to_next(self) -> BaseModel:
        pass


class Model1(VersionModel):
    # https://pydantic-docs.helpmanual.io/usage/types/#arguments-to-constr
    type: str  # constr(to_lower=True)
    num_layers: conint(ge=3)
    width: Optional[conint(ge=0)]
    # v: float = Field(1.0, const=True)
    zoom: conint(gt=18) = 19
    v: Literal[1.0] = Field(1.0, const=True)

    # Python class methods. URL: [https://docs.python.org/3/library/functions.html\#classmethod](https://docs.python.org/3/library/functions.html#classmethod).
    # def to_next(cls, instance: Type[VBT]) -> BaseModel:
    def to_next(self) -> Optional[BaseModel]:
        next_class = self.next_version()
        if next_class != Model2:
            raise Exception("WTH")

        logging.warning("================================================")
        logging.warning("==============Migrating from 1->2 ==============")
        logging.warning("================================================")

        d = self.dict()
        d.pop("v")
        return Model2(**d)


class Model2(Model1):
    width: conint(ge=5)
    context: conint(ge=256) = 256
    # v: float = Field(2.0, const=True)
    v: Literal[2.0] = Field(2.0, const=True)

    def to_next(self) -> Optional[BaseModel]:
        next_class = self.next_version()
        if next_class != Model3:
            raise Exception("WTH")

        logging.warning("================================================")
        logging.warning("==============Migrating from 2->3 ==============")
        logging.warning("================================================")

        return Model3(
            new_context=self.context,
            num_layers=self.num_layers,
            type=self.type,
            width=self.width,
            zoom=self.zoom,
        )

class Model3(Model1):
    new_context: conint(ge=512, le=1300) = 512
    v: Literal[3.0] = Field(3.0, const=True)

    def to_next(self) -> Optional[BaseModel]:
        logging.warning("================================================")
        logging.warning("============== Latest no migration =============")
        logging.warning("================================================")

        return None

        _Models = Annotated[Union[Model1, Model2, Model3], Field(discriminator="v")]
        _schema = schema_json_of(_Models, title="Model Schema", indent=2)

{
  "title": "Model Schema",
  "discriminator": {
    "propertyName": "v",
    "mapping": {
      "1.0": "#/definitions/Model1",
      "2.0": "#/definitions/Model2",
      "3.0": "#/definitions/Model3"
    }
  },
  "anyOf": [
    {
      "$ref": "#/definitions/Model1"
    },
    {
      "$ref": "#/definitions/Model2"
    },
    {
      "$ref": "#/definitions/Model3"
    }
  ],
  "definitions": {
    "Model1": {
      "title": "Model",
      "type": "object",
      "properties": {
        "v": {
          "title": "V",
          "const": 1.0,
          "enum": [
            1.0
          ],
          "type": "number"
        },
        "type": {
          "title": "Type",
          "type": "string"
        },
        "num_layers": {
          "title": "Num Layers",
          "minimum": 3,
          "type": "integer"
        },
        "width": {
          "title": "Width",
          "minimum": 0,
          "type": "integer"
        },
        "zoom": {
          "title": "Zoom",
          "default": 19,
          "exclusiveMinimum": 18,
          "type": "integer"
        }
      },
      "required": [
        "type",
        "num_layers"
      ]
    },
    "Model2": {
      "title": "Model",
      "type": "object",
      "properties": {
        "v": {
          "title": "V",
          "const": 2.0,
          "enum": [
            2.0
          ],
          "type": "number"
        },
        "type": {
          "title": "Type",
          "type": "string"
        },
        "num_layers": {
          "title": "Num Layers",
          "minimum": 3,
          "type": "integer"
        },
        "width": {
          "title": "Width",
          "minimum": 5,
          "type": "integer"
        },
        "zoom": {
          "title": "Zoom",
          "default": 19,
          "exclusiveMinimum": 18,
          "type": "integer"
        },
        "context": {
          "title": "Context",
          "default": 256,
          "minimum": 256,
          "type": "integer"
        }
      },
      "required": [
        "type",
        "num_layers",
        "width"
      ]
    },
    "Model3": {
      "title": "Model",
      "type": "object",
      "properties": {
        "v": {
          "title": "V",
          "const": 3.0,
          "enum": [
            3.0
          ],
          "type": "number"
        },
        "type": {
          "title": "Type",
          "type": "string"
        },
        "num_layers": {
          "title": "Num Layers",
          "minimum": 3,
          "type": "integer"
        },
        "width": {
          "title": "Width",
          "minimum": 0,
          "type": "integer"
        },
        "zoom": {
          "title": "Zoom",
          "default": 19,
          "exclusiveMinimum": 18,
          "type": "integer"
        },
        "new_context": {
          "title": "New Context",
          "default": 512,
          "minimum": 512,
          "maximum": 1300,
          "type": "integer"
        }
      },
      "required": [
        "type",
        "num_layers"
      ]
    }
  }
}