diff --git a/README.md b/README.md index 3a6f34d..0837d09 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,61 @@ ## Introduction -High-level interface to grid pools for the Frequenz platform. +High-level interface to gridpools for the Frequenz platform. -TODO(cookiecutter): Improve the README file +## Market topology configuration + +Market topology is stored under the `assets` namespace. A relation names at +least two of a gridpool, microgrid and market location: + +```toml +assets.microgrids.241.meta.microgrid_id = 241 + +assets.market_locations.10208446344.id = "10208446344" + +assets.relations.G80M241L10208446344.gridpool_id = 80 +assets.relations.G80M241L10208446344.microgrid_id = 241 +assets.relations.G80M241L10208446344.market_location_id = "10208446344" +assets.relations.G80M241L10208446344.delivery_area = "10YDE-RWENET---I" +assets.relations.G80M241L10208446344.validity.trading.participation = "ENERGY_TRADING" +assets.relations.G80M241L10208446344.validity.trading.start = 2026-01-01T00:00:00Z +``` + +A relation naming a gridpool requires a delivery area. A gridpool-free relation +must link a microgrid and market location, and may carry a delivery area for a +direct market-location-to-area mapping. Omitted market-location types default +to `MALO_ID`; omitted market areas default to `101` (`EU_DE`). + +**Current limitation:** Delivery areas support only 16-character EIC codes; +other identifier types are rejected. Validation checks the format and check +character, not registration with an EIC issuing office. + +Validity periods are half-open: the start is inclusive, the end exclusive, and +an omitted bound is open. Bounds and query instants must include a UTC offset. + +Load one or more files and query the merged document with: + +```python +from datetime import datetime, timezone +from pathlib import Path + +from frequenz.gridpool.config import load_assets_from_files + +config = load_assets_from_files( + [Path("topology.toml"), Path("topology-overrides.toml")] +) +relations = config.find_relations( + microgrid_id=241, + at=datetime(2026, 1, 15, tzinfo=timezone.utc), +) +``` + +Later files override individual fields from earlier files. The projections +`find_delivery_areas`, `find_market_locations` and `find_microgrids` accept +filters for the other relation sides and an instant. + +Market locations are keyed by raw ID, so the same raw ID cannot be used in +several market areas within one document. ## Supported Platforms @@ -86,7 +138,11 @@ Redirect stdout to save the result: gridpool-cli generate-config > microgrid.toml ``` -You can layer existing config files with the Assets API by precedence +This command emits only microgrid entries. Topology relations, including their +delivery-area codes, and market-location entries from input files are not +included in its stdout output. + +You can layer existing microgrid config files with the Assets API by precedence (`--default` < Assets API < `--override`). Values from a `--default` file are overridden by the API, while a `--override` file keeps its own values and the API only fills the gaps: @@ -97,7 +153,8 @@ gridpool-cli generate-config \ --override overrides.toml > microgrid.toml ``` -If no microgrid IDs are given, they are taken from the supplied files: +If no microgrid IDs are given, they are taken from the microgrid entries in the +supplied files: ```bash gridpool-cli generate-config --override existing.toml > microgrid.toml diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 592d704..cc74e88 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -13,18 +13,62 @@ configs = AssetsConfig.load_from_file(path).microgrids ``` - `load_configs_from_files` and `load_configs` are unchanged. + `load_configs` keeps its existing interface. `load_configs_from_files` now + layers files field by field instead of replacing a complete microgrid entry; + fields omitted by a later file retain the value from the earlier layer and + cannot be removed by omission. + +- `Metadata.delivery_area` is removed. Delivery areas are read from topology + relations instead. When relations are present, the legacy `Metadata.gid` + must be their sole gridpool ID; remove it for a microgrid that participates + in several gridpools. + +- Relation validity bounds and `at` query instants must include a UTC offset. + +- The implementation modules `config.assets`, `config.load` and + `config.microgrid` are now private. Import their public names from + `frequenz.gridpool.config` instead. ## New Features - `AssetsConfig` gives the `assets` namespace a type, so the entities still to - come are added as fields rather than as more dict lookups. Entries are checked - against the ID they are filed under wherever the class is loaded, not only via - `load_from_file`. + come are added as fields rather than as more dict lookups. Microgrid IDs are + checked during construction. `AssetsConfig.check()` performs the topology-wide + checks after all layers have been merged; the file loaders call it unless + `AssetsConfig.load_from_file` is passed `check=False`. + + File loaders ignore unknown entity tables with a warning, so a reader keeps + working against files that already carry newer entities. - Entity tables a version does not know are ignored with a warning, so a reader - keeps working against files that already carry newer entities. +- Market topology is described under `assets.relations`, based on the Assets + API `MarketTopologyRelation`: each record links at least two of a gridpool, a + microgrid and a market location, filed under a `G<>M<>L<>` key derived from its + own sides. A relation naming a gridpool sits in a `delivery_area` that rides on + the relation, so a gridpool-to-microgrid relation with no market location still + carries one. A relation's validity lives in `validity`, each entry a half-open + `[start, end)` datetime period it applies over. Use-case-specific periods + qualify a relation; separate relations let one microgrid participate in + several gridpools. The config extends the API with plain periods for relations + that do not distinguish use cases. A gridpool-free microgrid-to-market-location + relation may also carry a delivery area for a direct mapping. Market locations + live under `assets.market_locations` as self-describing entries carrying their + own identifier, how to read it (MALO by default), and the Assets API market + area (`EU_DE` by default). Delivery areas currently support only + check-character-validated EIC code strings on relations. Raw market-location + IDs must be unique within a document, including across market areas. + + `load_assets_from_files` layers several files into one document. Later files + override individual fields from earlier ones. `AssetsConfig` answers the common + lookups with `find_relations` and the projections `find_delivery_areas`, + `find_market_locations` and `find_microgrids`, each filtered by the other + sides and an instant. + + Time-varying enterprise ownership is outside this change; + `Metadata.enterprise_id` remains as a scalar field. ## Bug Fixes +- Layering config files no longer resets a field a later file leaves unset back + to its default. The raw tables are merged before they are loaded. + diff --git a/pyproject.toml b/pyproject.toml index 94f2277..0ed785c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -148,6 +148,7 @@ min-similarity-lines = 40 disable = [ "too-few-public-methods", "too-many-return-statements", + "too-many-arguments", # disabled because it conflicts with isort "wrong-import-order", "ungrouped-imports", diff --git a/src/frequenz/gridpool/__init__.py b/src/frequenz/gridpool/__init__.py index 15830f4..54b962b 100644 --- a/src/frequenz/gridpool/__init__.py +++ b/src/frequenz/gridpool/__init__.py @@ -15,7 +15,7 @@ merge_config_maps, merge_microgrid_configs, ) -from .config.assets import AssetsConfig +from .config._assets import AssetsConfig __all__ = [ "ComponentGraphConfig", diff --git a/src/frequenz/gridpool/config/__init__.py b/src/frequenz/gridpool/config/__init__.py index b0d399a..6bd4009 100644 --- a/src/frequenz/gridpool/config/__init__.py +++ b/src/frequenz/gridpool/config/__init__.py @@ -1,17 +1,18 @@ # License: MIT # Copyright © 2025 Frequenz Energy-as-a-Service GmbH -"""Microgrid configuration data model and loading.""" +"""Asset configuration data models and loading.""" from frequenz.microgrid_component_graph import ComponentGraphConfig, FormulaOverrides -from .assets import AssetsConfig -from .load import ( +from ._assets import AssetsConfig +from ._load import ( + load_assets_from_files, load_configs, load_configs_from_api, load_configs_from_files, ) -from .microgrid import ( +from ._microgrid import ( BatteryConfig, ComponentCategory, ComponentType, @@ -23,19 +24,28 @@ merge_config_maps, merge_microgrid_configs, ) +from ._topology import ( + MarketLocationConfig, + RelationConfig, + ValidityConfig, +) __all__ = [ + "AssetsConfig", "BatteryConfig", "ComponentCategory", "ComponentGraphConfig", "ComponentType", "ComponentTypeConfig", "FormulaOverrides", + "MarketLocationConfig", "Metadata", - "AssetsConfig", "MicrogridConfig", "PVConfig", + "RelationConfig", + "ValidityConfig", "WindConfig", + "load_assets_from_files", "load_configs", "load_configs_from_api", "load_configs_from_files", diff --git a/src/frequenz/gridpool/config/_assets.py b/src/frequenz/gridpool/config/_assets.py new file mode 100644 index 0000000..c0321d9 --- /dev/null +++ b/src/frequenz/gridpool/config/_assets.py @@ -0,0 +1,351 @@ +# License: MIT +# Copyright © 2026 Frequenz Energy-as-a-Service GmbH + +"""Data model for the `assets` config namespace.""" + +import logging +import tomllib +from dataclasses import field +from datetime import datetime +from pathlib import Path +from typing import Any, ClassVar, Self, Type + +import marshmallow +from frequenz.client.assets import MarketParticipationType +from marshmallow import Schema +from marshmallow_dataclass import dataclass + +from ._microgrid import MicrogridConfig +from ._topology import MarketLocationConfig, RelationConfig + +_logger = logging.getLogger(__name__) + + +@dataclass +class AssetsConfig: + """Entities described by a config document, keyed by their ID.""" + + microgrids: dict[str, MicrogridConfig] = field(default_factory=dict) + """Microgrids, keyed by microgrid ID.""" + + market_locations: dict[str, MarketLocationConfig] = field(default_factory=dict) + """Market locations, keyed by their identifier.""" + + relations: dict[str, RelationConfig] = field(default_factory=dict) + """Market topology relations, keyed by the composite of the sides they connect.""" + + class Meta: + """Ignore entity tables this version does not know about. + + A reader must keep working against files that already carry entities + added after it, so unknown tables are skipped rather than rejected. + `_warn_unknown_entities` reports them, so a mistyped table is still + visible instead of silently loading as empty. + """ + + unknown = marshmallow.EXCLUDE + + Schema: ClassVar[Type[Schema]] = Schema + + def __post_init__(self) -> None: + """Check that every microgrid is filed under its own ID. + + Relations are not checked here: an override names only the fields it + changes, so a single document may hold an incomplete record. `check` + looks at the merged result. + + Raises: + ValueError: If a key is not the ID of the entry it holds. + """ + for mid, cfg in self.microgrids.items(): + if not mid.isdigit(): + raise ValueError(f"Microgrid ID key must be numeric, got {mid}") + if int(cfg.meta.microgrid_id) != int(mid): + raise ValueError( + f"Microgrid ID mismatch: key {mid} != {cfg.meta.microgrid_id}" + ) + + def check(self) -> None: + """Check the document as a whole, once every layer has been merged. + + Raises: + ValueError: If an entry's identifier disagrees with the key it is + filed under, a relation names fewer than two sides, its key + disagrees with its fields, a gridpool relation names no delivery + area, one market location is placed in two of them, or a legacy + microgrid gridpool ID disagrees with its relations. + """ + for key, location in self.market_locations.items(): + if location.id is None: + raise ValueError(f"Market location {key}: must name its id") + if location.id != key: + raise ValueError( + f"Market location key mismatch: key {key} != {location.id}" + ) + self._check_relations() + self._check_legacy_gridpool_ids() + + def _check_relations(self) -> None: + """Check the relations are complete, well-keyed and area-consistent. + + Raises: + ValueError: If a relation names fewer than two sides, its key + disagrees with its fields, a gridpool relation names no delivery + area, or one market location is placed in two of them. + """ + for key, relation in self.relations.items(): + if not relation.is_complete: + raise ValueError( + f"Relation {key}: must name at least two of gridpool, microgrid " + "and market location" + ) + if key != relation.key: + raise ValueError( + f"Relation key mismatch: key {key} != {relation.key}, derived " + "from the sides the record names" + ) + if relation.gridpool_id is not None and relation.delivery_area is None: + raise ValueError( + f"Relation {key}: a gridpool relation must name a delivery area" + ) + + zones: dict[str, str] = {} + for relation in self.relations.values(): + mlid, zone = relation.market_location_id, relation.delivery_area + if mlid is None or zone is None: + continue + if zones.setdefault(mlid, zone) != zone: + raise ValueError( + f"Market location {mlid} is placed in two delivery areas: " + f"{zones[mlid]} and {zone}" + ) + + def _check_legacy_gridpool_ids(self) -> None: + """Check legacy microgrid gridpool IDs against the relations.""" + gridpools_by_microgrid: dict[int, set[int]] = {} + for relation in self.relations.values(): + if relation.microgrid_id is None or relation.gridpool_id is None: + continue + gridpools_by_microgrid.setdefault(relation.microgrid_id, set()).add( + relation.gridpool_id + ) + + for microgrid in self.microgrids.values(): + legacy_gid = microgrid.meta.gid + if legacy_gid is None: + continue + relation_gids = gridpools_by_microgrid.get(microgrid.meta.microgrid_id) + if relation_gids and relation_gids != {legacy_gid}: + raise ValueError( + f"Microgrid {microgrid.meta.microgrid_id}: legacy meta.gid " + f"{legacy_gid} disagrees with relation gridpools " + f"{sorted(relation_gids)}; remove meta.gid when several apply" + ) + + def find_relations( + self, + *, + gridpool_id: int | None = None, + microgrid_id: int | None = None, + market_location_id: str | None = None, + delivery_area: str | None = None, + participation: MarketParticipationType | None = None, + at: datetime | None = None, + ) -> list[RelationConfig]: + """Find the relations naming all of the given sides. + + Args: + gridpool_id: Gridpool to match, or `None` to ignore. + microgrid_id: Microgrid to match, or `None` to ignore. + market_location_id: Market location to match, or `None` to ignore. + delivery_area: Delivery area to match, or `None` to ignore. + participation: Use case the relation must serve, or `None` to ignore. + at: Instant the relations, or the given participation, must apply at, + or `None` to ignore. + + Returns: + The matching relations, in document order. + """ + return [ + relation + for relation in self.relations.values() + if relation.matches( + gridpool_id=gridpool_id, + microgrid_id=microgrid_id, + market_location_id=market_location_id, + delivery_area=delivery_area, + participation=participation, + at=at, + ) + ] + + def find_delivery_areas( + self, + *, + gridpool_id: int | None = None, + microgrid_id: int | None = None, + market_location_id: str | None = None, + at: datetime | None = None, + ) -> list[str]: + """List the delivery areas of the matching relations. + + Args: + gridpool_id: Gridpool to match, or `None` to ignore. + microgrid_id: Microgrid to match, or `None` to ignore. + market_location_id: Market location to match, or `None` to ignore. + at: Instant the relations must apply at, or `None` to ignore. + + Returns: + The delivery areas, deduplicated, in document order. + """ + return list( + dict.fromkeys( + relation.delivery_area + for relation in self.find_relations( + gridpool_id=gridpool_id, + microgrid_id=microgrid_id, + market_location_id=market_location_id, + at=at, + ) + if relation.delivery_area is not None + ) + ) + + def find_market_locations( + self, + *, + gridpool_id: int | None = None, + microgrid_id: int | None = None, + delivery_area: str | None = None, + at: datetime | None = None, + ) -> list[str]: + """List the market locations of the matching relations. + + Args: + gridpool_id: Gridpool to match, or `None` to ignore. + microgrid_id: Microgrid to match, or `None` to ignore. + delivery_area: Delivery area to match, or `None` to ignore. + at: Instant the relations must apply at, or `None` to ignore. + + Returns: + The market locations, deduplicated, in document order. + """ + return list( + dict.fromkeys( + relation.market_location_id + for relation in self.find_relations( + gridpool_id=gridpool_id, + microgrid_id=microgrid_id, + delivery_area=delivery_area, + at=at, + ) + if relation.market_location_id is not None + ) + ) + + def find_microgrids( + self, + *, + gridpool_id: int | None = None, + market_location_id: str | None = None, + delivery_area: str | None = None, + at: datetime | None = None, + ) -> list[int]: + """List the microgrids of the matching relations. + + Args: + gridpool_id: Gridpool to match, or `None` to ignore. + market_location_id: Market location to match, or `None` to ignore. + delivery_area: Delivery area to match, or `None` to ignore. + at: Instant the relations must apply at, or `None` to ignore. + + Returns: + The microgrids, deduplicated, in document order. + """ + return list( + dict.fromkeys( + relation.microgrid_id + for relation in self.find_relations( + gridpool_id=gridpool_id, + market_location_id=market_location_id, + delivery_area=delivery_area, + at=at, + ) + if relation.microgrid_id is not None + ) + ) + + @classmethod + def _warn_unknown_entities(cls, assets: dict[str, Any], source: Path) -> None: + """Warn about entity tables that this version drops on load.""" + if unknown := sorted(set(assets) - set(cls.Schema().fields)): + _logger.warning( + "%s: ignoring unknown entity tables under `assets`: %s", + source, + ", ".join(unknown), + ) + + @classmethod + def _read_assets_table(cls, config_path: Path) -> dict[str, Any]: + """Read the raw `assets` table from a TOML file. + + Entries live under `assets`. A document without an `assets` table is + read in the deprecated layout, where the microgrid entries sit at the + top level. + + Args: + config_path: The path to the TOML configuration file. + + Returns: + The raw `assets` table, unvalidated, for merging before it is loaded. + + Raises: + TypeError: If `assets` is not a table. + ValueError: If both layouts are present, which means a half-migrated + file rather than a merge. + """ + with config_path.open("rb") as f: + data: dict[str, Any] = tomllib.load(f) + + if "assets" not in data: + _logger.warning( + "%s: top-level microgrid IDs are deprecated, " + "nest the entries under `assets.microgrids` instead.", + config_path, + ) + data = {"assets": {"microgrids": data}} + + assets = data["assets"] + if not isinstance(assets, dict): + raise TypeError( + f"{config_path}: `assets` must be a table, got {type(assets)}" + ) + + if unprefixed := sorted(k for k in data if k != "assets"): + raise ValueError( + f"{config_path}: keys {unprefixed} sit outside `assets` while the " + "file already has an `assets` table; move them under " + "`assets.microgrids`." + ) + + cls._warn_unknown_entities(assets, config_path) + return assets + + @classmethod + def load_from_file(cls, config_path: Path, check: bool = True) -> Self: + """Load and validate a config document from a TOML file. + + Args: + config_path: The path to the TOML configuration file. + check: Whether to check the document as a whole. Pass `False` for a + file that is one layer of several, since an override is + incomplete until it has been merged. + + Returns: + The loaded configuration. + """ + loaded = cls.Schema().load(cls._read_assets_table(config_path)) + assert isinstance(loaded, cls) + if check: + loaded.check() + return loaded diff --git a/src/frequenz/gridpool/config/load.py b/src/frequenz/gridpool/config/_load.py similarity index 82% rename from src/frequenz/gridpool/config/load.py rename to src/frequenz/gridpool/config/_load.py index 2dacf64..2266d48 100644 --- a/src/frequenz/gridpool/config/load.py +++ b/src/frequenz/gridpool/config/_load.py @@ -1,10 +1,11 @@ # License: MIT # Copyright © 2025 Frequenz Energy-as-a-Service GmbH -"""Loading and merging of microgrid configurations.""" +"""Loading and merging of asset configurations.""" import logging from pathlib import Path +from typing import Any from frequenz.client.assets import AssetsApiClient from frequenz.client.common.microgrid import MicrogridId @@ -24,8 +25,8 @@ pv_inverter_ids, pv_meter_ids, ) -from .assets import AssetsConfig -from .microgrid import ( +from ._assets import AssetsConfig +from ._microgrid import ( ComponentTypeConfig, Metadata, MicrogridConfig, @@ -124,55 +125,89 @@ async def load_configs( return merge_config_maps(base=configs, override=override_configs) -def load_configs_from_files( - microgrid_config_files: str | Path | list[str | Path] | None = None, -) -> dict[str, "MicrogridConfig"]: - """Load multiple microgrid configurations from one or more files. +def _deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]: + """Merge two raw config tables, with *override* winning. - Configs for a single microgrid are expected to be in a single file. - Later files with the same microgrid ID will overwrite the previous configs. + Nested tables are merged recursively; any other value in *override* replaces + the one in *base*. Merging the raw tables, before they are loaded, keeps a + field left unset in an override from resetting the base value to its default. Args: - microgrid_config_files: Path to a single microgrid config file or list of paths. + base: The table to merge into. + override: The table taking precedence. Returns: - Dictionary of single microgrid formula configs with microgrid IDs as keys. + A new table representing the merged result. + """ + result = dict(base) + for key, value in override.items(): + if isinstance(value, dict) and isinstance(result.get(key), dict): + result[key] = _deep_merge(result[key], value) + else: + result[key] = value + return result + + +def load_assets_from_files( + config_files: str | Path | list[str | Path] | None = None, +) -> AssetsConfig: + """Load one config document from one or more files. + + Later files take precedence, entry by entry, so a file can override single + fields of an entry another file defines. The raw tables are merged before + they are loaded, so a field left unset in an override keeps the base value + rather than being reset to its default. Only the merged result is checked, + since a file that overrides one field is incomplete on its own. Paths that + are not files are skipped with a warning. + + Args: + config_files: Path to a single config file or list of paths. + + Returns: + The merged document. Raises: - ValueError: If no config files are provided, or if no config files are found. + ValueError: If no config files are provided. """ - if microgrid_config_files is None: - raise ValueError( - "No microgrid config files provided. Please provide at least one." - ) - - config_files: list[Path] = [] + if config_files is None: + raise ValueError("No config files provided. Please provide at least one.") - if microgrid_config_files: - if isinstance(microgrid_config_files, str): - config_files = [Path(microgrid_config_files)] - elif isinstance(microgrid_config_files, Path): - config_files = [microgrid_config_files] - elif isinstance(microgrid_config_files, list): - config_files = [Path(f) for f in microgrid_config_files] + if isinstance(config_files, (str, Path)): + paths = [Path(config_files)] + else: + paths = [Path(f) for f in config_files] - if len(config_files) == 0: + if not paths: raise ValueError( - "No microgrid config files found. " - "Please provide at least one valid config file." + "No config files found. Please provide at least one valid config file." ) - microgrid_configs: dict[str, "MicrogridConfig"] = {} - - for config_path in config_files: + merged: dict[str, Any] = {} + for config_path in paths: if not config_path.is_file(): _logger.warning("Config path %s is not a file, skipping.", config_path) continue + # pylint: disable-next=protected-access + merged = _deep_merge(merged, AssetsConfig._read_assets_table(config_path)) + + loaded = AssetsConfig.Schema().load(merged) + assert isinstance(loaded, AssetsConfig) + loaded.check() + return loaded - mcfgs = AssetsConfig.load_from_file(config_path).microgrids - microgrid_configs.update({str(key): value for key, value in mcfgs.items()}) - return microgrid_configs +def load_configs_from_files( + microgrid_config_files: str | Path | list[str | Path] | None = None, +) -> dict[str, "MicrogridConfig"]: + """Load multiple microgrid configurations from one or more files. + + Args: + microgrid_config_files: Path to a single microgrid config file or list of paths. + + Returns: + Dictionary of single microgrid formula configs with microgrid IDs as keys. + """ + return load_assets_from_files(microgrid_config_files).microgrids async def load_configs_from_api( diff --git a/src/frequenz/gridpool/config/microgrid.py b/src/frequenz/gridpool/config/_microgrid.py similarity index 99% rename from src/frequenz/gridpool/config/microgrid.py rename to src/frequenz/gridpool/config/_microgrid.py index be0c6d5..5210a32 100644 --- a/src/frequenz/gridpool/config/microgrid.py +++ b/src/frequenz/gridpool/config/_microgrid.py @@ -186,10 +186,7 @@ class Metadata: """Enterprise ID of the microgrid.""" gid: int | None = None - """Gridpool ID of the microgrid.""" - - delivery_area: str | None = None - """Delivery area of the microgrid.""" + """Legacy ID; if set, all gridpool relations must name it.""" latitude: float | None = None """Geographic latitude of the microgrid.""" diff --git a/src/frequenz/gridpool/config/_topology.py b/src/frequenz/gridpool/config/_topology.py new file mode 100644 index 0000000..5ef5115 --- /dev/null +++ b/src/frequenz/gridpool/config/_topology.py @@ -0,0 +1,330 @@ +# License: MIT +# Copyright © 2026 Frequenz Energy-as-a-Service GmbH + +"""Data model for market topology relations. + +A relation links a gridpool, a microgrid and a market location, based on the +Assets API `MarketTopologyRelation`. It names at least two of the three, and its +participations say which market use cases it serves and over which periods. The +config also supports plain validity periods and delivery-area mappings without +a gridpool. + +Ownership of an asset by an enterprise is deliberately left out: it is not a +topology link but an exclusive, time-varying property of the asset, and will be +modelled on the asset itself. +""" + +from dataclasses import field +from datetime import datetime +from typing import ClassVar, Type + +from frequenz.client.assets import ( + MarketLocationIdType, + MarketParticipationType, +) +from marshmallow import Schema +from marshmallow_dataclass import dataclass + +_EIC_ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ-" +_MARKET_AREA_EU_DE = 101 + + +def _require_offset_aware(value: datetime, name: str) -> None: + """Reject a datetime that does not identify an absolute instant.""" + if value.utcoffset() is None: + raise ValueError(f"{name} must include a UTC offset, got {value}") + + +def _require_eic(value: str) -> None: + """Reject a malformed EIC code or an invalid check character.""" + valid_format = ( + len(value) == 16 + and value[-1] != "-" + and all(char in _EIC_ALPHABET for char in value) + ) + if valid_format: + checksum = sum( + (16 - index) * _EIC_ALPHABET.index(char) + for index, char in enumerate(value[:15]) + ) + check_character = _EIC_ALPHABET[36 - ((checksum - 1) % 37)] + if value[-1] == check_character: + return + raise ValueError(f"Delivery area must be a valid EIC code, got {value!r}") + + +@dataclass(frozen=True) +class MarketLocationConfig: + """Configuration of a market location. + + Mirrors the Assets API `MarketLocationRef`: the market area, identifier `id` + and how to read it. `id` repeats the key it is filed under, so the object is + self-describing; `AssetsConfig.check` verifies the two agree. The grid zone + the location sits in is not kept here but on the relations that name it. Raw + IDs must be unique within a document, including across market areas. + """ + + id: str | None = None + """The market location identifier.""" + + type: MarketLocationIdType = MarketLocationIdType.MALO_ID + """Identifier scheme of the market location.""" + + market_area: int = _MARKET_AREA_EU_DE + """Assets API market area, defaulting to EU_DE (`101`).""" + + Schema: ClassVar[Type[Schema]] = Schema + + def __post_init__(self) -> None: + """Check that the identifier scheme and market area are specified. + + Raises: + ValueError: If the identifier scheme or market area is unspecified. + """ + if self.type is MarketLocationIdType.UNSPECIFIED: + raise ValueError("Market location type must be specified") + if self.market_area <= 0: + raise ValueError("Market area must be specified") + + +def _relation_key( + gridpool_id: int | None = None, + microgrid_id: int | None = None, + market_location_id: str | None = None, +) -> str: + """Derive the key a relation is filed under. + + Args: + gridpool_id: Gridpool the relation names. + microgrid_id: Microgrid the relation names. + market_location_id: Market location the relation names. + + Returns: + The key, with the segments of the unset sides omitted. + """ + segments = ( + ("G", gridpool_id), + ("M", microgrid_id), + ("L", market_location_id), + ) + return "".join( + f"{prefix}{value}" for prefix, value in segments if value is not None + ) + + +@dataclass(frozen=True) +class ValidityConfig: + """A period a relation applies over, optionally a market use case. + + A period naming a `participation` is a market participation, which applies + only to a gridpool relation; one without is a plain validity window. Filed + under a free label, which is never read or parsed. The half-open interval + `[start, end)` mirrors the Assets API: the start is inclusive, the end + exclusive, and an unset bound is open. + """ + + participation: MarketParticipationType | None = None + """The market use case, unset for a plain validity window.""" + + start: datetime | None = None + """Inclusive start of the period.""" + + end: datetime | None = None + """Exclusive end of the period.""" + + Schema: ClassVar[Type[Schema]] = Schema + + def __post_init__(self) -> None: + """Check the use case, bounds and order of the period. + + Raises: + ValueError: If the use case is unspecified, a bound has no UTC + offset, or the period ends before it starts. + """ + if self.participation is MarketParticipationType.UNSPECIFIED: + raise ValueError( + "Participation is unspecified; name a use case or leave it unset " + "for a plain period" + ) + if self.start is not None: + _require_offset_aware(self.start, "Period start") + if self.end is not None: + _require_offset_aware(self.end, "Period end") + if self.start is not None and self.end is not None and self.end < self.start: + raise ValueError(f"Period ends {self.end} before it starts {self.start}") + + def covers(self, at: datetime) -> bool: + """Check whether this period applies at an instant. + + Args: + at: The instant to check. + + Returns: + Whether the instant falls within the half-open period. + """ + _require_offset_aware(at, "Instant") + return (self.start is None or self.start <= at) and ( + self.end is None or at < self.end + ) + + def overlaps(self, other: "ValidityConfig") -> bool: + """Check whether two periods share an instant. + + Args: + other: The period to compare with. + + Returns: + Whether the two half-open periods overlap. + """ + return (self.start is None or other.end is None or self.start < other.end) and ( + other.start is None or self.end is None or other.start < self.end + ) + + +@dataclass(frozen=True) +class RelationConfig: + """A relation between a gridpool, a microgrid and a market location. + + It names at least two of the three. The key it is filed under is + `GML` with the segments of the unset + sides dropped. The key clusters the lines of one record and is never read: + every value comes from the fields, and `AssetsConfig.check` verifies that the + two agree. + + A relation naming a gridpool sits in a `delivery_area`; the grid zone rides + on the relation, not on the market location, so a gridpool-to-microgrid + relation with no market location still carries one. A gridpool-free + microgrid-to-market-location relation may also carry one to map that market + location to its delivery area. Delivery areas currently support only EIC + identifiers. + + Its validity lives in `validity`, each entry a period the relation applies + over. A period naming a market use case tells the relation's gridpool + participations apart; one without is a plain window, for a relation that + serves a single use or none, such as a bare microgrid-to-market-location + metering relation. A relation with no periods applies at all times. + """ + + gridpool_id: int | None = None + """Gridpool participating in this relation.""" + + microgrid_id: int | None = None + """Microgrid participating in this relation.""" + + market_location_id: str | None = None + """Market location participating in this relation.""" + + delivery_area: str | None = None + """EIC grid-zone code; required once a gridpool is named.""" + + validity: dict[str, ValidityConfig] = field(default_factory=dict) + """Periods this relation applies over, each optionally a market use case.""" + + Schema: ClassVar[Type[Schema]] = Schema + + def __post_init__(self) -> None: + """Check the periods against the relation. + + A record may be incomplete here, since an override names only the fields + it changes; `AssetsConfig.check` looks at the merged result. + + Raises: + ValueError: If the delivery area is not a valid EIC code, it names a + market use case without a gridpool, or periods of one use case + overlap. + """ + if self.delivery_area is not None: + _require_eic(self.delivery_area) + + by_use: dict[MarketParticipationType | None, list[ValidityConfig]] = {} + for period in self.validity.values(): + if period.participation is not None and self.gridpool_id is None: + raise ValueError( + f"Relation {self.key}: a {period.participation.name} " + "participation applies only to a gridpool relation" + ) + by_use.setdefault(period.participation, []).append(period) + for use, group in by_use.items(): + label = use.name if use is not None else "untyped" + for i, period in enumerate(group): + for other in group[i + 1 :]: + if period.overlaps(other): + raise ValueError( + f"Relation {self.key}: {label} periods " + f"{period.start}..{period.end} and " + f"{other.start}..{other.end} overlap" + ) + + @property + def key(self) -> str: + """The key this relation belongs under, derived from its own fields.""" + return _relation_key( + self.gridpool_id, self.microgrid_id, self.market_location_id + ) + + @property + def is_complete(self) -> bool: + """Whether this relation names at least two of the three sides.""" + sides = (self.gridpool_id, self.microgrid_id, self.market_location_id) + return sum(side is not None for side in sides) >= 2 + + def covers(self, at: datetime) -> bool: + """Check whether this relation applies at an instant. + + Args: + at: The instant to check. + + Returns: + Whether any of its periods covers the instant, or `True` when it + lists none and so applies always. + """ + _require_offset_aware(at, "Instant") + if not self.validity: + return True + return any(period.covers(at) for period in self.validity.values()) + + def matches( + self, + *, + gridpool_id: int | None = None, + microgrid_id: int | None = None, + market_location_id: str | None = None, + delivery_area: str | None = None, + participation: MarketParticipationType | None = None, + at: datetime | None = None, + ) -> bool: + """Check whether this relation has all the given sides. + + Args: + gridpool_id: Gridpool to match, or `None` to ignore. + microgrid_id: Microgrid to match, or `None` to ignore. + market_location_id: Market location to match, or `None` to ignore. + delivery_area: Delivery area to match, or `None` to ignore. + participation: Use case the relation must serve, or `None` to ignore. + at: Instant the relation, or the given participation, must apply at, + or `None` to ignore. + + Returns: + Whether every side given matches this relation. + """ + if at is not None: + _require_offset_aware(at, "Instant") + if gridpool_id is not None and gridpool_id != self.gridpool_id: + return False + if microgrid_id is not None and microgrid_id != self.microgrid_id: + return False + if market_location_id is not None and ( + market_location_id != self.market_location_id + ): + return False + if delivery_area is not None and delivery_area != self.delivery_area: + return False + if participation is not None: + served = [ + p for p in self.validity.values() if p.participation == participation + ] + if not served or (at is not None and not any(p.covers(at) for p in served)): + return False + elif at is not None and not self.covers(at): + return False + return True diff --git a/src/frequenz/gridpool/config/assets.py b/src/frequenz/gridpool/config/assets.py deleted file mode 100644 index 4785f38..0000000 --- a/src/frequenz/gridpool/config/assets.py +++ /dev/null @@ -1,112 +0,0 @@ -# License: MIT -# Copyright © 2026 Frequenz Energy-as-a-Service GmbH - -"""Data model for the `assets` config namespace.""" - -import logging -import tomllib -from dataclasses import field -from pathlib import Path -from typing import Any, ClassVar, Self, Type - -import marshmallow -from marshmallow import Schema -from marshmallow_dataclass import dataclass - -from .microgrid import MicrogridConfig - -_logger = logging.getLogger(__name__) - - -@dataclass -class AssetsConfig: - """Entities described by a config document, keyed by their ID.""" - - microgrids: dict[str, MicrogridConfig] = field(default_factory=dict) - """Microgrids, keyed by microgrid ID.""" - - class Meta: - """Ignore entity tables this version does not know about. - - A reader must keep working against files that already carry entities - added after it, so unknown tables are skipped rather than rejected. - `_warn_unknown_entities` reports them, so a mistyped table is still - visible instead of silently loading as empty. - """ - - unknown = marshmallow.EXCLUDE - - Schema: ClassVar[Type[Schema]] = Schema - - def __post_init__(self) -> None: - """Check that each entry is filed under its own ID. - - Raises: - ValueError: If a key is not a numeric microgrid ID, or does not - match its entry's `meta.microgrid_id`. - """ - for mid, cfg in self.microgrids.items(): - if not mid.isdigit(): - raise ValueError(f"Microgrid ID key must be numeric, got {mid}") - if int(cfg.meta.microgrid_id) != int(mid): - raise ValueError( - f"Microgrid ID mismatch: key {mid} != {cfg.meta.microgrid_id}" - ) - - @classmethod - def _warn_unknown_entities(cls, assets: dict[str, Any], source: Path) -> None: - """Warn about entity tables that this version drops on load.""" - if unknown := sorted(set(assets) - set(cls.Schema().fields)): - _logger.warning( - "%s: ignoring unknown entity tables under `assets`: %s", - source, - ", ".join(unknown), - ) - - @classmethod - def load_from_file(cls, config_path: Path) -> Self: - """Load and validate a config document from a TOML file. - - Entries live under `assets`. A document without an `assets` table is - read in the deprecated layout, where the microgrid entries sit at the - top level. - - Args: - config_path: The path to the TOML configuration file. - - Returns: - The loaded configuration. - - Raises: - TypeError: If `assets` is not a table. - ValueError: If both layouts are present, which means a - half-migrated file rather than a merge. - """ - with config_path.open("rb") as f: - data: dict[str, Any] = tomllib.load(f) - - if "assets" not in data: - _logger.warning( - "%s: top-level microgrid IDs are deprecated, " - "nest the entries under `assets.microgrids` instead.", - config_path, - ) - data = {"assets": {"microgrids": data}} - - assets = data["assets"] - if not isinstance(assets, dict): - raise TypeError( - f"{config_path}: `assets` must be a table, got {type(assets)}" - ) - - if unprefixed := sorted(k for k in data if k != "assets"): - raise ValueError( - f"{config_path}: keys {unprefixed} sit outside `assets` while the " - "file already has an `assets` table; move them under " - "`assets.microgrids`." - ) - - cls._warn_unknown_entities(assets, config_path) - loaded = cls.Schema().load(assets) - assert isinstance(loaded, cls) - return loaded diff --git a/tests/test_config.py b/tests/test_config.py index 6629cea..71be7f4 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -8,6 +8,7 @@ from typing import Any import pytest +from marshmallow import ValidationError from pytest_mock import MockerFixture from frequenz.gridpool import MicrogridConfig @@ -16,7 +17,7 @@ load_configs, load_configs_from_files, ) -from frequenz.gridpool.config.assets import AssetsConfig +from frequenz.gridpool.config._assets import AssetsConfig VALID_CONFIG: dict[str, dict[str, Any]] = { "1": { @@ -243,6 +244,22 @@ async def test_merge_prefixed_base_with_legacy_override(tmp_path: Path) -> None: assert configs["1"].component_type_ids("pv") == [101, 102] +def test_load_configs_from_files_layers_fields(tmp_path: Path) -> None: + """Later files override fields without replacing the microgrid entry.""" + base = _write(tmp_path, "base.toml", _PREFIXED_TOML) + override = _write( + tmp_path, + "override.toml", + "assets.microgrids.1.meta.microgrid_id = 1\n" + 'assets.microgrids.1.meta.name = "Renamed"\n', + ) + + configs = load_configs_from_files([base, override]) + + assert configs["1"].meta.name == "Renamed" + assert configs["1"].component_type_ids("pv") == [101, 102] + + def test_assets_config_rejects_mismatched_id() -> None: """An entry filed under the wrong ID is rejected wherever it is loaded.""" with pytest.raises(ValueError, match="Microgrid ID mismatch"): @@ -251,6 +268,18 @@ def test_assets_config_rejects_mismatched_id() -> None: ) +def test_metadata_delivery_area_removed() -> None: + """Delivery areas live on topology relations, not microgrid metadata.""" + with pytest.raises(ValidationError, match="Unknown field"): + AssetsConfig.Schema().load( + { + "microgrids": { + "23": {"meta": {"microgrid_id": 23, "delivery_area": "area"}} + } + } + ) + + def test_assets_config_warns_on_unknown_entities( tmp_path: Path, caplog: pytest.LogCaptureFixture ) -> None: diff --git a/tests/test_dump_config.py b/tests/test_dump_config.py index edc547b..a01654d 100644 --- a/tests/test_dump_config.py +++ b/tests/test_dump_config.py @@ -7,7 +7,7 @@ from frequenz.gridpool import MicrogridConfig from frequenz.gridpool.cli._dump_config import dump_map -from frequenz.gridpool.config.microgrid import ComponentTypeConfig, Metadata, PVConfig +from frequenz.gridpool.config._microgrid import ComponentTypeConfig, Metadata, PVConfig def test_dump_map_round_trips() -> None: diff --git a/tests/test_load.py b/tests/test_load.py index da71aa2..9d4ad63 100644 --- a/tests/test_load.py +++ b/tests/test_load.py @@ -21,7 +21,7 @@ ComponentGraphGenerator, MicrogridComponentGraph, ) -from frequenz.gridpool.config.load import ( +from frequenz.gridpool.config._load import ( _derive_component_configs, load_configs, load_configs_from_api, diff --git a/tests/test_patch_config.py b/tests/test_patch_config.py index 36d2d2d..04b1370 100644 --- a/tests/test_patch_config.py +++ b/tests/test_patch_config.py @@ -4,7 +4,7 @@ """Tests for in-place patching of existing dotted-key TOML config files.""" from frequenz.gridpool.cli._patch_config import patch_text -from frequenz.gridpool.config.microgrid import ( +from frequenz.gridpool.config._microgrid import ( ComponentTypeConfig, Metadata, MicrogridConfig, diff --git a/tests/test_topology.py b/tests/test_topology.py new file mode 100644 index 0000000..6fcda60 --- /dev/null +++ b/tests/test_topology.py @@ -0,0 +1,671 @@ +# License: MIT +# Copyright © 2026 Frequenz Energy-as-a-Service GmbH + +"""Tests for the market topology relations.""" + +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +import pytest +from frequenz.client.assets import ( + MarketLocationIdType, + MarketParticipationType, +) + +from frequenz.gridpool.config import ( + AssetsConfig, + RelationConfig, + ValidityConfig, + load_assets_from_files, +) +from frequenz.gridpool.config._topology import _relation_key + +_AREA_A = "10YDE-RWENET---I" +_AREA_B = "10YDE-EON------1" + +_TOML = """ +assets.relations.G80M241L10208446344.gridpool_id = 80 +assets.relations.G80M241L10208446344.microgrid_id = 241 +assets.relations.G80M241L10208446344.market_location_id = "10208446344" +assets.relations.G80M241L10208446344.delivery_area = "10YDE-RWENET---I" + +assets.relations.G80L51171875559.gridpool_id = 80 +assets.relations.G80L51171875559.market_location_id = "51171875559" +assets.relations.G80L51171875559.delivery_area = "10YDE-EON------1" + +assets.relations.M217L33333333333.microgrid_id = 217 +assets.relations.M217L33333333333.market_location_id = "33333333333" +""" + + +def _dt(year: int, month: int, day: int) -> datetime: + """Build a UTC instant at midnight.""" + return datetime(year, month, day, tzinfo=timezone.utc) + + +def _write(tmp_path: Path, name: str, content: str) -> Path: + """Write a config file and return its path.""" + path = tmp_path / name + path.write_text(content) + return path + + +def _load(**tables: Any) -> AssetsConfig: + """Load a document straight from its tables and check it as a whole.""" + loaded = AssetsConfig.Schema().load(tables) + assert isinstance(loaded, AssetsConfig) + loaded.check() + return loaded + + +def _relations(*records: dict[str, Any]) -> dict[str, Any]: + """Build a relation table, each record filed under its own key.""" + return { + "relations": { + _relation_key( + record.get("gridpool_id"), + record.get("microgrid_id"), + record.get("market_location_id"), + ): record + for record in records + } + } + + +def test_relation_key() -> None: + """The key names the sides a relation connects, in a fixed order.""" + assert _relation_key(80, 241, "10208446344") == "G80M241L10208446344" + assert _relation_key(80, market_location_id="5117") == "G80L5117" + assert _relation_key(microgrid_id=217, market_location_id="333") == "M217L333" + + +def test_load_relations(tmp_path: Path) -> None: + """A record states its own sides, and its key clusters its lines.""" + config = AssetsConfig.load_from_file(_write(tmp_path, "relations.toml", _TOML)) + + assert len(config.relations) == 3 + + relation = config.relations["G80M241L10208446344"] + assert relation.gridpool_id == 80 + assert relation.microgrid_id == 241 + assert relation.market_location_id == "10208446344" + assert relation.delivery_area == "10YDE-RWENET---I" + + +def test_find_relations(tmp_path: Path) -> None: + """Relations are searchable by any combination of their sides.""" + config = AssetsConfig.load_from_file(_write(tmp_path, "relations.toml", _TOML)) + + assert [r.market_location_id for r in config.find_relations(gridpool_id=80)] == [ + "10208446344", + "51171875559", + ] + assert config.find_relations(microgrid_id=241) == [ + config.relations["G80M241L10208446344"] + ] + assert not config.find_relations(gridpool_id=68) + + +def test_at_least_two_sides_required() -> None: + """A relation names at least two of gridpool, microgrid, market location.""" + with pytest.raises(ValueError, match="at least two"): + _load(relations={"G80": {"gridpool_id": 80}}) + + with pytest.raises(ValueError, match="at least two"): + _load(relations={"M241": {"microgrid_id": 241}}) + + +def test_metering_relation_maps_delivery_area_without_gridpool() -> None: + """A metering relation can map a market location without a gridpool.""" + config = _load( + **_relations( + { + "microgrid_id": 217, + "market_location_id": "33333333333", + "delivery_area": _AREA_A, + } + ) + ) + + relation = config.relations["M217L33333333333"] + assert relation.gridpool_id is None + assert relation.delivery_area == _AREA_A + assert relation.is_complete + assert relation.covers(_dt(2026, 3, 1)) + + +def test_nothing_is_read_from_the_key() -> None: + """The key clusters the lines of a record; the values come from its fields.""" + config = AssetsConfig.Schema().load( + {"relations": {"G80M241L10208446344": {"gridpool_id": 80}}} + ) + assert isinstance(config, AssetsConfig) + + relation = config.relations["G80M241L10208446344"] + assert relation.microgrid_id is None + assert relation.market_location_id is None + + with pytest.raises(ValueError, match="at least two"): + config.check() + + +def test_key_must_agree_with_the_fields() -> None: + """A key that says something else than its record is an error.""" + with pytest.raises(ValueError, match="Relation key mismatch"): + _load(relations={"G80M241": {"gridpool_id": 80, "microgrid_id": 242}}) + + +def test_gridpool_relation_needs_a_delivery_area() -> None: + """A relation naming a gridpool must place it in a delivery area.""" + with pytest.raises(ValueError, match="must name a delivery area"): + _load(**_relations({"gridpool_id": 80, "microgrid_id": 241})) + + +@pytest.mark.parametrize( + "delivery_area", + ["10YDE-RWENET---X", "10YDE-RWENET--I", "10yDE-RWENET---I"], +) +def test_delivery_area_must_be_a_valid_eic(delivery_area: str) -> None: + """Delivery areas have valid EIC syntax and a matching check character.""" + with pytest.raises(ValueError, match="valid EIC code"): + _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": delivery_area, + } + ) + ) + + +def test_microgrid_carries_a_delivery_area_without_a_market_location() -> None: + """A gridpool-to-microgrid relation carries a delivery area of its own.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + } + ) + ) + + relation = config.relations["G80M241"] + assert relation.market_location_id is None + assert relation.delivery_area == "10YDE-RWENET---I" + + +def test_market_location_in_two_delivery_areas_rejected() -> None: + """One market location cannot sit in two delivery areas across relations.""" + with pytest.raises(ValueError, match="two delivery areas"): + _load( + **_relations( + { + "gridpool_id": 80, + "market_location_id": "77777777777", + "delivery_area": "10YDE-RWENET---I", + }, + { + "gridpool_id": 81, + "market_location_id": "77777777777", + "delivery_area": "10YDE-EON------1", + }, + ) + ) + + +def test_microgrid_in_two_gridpools_by_participation() -> None: + """One microgrid can take part in two gridpools, told apart by use case.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + "validity": {"a": {"participation": "ENERGY_TRADING"}}, + }, + { + "gridpool_id": 46, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + "validity": {"a": {"participation": "FLEX_MARKETS"}}, + }, + ) + ) + + trading = config.find_relations( + microgrid_id=241, participation=MarketParticipationType.ENERGY_TRADING + ) + flex = config.find_relations( + microgrid_id=241, participation=MarketParticipationType.FLEX_MARKETS + ) + assert [r.gridpool_id for r in trading] == [80] + assert [r.gridpool_id for r in flex] == [46] + assert len(config.find_relations(microgrid_id=241)) == 2 + + +def test_legacy_gridpool_id_must_match_relations() -> None: + """The legacy scalar gridpool ID must describe all current relations.""" + microgrids = {"241": {"meta": {"microgrid_id": 241, "gid": 80}}} + + config = _load( + microgrids=microgrids, + **_relations( + {"gridpool_id": 80, "microgrid_id": 241, "delivery_area": _AREA_A} + ), + ) + assert config.microgrids["241"].meta.gid == 80 + + _load( + microgrids={"241": {"meta": {"microgrid_id": 241}}}, + **_relations( + {"gridpool_id": 80, "microgrid_id": 241, "delivery_area": _AREA_A}, + {"gridpool_id": 46, "microgrid_id": 241, "delivery_area": _AREA_A}, + ), + ) + + with pytest.raises(ValueError, match="legacy meta.gid"): + _load( + microgrids=microgrids, + **_relations( + { + "gridpool_id": 46, + "microgrid_id": 241, + "delivery_area": _AREA_A, + } + ), + ) + + with pytest.raises(ValueError, match="remove meta.gid"): + _load( + microgrids=microgrids, + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": _AREA_A, + }, + { + "gridpool_id": 46, + "microgrid_id": 241, + "delivery_area": _AREA_A, + }, + ), + ) + + +def test_two_use_cases_on_one_relation_may_overlap() -> None: + """Different use cases can run at once on the same relation.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + "validity": { + "trading": {"participation": "ENERGY_TRADING"}, + "flex": {"participation": "FLEX_MARKETS"}, + }, + } + ) + ) + + relation = config.relations["G80M241"] + assert relation.matches(participation=MarketParticipationType.ENERGY_TRADING) + assert relation.matches(participation=MarketParticipationType.FLEX_MARKETS) + + +def test_one_use_case_cannot_overlap_itself() -> None: + """The same use case cannot apply twice at the same instant.""" + with pytest.raises(ValueError, match="ENERGY_TRADING periods"): + _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + "validity": { + "first": { + "participation": "ENERGY_TRADING", + "start": _dt(2026, 1, 15), + "end": _dt(2026, 6, 30), + }, + "second": { + "participation": "ENERGY_TRADING", + "start": _dt(2026, 6, 1), + }, + }, + } + ) + ) + + +def test_market_participation_needs_a_gridpool() -> None: + """A period naming a market use case applies only in a gridpool context.""" + with pytest.raises(ValueError, match="applies only to a gridpool"): + _load( + **_relations( + { + "microgrid_id": 241, + "market_location_id": "10208446344", + "validity": {"a": {"participation": "ENERGY_TRADING"}}, + } + ) + ) + + +def test_untyped_period_needs_no_gridpool() -> None: + """A period without a use case is a plain validity window, gridpool or not.""" + config = _load( + **_relations( + { + "microgrid_id": 217, + "market_location_id": "33333333333", + "validity": { + "a": { + "start": _dt(2026, 1, 15), + "end": _dt(2026, 6, 30), + } + }, + } + ) + ) + + relation = config.relations["M217L33333333333"] + assert relation.gridpool_id is None + assert relation.covers(_dt(2026, 3, 1)) + assert not relation.covers(_dt(2026, 6, 30)) # end is exclusive + assert not relation.covers(_dt(2026, 7, 1)) + assert not relation.matches(participation=MarketParticipationType.ENERGY_TRADING) + + +def test_untyped_periods_cannot_overlap() -> None: + """Two plain periods of one relation cannot cover the same instant.""" + with pytest.raises(ValueError, match="untyped periods"): + _load( + **_relations( + { + "microgrid_id": 217, + "market_location_id": "33333333333", + "validity": { + "first": { + "start": _dt(2026, 1, 15), + "end": _dt(2026, 6, 30), + }, + "second": {"start": _dt(2026, 6, 1)}, + }, + } + ) + ) + + +def test_gridpool_relation_without_participations_always_applies() -> None: + """A gridpool relation naming no use case is still a relation, applying always.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + } + ) + ) + + relation = config.relations["G80M241"] + assert relation.covers(_dt(2026, 3, 1)) + assert not relation.matches(participation=MarketParticipationType.ENERGY_TRADING) + + +def test_participation_period() -> None: + """A relation applies at an instant when a participation covers it.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + "validity": { + "a": { + "participation": "ENERGY_TRADING", + "start": _dt(2026, 1, 15), + "end": _dt(2026, 6, 30), + } + }, + } + ) + ) + + relation = config.relations["G80M241"] + assert relation.covers(_dt(2026, 3, 1)) + assert not relation.covers(_dt(2026, 7, 1)) + assert config.find_relations(microgrid_id=241, at=_dt(2026, 3, 1)) + assert not config.find_relations(microgrid_id=241, at=_dt(2026, 7, 1)) + + +def test_inverted_period_rejected() -> None: + """A participation cannot end before it starts.""" + with pytest.raises(ValueError, match="before it starts"): + _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "delivery_area": "10YDE-RWENET---I", + "validity": { + "a": { + "participation": "ENERGY_TRADING", + "start": _dt(2026, 6, 30), + "end": _dt(2026, 1, 15), + } + }, + } + ) + ) + + +def test_period_and_query_require_a_utc_offset() -> None: + """Validity bounds and query instants identify absolute instants.""" + naive = datetime(2026, 1, 1) # noqa: DTZ001 + + with pytest.raises(ValueError, match="UTC offset"): + ValidityConfig(start=naive) + + relation = RelationConfig(microgrid_id=217, market_location_id="33333333333") + with pytest.raises(ValueError, match="UTC offset"): + relation.covers(naive) + + +def test_market_location_properties() -> None: + """A location identifies its market and how to read its identifier.""" + config = _load( + market_locations={ + "51171875559": { + "id": "51171875559", + "type": "ZAEHLPUNKT", + "market_area": 109, + }, + "10208446344": {"id": "10208446344"}, + } + ) + + assert config.market_locations["51171875559"].id == "51171875559" + assert ( + config.market_locations["51171875559"].type is MarketLocationIdType.ZAEHLPUNKT + ) + assert config.market_locations["51171875559"].market_area == 109 + assert config.market_locations["10208446344"].type is MarketLocationIdType.MALO_ID + assert config.market_locations["10208446344"].market_area == 101 + + with pytest.raises(Exception, match="Must be one of"): + _load(market_locations={"10208446344": {"id": "10208446344", "type": "MALO"}}) + + with pytest.raises(ValueError, match="Market area must be specified"): + _load(market_locations={"10208446344": {"id": "10208446344", "market_area": 0}}) + + +def test_market_location_id_must_agree_with_the_key() -> None: + """A market location's id must match the key it is filed under.""" + with pytest.raises(ValueError, match="Market location key mismatch"): + _load(market_locations={"10208446344": {"id": "99999999999"}}) + + +def test_merge_files_into_one_document(tmp_path: Path) -> None: + """The relations of a gridpool can be spread over several files.""" + first = _write(tmp_path, "first.toml", _TOML) + second = _write( + tmp_path, + "second.toml", + "assets.relations.G80L44444444444.gridpool_id = 80\n" + 'assets.relations.G80L44444444444.market_location_id = "44444444444"\n' + 'assets.relations.G80L44444444444.delivery_area = "10YDE-EON------1"\n', + ) + + assert len(load_assets_from_files([first, second]).relations) == 4 + + +def test_override_keeps_untouched_fields(tmp_path: Path) -> None: + """Overriding one field of a relation leaves the rest as they were.""" + base = _write( + tmp_path, + "base.toml", + "assets.relations.G80L51171875559.gridpool_id = 80\n" + 'assets.relations.G80L51171875559.market_location_id = "51171875559"\n' + 'assets.relations.G80L51171875559.delivery_area = "10YDE-EON------1"\n', + ) + override = _write( + tmp_path, + "override.toml", + 'assets.relations.G80L51171875559.delivery_area = "10YDE-RWENET---I"\n', + ) + + merged = load_assets_from_files([base, override]) + relation = merged.relations["G80L51171875559"] + + assert relation.gridpool_id == 80 + assert relation.market_location_id == "51171875559" + assert relation.delivery_area == "10YDE-RWENET---I" + + +def test_override_completes_only_after_the_merge(tmp_path: Path) -> None: + """A file that changes one field is incomplete until it is layered.""" + base = _write(tmp_path, "base.toml", _TOML) + override = _write( + tmp_path, + "override.toml", + "assets.relations.M217L33333333333.microgrid_id = 217\n", + ) + + with pytest.raises(ValueError, match="at least two"): + AssetsConfig.load_from_file(override) + + relation = load_assets_from_files([base, override]).relations["M217L33333333333"] + assert relation == RelationConfig( + microgrid_id=217, market_location_id="33333333333" + ) + + +def test_delivery_areas_of_a_gridpool() -> None: + """A gridpool's delivery areas come from the relations that name it.""" + config = _load( + **_relations( + {"gridpool_id": 80, "microgrid_id": 241, "delivery_area": _AREA_A}, + {"gridpool_id": 80, "market_location_id": "511", "delivery_area": _AREA_B}, + {"gridpool_id": 80, "microgrid_id": 300, "delivery_area": _AREA_A}, + ) + ) + + assert config.find_delivery_areas(gridpool_id=80) == [_AREA_A, _AREA_B] + assert config.find_delivery_areas(microgrid_id=241) == [_AREA_A] + assert config.find_delivery_areas(gridpool_id=68) == [] + + +def test_delivery_areas_honour_the_instant() -> None: + """A relation that has lapsed drops out of its gridpool's delivery areas.""" + config = _load( + **_relations( + {"gridpool_id": 80, "microgrid_id": 241, "delivery_area": _AREA_A}, + { + "gridpool_id": 80, + "microgrid_id": 300, + "delivery_area": _AREA_B, + "validity": {"w": {"start": _dt(2026, 1, 1), "end": _dt(2026, 6, 1)}}, + }, + ) + ) + + assert config.find_delivery_areas(gridpool_id=80, at=_dt(2026, 3, 1)) == [ + _AREA_A, + _AREA_B, + ] + assert config.find_delivery_areas(gridpool_id=80, at=_dt(2026, 9, 1)) == [_AREA_A] + + +def test_market_locations_of_a_microgrid_or_gridpool() -> None: + """Market locations are searchable by microgrid or by gridpool.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "market_location_id": "111", + "delivery_area": _AREA_A, + }, + {"microgrid_id": 241, "market_location_id": "222"}, + { + "gridpool_id": 46, + "microgrid_id": 300, + "market_location_id": "333", + "delivery_area": _AREA_A, + }, + ) + ) + + assert config.find_market_locations(microgrid_id=241) == ["111", "222"] + assert config.find_market_locations(gridpool_id=80) == ["111"] + assert config.find_market_locations(microgrid_id=999) == [] + + +def test_find_by_gridpool_and_delivery_area() -> None: + """Microgrids and market locations narrow to a gridpool's delivery area.""" + config = _load( + **_relations( + {"gridpool_id": 80, "microgrid_id": 241, "delivery_area": _AREA_A}, + { + "gridpool_id": 80, + "microgrid_id": 300, + "market_location_id": "111", + "delivery_area": _AREA_A, + }, + {"gridpool_id": 80, "market_location_id": "222", "delivery_area": _AREA_A}, + {"gridpool_id": 80, "market_location_id": "333", "delivery_area": _AREA_B}, + ) + ) + + assert config.find_microgrids(gridpool_id=80, delivery_area=_AREA_A) == [241, 300] + assert config.find_market_locations(gridpool_id=80, delivery_area=_AREA_A) == [ + "111", + "222", + ] + assert config.find_market_locations(gridpool_id=80, delivery_area=_AREA_B) == [ + "333" + ] + + +def test_find_by_market_location() -> None: + """A market location resolves to its microgrid and its delivery area.""" + config = _load( + **_relations( + { + "gridpool_id": 80, + "microgrid_id": 241, + "market_location_id": "111", + "delivery_area": _AREA_A, + } + ) + ) + + assert config.find_microgrids(market_location_id="111") == [241] + assert config.find_delivery_areas(market_location_id="111") == [_AREA_A] + assert config.find_microgrids(market_location_id="999") == []