Add harmonised status column

docs/basics.md

@@ -12,6 +12,7 @@ The dataset combines the data of all the data sources listed in [Data-Sources](#
 - **Fueltype** - {Solid Biomass, Biogas, Geothermal, Hard Coal, Hydro, Lignite, Nuclear, Natural Gas, Oil, Solar, Wind, Other}
 - **Technology** - {CCGT, OCGT, Steam Turbine, Combustion Engine, Run-Of-River, Pumped Storage, Reservoir}
 - **Set** - {Power Plant (PP), Combined Heat and Power (CHP), Storages (Stores)}
+- **Status** - life-cycle stage {announced, construction, operating, mothballed, retired}. If the data sources do not agree, the latest stage is used.
 - **Capacity** - [MW]
 - **Duration** - Maximum state of charge capacity in terms of hours at full output capacity
 - **Dam Information** - Dam volume [Mm^3] and Dam Height [m]

docs/custom-config.md

@@ -16,6 +16,7 @@ and change the `.powerplantmaching_config.yaml` file according to your wishes. T
 
 - determine the global set of **countries** and **fueltypes**
 - determine which data sources to combine and which data sources should completely be contained in the final dataset
+- select the life-cycle stages of the power plants with the key `status`, for all data sources and optionally per data source (see `target_status` for the stages and the mapping of the status labels of each data source)
 - individually filter data sources via [`pandas.DataFrame.query`](http://pandas.pydata.org/pandas-docs/stable/indexing.html#the-query-method) statements set as an argument of data source name. See the default [config.yaml file](https://github.com/PyPSA/powerplantmatching/blob/master/powerplantmatching/package_data/config.yaml) as an example
 
 Optionally you can:

docs/release-notes.md

@@ -8,7 +8,16 @@ SPDX-License-Identifier: MIT
 
 ## Upcoming Version
 
-*
+* New column `Status` with the life-cycle stage of the power plant: `announced`, `construction`, `operating`, `mothballed` or `retired`. ([#322](https://github.com/PyPSA/powerplantmatching/issues/322))
+    * **Important:** the output now also includes announced, mothballed and retired power plants. Announced projects add several hundred GW, and most of them have no `DateIn`. Filter with the `Status` column, e.g. `df.query("Status in ['operating', 'construction']")`; a filter on `DateIn` and `DateOut` alone is not sufficient.
+    * The status labels of the data sources are mapped to the stages in the new config section `target_status`. The new key `status` selects the stages to keep for all data sources; by default, all stages are kept. The key `status` of a data source overrides it and now selects stages and not the status labels of the data source.
+    * Data sources without status data list existing power plants and get the status `operating` (ENTSOE, GPD, JRC, OSM, GHR and OPSD outside Germany). GEO gets the status `retired` if `DateOut` is before the current year. The status of OPSD for Germany is not used, as it is outdated.
+    * If matched data sources do not agree, the latest stage is used. Units with different stages are not aggregated into one power plant.
+    * Announced projects and projects under construction are not matched with other data sources, because they often have the name and site of existing power plants. They are added without matching from the fully included data sources.
+    * BEYONDCOAL is no longer a fully included data source, as GEM includes the same coal units with a higher reliability. This prevents double counting of units with a different status in the two data sources. Known limitation: if two matched data sources group the units of a power plant differently by status, a unit can still be counted twice (e.g. the retired unit 9 of Walsum in Germany, from MASTR and GEM).
+    * The `DateIn` and `DateOut` values that were estimated from the status are removed (mothballed plants in the GEM coal and oil and gas trackers, temporarily shut down units in MASTR, nuclear plants under construction in WIKIPEDIA).
+* Bugfix in `aggregate_units()`: nuclear units are not aggregated within a data source, as set by `fueltypes_with_blocks` in the config section `clean_name`. Before, a wrong config key disabled this setting.
+* Bugfix in `pm.data.GHR()`: read capacity, coordinates and technology from the correct columns, and read the commissioning year without conversion to a date (all values were 1970 before).
 
 ## [v0.9.0](https://github.com/PyPSA/powerplantmatching/releases/tag/v0.9.0) (3rd October 2026)
 

powerplantmatching/cleaning.py

@@ -34,6 +34,7 @@ def mode(x):
     "Fueltype": mode,
     "Technology": mode,
     "Set": mode,
+    "Status": mode,
     "Country": mode,
     "Capacity": "sum",
     "lat": "mean",
@@ -345,6 +346,29 @@ def gather_set_info(df, search_col=["Name", "Fueltype", "Technology"], config=No
     return df.assign(Set=Set)
 
 
+def map_status(df, config=None):
+    """
+    Map the raw labels in column "Status" to the life-cycle stages given by
+    the `config` under the section `target_status`. Entries with other labels
+    (e.g. cancelled) are removed.
+
+    Parameters
+    ----------
+    df : pandas.DataFrame
+        DataFrame with raw labels in column "Status".
+    config : dict, default None
+        Custom configuration, defaults to
+        `powerplantmatching.config.get_config()`.
+    """
+    if config is None:
+        config = get_config()
+
+    stages = config["target_status"]
+    labels = {label: stage for stage, labels in stages.items() for label in labels}
+    status = df.Status.str.lower().map(labels)
+    return df.assign(Status=status)[status.notna()]
+
+
 # @deprecated(
 #     deprecated_in="0.5",
 # )
@@ -476,7 +500,9 @@ def aggregate_units(
 
     cols = config["target_columns"]
     weighted_cols = list({"Efficiency", "Duration"} & set(cols))
-    str_cols = list({"Name", "Country", "Fueltype", "Technology", "Set"} & set(cols))
+    str_cols = list(
+        {"Name", "Country", "Fueltype", "Technology", "Set", "Status"} & set(cols)
+    )
     props_for_groups = {k: v for k, v in AGGREGATION_FUNCTIONS.items() if k in cols}
 
     df = df.assign(
@@ -497,8 +523,8 @@ def aggregate_units(
                 break
 
     block_query = None
-    if with_blocks := config["clean_name"].get("fuel_type_with_blocks", []):  # noqa
-        block_query = "Fueltype in @with_blocks"
+    if with_blocks := config["clean_name"].get("fueltypes_with_blocks", []):  # noqa
+        block_query = "Fueltype not in @with_blocks"
 
     if country_wise:
         countries = df.Country.unique()
@@ -512,7 +538,9 @@ def aggregate_units(
         duplicates = duke(df.query(query) if query else df, threads=threads)
 
     df = cliques(df, duplicates)
-    df = df.groupby("grouped").agg(props_for_groups)
+    # do not merge units at different life-cycle stages
+    by = ["grouped", "Status"] if "Status" in cols else ["grouped"]
+    df = df.groupby([df[c] for c in by]).agg(props_for_groups)
 
     no_downcast_ctx = (
         contextlib.nullcontext()

powerplantmatching/collection.py

@@ -243,14 +243,12 @@ def powerplants(
         else:
             matched = matched[matched.lat.notnull()]
 
+    # keep units of one power plant at different life-cycle stages
+    subset = [c for c in ["Name", "Fueltype", "Country", "Status"] if c in matched]
     if isinstance(matched.columns, pd.MultiIndex):
-        matched = (
-            matched.stack(future_stack=True)
-            .drop_duplicates(["Name", "Fueltype", "Country"])
-            .unstack(-1)
-        )
+        matched = matched.stack(future_stack=True).drop_duplicates(subset).unstack(-1)
     else:
-        matched = matched.drop_duplicates(["Name", "Fueltype", "Country"])
+        matched = matched.drop_duplicates(subset)
 
     matched.reset_index(drop=True).to_csv(fn, index_label="id", encoding="utf-8")