Skip to content

Bugfix/materials myopic#277

Closed
idelder wants to merge 43 commits intoTemoaProject:unstablefrom
idelder:bugfix/materials_myopic
Closed

Bugfix/materials myopic#277
idelder wants to merge 43 commits intoTemoaProject:unstablefrom
idelder:bugfix/materials_myopic

Conversation

@idelder
Copy link

@idelder idelder commented Mar 12, 2026

Materials wasn't playing well with myopic. This fixes some tricky issues around checking viable data for myopic materials databases.

  1. Update construction_input and emission_embodied to check for only future vintages
  2. Update end_of_life_output and emission_end_of_life to properly filter by valid retirement/eol r, t, v tuples
  3. In the network diagram, for material flows, the technology is now both the edge and the node (input for end of life out, output for construction in)
  4. In output flow tables, the inputs to end of life out flows is now NULL, vice versa for outputs from construction inputs, to avoid foreign key constraint violations.
  5. Update various logging to better handle all this
  6. Update materials test db so that it runs (though fails) in myopic mode. Could add a myopic version of this test.

Summary by CodeRabbit

  • New Features

    • Myopic/base-year filtering added for construction inputs and embodied emissions; new end-of-life viability dimension and related model sets.
    • Capacity indexing extended to include vintage-aware capacity tracking and active-capacity sets; technologies used during construction and end-of-life are recorded.
  • Bug Fixes

    • Flow validation now skips missing/non-applicable flow identifiers.
    • Small-value pruning made sign-aware.
  • Chores

    • Tests, test data, and SQL schema/tutorial assets updated for dual-mode (perfect foresight/myopic) and tightened referential integrity.

@coderabbitai
Copy link

coderabbitai bot commented Mar 12, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Replace commodity-string markers for construction/end-of-life flows with None, add an end-of-life viability filter and myopic custom loaders, change small-value pruning to sign-aware checks, aggregate retirements and active-capacity indexing by (region,technology,vintage), extend network EOL/construction edges with lifetime-aware logic, update DB FKs and nullability, and extend tests to run two scenario modes.

Changes

Cohort / File(s) Summary
Flow identifier / writer
temoa/_internal/table_data_puller.py, temoa/_internal/table_writer.py
Replace commodity-string markers for construction/end-of-life with None in FI construction; flow validation now skips entries where fi.i or fi.o is None; small-value pruning changed from abs(...) < eps to sign-aware val < eps checks.
Capacity logic & public indices
temoa/components/capacity.py, temoa/components/technology.py, temoa/core/model.py
Retirement and capacity sets re-indexed to (region, technology, vintage); deduplicate (r,t,v) triplets, gate retirements by is_p0_eol/is_living and survival rules, add model.used_techs updates for construction/EOL links, introduce active_capacity_available_rpt* sets, and minor logging text change.
Data loading / myopic filters
temoa/data_io/component_manifest.py, temoa/data_io/hybrid_loader.py
Add _load_construction_input and _load_emission_embodied myopic-aware loaders; introduce viable_rtv_eol filter field and wire custom loader names/validators in manifest entries.
Network building & viability
temoa/model_checking/commodity_network_manager.py, temoa/model_checking/network_model_data.py
Add rtv_eol viable set; expand eol lookup to include lifetime (key (r,t,v,lifetime)); track living_rtv; generate construction/EOL EdgeTuples with actual tech and lifetime; skip uncap techs and unresolved vintages during construction.
DB schema & tutorial assets
temoa/db_schema/temoa_schema_v4.sql, temoa/tutorial_assets/utopia.sql
Add/adjust foreign-key references for commodity/technology/time_period columns in myopic_efficiency and output_flow_out_summary; change output_curtailment.season FK target; relax several NOT NULL constraints.
Tests & test data
tests/test_material_results.py, tests/testing_data/materials.sql, tests/testing_data/mediumville.sql, tests/testing_data/mediumville_sets.json, tests/test_network_model_data.py
Extend tests to run both perfect_foresight and myopic modes and pass mode through fixtures; add sector rows; remove an existing_capacity test row; clear tech_exist; add mock branch for technology WHERE unlim_cap==1 queries.
Schema wiring & manifest updates
temoa/_internal/*, temoa/data_io/*, temoa/model_checking/*, temoa/data_io/component_manifest.py
Register new viable filter rtv_eol in filters, wire custom loaders/validators, and propagate small-value pruning semantics change in table data puller.

Sequence Diagram(s)

sequenceDiagram
    participant Loader as HybridLoader
    participant Model as TemoaModel
    participant Network as CommodityNetworkManager
    participant DB as Database
    participant Validator as check_existing_capacity

    Loader->>Model: _load_construction_input / _load_emission_embodied (apply myopic filter)
    Loader->>Model: populate model.construction_input / model.emission_embodied
    Model->>Validator: run check_existing_capacity(model)
    Validator-->>Model: pass / raise
    Model->>Network: build_filters() (includes rtv_eol)
    Network->>DB: query lookup rows (periods, eol with lifetime)
    DB-->>Network: return lookup rows
    Network->>Model: emit EdgeTuples (construction/EOL edges with actual tech + lifetime)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Suggested labels

Maintenance, database-schema

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Bugfix/materials myopic' directly relates to the primary change: fixing interactions between the materials module and myopic mode.
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@idelder
Copy link
Author

idelder commented Mar 12, 2026

Waiting for Dijuan to stress test this a bit before clearing. Very finicky logic.

@idelder idelder marked this pull request as ready for review March 14, 2026 02:06
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/_internal/table_data_puller.py`:
- Line 193: The code constructs FI(...) with a None commodity sentinel but
TableWriter._insert_summary_flow_results writes fi.i and fi.o directly into
output_flow_out_summary whose input_comm/output_comm columns are declared NOT
NULL, causing end-of-life summary inserts to fail; either filter out FI
instances with a None commodity before calling _insert_summary_flow_results (or
skip writing rows where fi.i or fi.o is None) or change the summary
schema/column definitions to accept NULLs and propagate nullable types through
the writer; locate the FI construction site and the
TableWriter._insert_summary_flow_results method and implement one of these fixes
so fi.i/fi.o are never passed as non-nullable values to output_flow_out_summary.

In `@temoa/components/capacity.py`:
- Around line 607-610: Replace the unnecessary set(...) wrapping a generator
when building unique_rtv with a set comprehension; specifically, change the
expression that creates unique_rtv (which currently uses set((r, t, v) for r,
_i, t, v, _o in model.efficiency.sparse_iterkeys()) |
set(model.existing_capacity.sparse_iterkeys())) to use a single set
comprehension for the first part (e.g., {(r, t, v) for r, _i, t, v, _o in
model.efficiency.sparse_iterkeys()}) and keep the union with
set(model.existing_capacity.sparse_iterkeys()) so unique_rtv is created
idiomatically and without the unnecessary generator.

In `@temoa/components/technology.py`:
- Around line 441-464: The check in check_existing_capacity wrongly rejects
retirement-only existing_capacity tuples because it requires membership in
model.active_capacity_available_rptv (derived from process_vintages/efficiency);
change the validation so that for each (r,t,v) you first skip the
active_capacity check when the vintage is only present for retirement/EOL
bookkeeping (i.e., not part of the process network), or alternatively validate
against the retirement/EOL bookkeeping sets (the structures populated by
populate_core_dictionaries for retirement entries) instead of
active_capacity_available_rptv; update the loop that inspects
model.time_optimize and the membership test (r, p, t, v) not in
model.active_capacity_available_rptv to only apply when (r,t,v) participates in
the process network (or to check the retirement bookkeeping set) while keeping
the existing tech_uncap and lifetime_process checks.

In `@temoa/db_schema/temoa_schema_v4.sql`:
- Line 711: The schema change to output_curtailment (altered FK to
season_label(season)) requires updating the schema version or adding a
migration: either increment DB_MINOR in temoa_schema_v4.sql and update any
version constants, or add an explicit migration that alters the existing output
DBs to drop the old FK and recreate the new FK; ensure
TableWriter.clear_scenario() behavior is considered so existing output DBs are
migrated (or recreated) at runtime, and reference the output_curtailment table
name, the DB_MINOR constant, and the TableWriter.clear_scenario() call site when
implementing the change.

In `@temoa/model_checking/commodity_network_manager.py`:
- Around line 129-131: rtv_eol is being populated with input commodities instead
of technologies; change the additions to use edge_tuple.tech (not
edge_tuple.input_comm) and only add tuples for edges that are actual
retirement/EOL edges (i.e., where the edge represents an end-of-life/retirement
output), so replace the current unconditional adds to valid_elements['rtv_eol']
with logic that: checks the edge is a retirement/EOL edge and then adds
(edge_tuple.region, edge_tuple.tech, edge_tuple.vintage). Apply the same fix to
the other occurrence referenced around lines 163-167.

In `@tests/test_material_results.py`:
- Line 89: The unpacked variable `mode` from the `solved_connection` fixture is
unused; change the unpacking in the test from "con, name, tech, period,
flow_target, mode = solved_connection" to prefix the unused variable with an
underscore (e.g., "con, name, tech, period, flow_target, _mode =
solved_connection") so linters (Ruff RUF059) know it's intentionally unused;
keep all other uses of the other variables unchanged and only rename the
unpacked symbol in the test.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5de40ea2-4aeb-4a2b-a662-d438a2817791

📥 Commits

Reviewing files that changed from the base of the PR and between 7f7ff86 and 5619478.

📒 Files selected for processing (14)
  • temoa/_internal/table_data_puller.py
  • temoa/_internal/table_writer.py
  • temoa/components/capacity.py
  • temoa/components/technology.py
  • temoa/core/model.py
  • temoa/data_io/component_manifest.py
  • temoa/data_io/hybrid_loader.py
  • temoa/db_schema/temoa_schema_v4.sql
  • temoa/model_checking/commodity_network_manager.py
  • temoa/model_checking/network_model_data.py
  • tests/test_material_results.py
  • tests/testing_data/materials.sql
  • tests/testing_data/mediumville.sql
  • tests/testing_data/mediumville_sets.json
💤 Files with no reviewable changes (1)
  • tests/testing_data/mediumville.sql

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
temoa/model_checking/network_model_data.py (1)

359-443: ⚠️ Potential issue | 🟠 Major

Restrict survival-curve EOL edges to periods where the vintage can retire.

Line 359 no longer gates the rest of the loop, so the EOL branch now runs for every period. With has_survival defined only by tuple membership, survival-curve techs get EOL edges before v and after v + lifetime, which pollutes the network data. Bound that branch by v <= p <= v + lifetime.

♻️ Suggested fix
-            has_survival = (r, tech, v) in basic_data['tech_survival_curve']
+            has_survival = (
+                (r, tech, v) in basic_data['tech_survival_curve']
+                and v <= p <= v + lifetime
+            )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/model_checking/network_model_data.py` around lines 359 - 443, The
survival-curve EOL branch is executing for periods outside the technology's
active vintage because has_survival is only checked by tuple membership;
restrict that branch to periods where the vintage is active by adding a
condition v <= p <= v + lifetime (so the EOL loop runs only when the vintage
could be alive). Update the conditional that currently checks is_natural_eol or
is_retireable or has_survival (around the has_survival computation and the
lookup_data['eol'] loop) to include the v <= p <= v + lifetime guard before
adding entries to res.available_techs, res.source_commodities and
res.capacity_commodities and before adding to living_techs.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/components/technology.py`:
- Around line 441-470: In check_existing_capacity, after the tech_uncap guard
(inside the loop over model.existing_capacity.sparse_iterkeys), skip rows whose
existing capacity is explicitly zero instead of erroring; retrieve the stored
capacity for (r, t, v) from model.existing_capacity (or via
value(model.existing_capacity[r, t, v])) and if it equals 0, continue the loop
before computing lifetime and checking active_capacity_available_rptv so
explicit zero-capacity rows don't cause a ValueError.

---

Outside diff comments:
In `@temoa/model_checking/network_model_data.py`:
- Around line 359-443: The survival-curve EOL branch is executing for periods
outside the technology's active vintage because has_survival is only checked by
tuple membership; restrict that branch to periods where the vintage is active by
adding a condition v <= p <= v + lifetime (so the EOL loop runs only when the
vintage could be alive). Update the conditional that currently checks
is_natural_eol or is_retireable or has_survival (around the has_survival
computation and the lookup_data['eol'] loop) to include the v <= p <= v +
lifetime guard before adding entries to res.available_techs,
res.source_commodities and res.capacity_commodities and before adding to
living_techs.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5f1be936-055d-424e-b67f-9c27546b50ba

📥 Commits

Reviewing files that changed from the base of the PR and between 5619478 and 0531717.

📒 Files selected for processing (4)
  • temoa/components/capacity.py
  • temoa/components/technology.py
  • temoa/model_checking/network_model_data.py
  • tests/test_material_results.py

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (5)
temoa/tutorial_assets/utopia.sql (3)

1134-1151: 🧹 Nitpick | 🔵 Trivial

Remove redundant FK constraint (same as schema).

Line 1149 declares FOREIGN KEY (tech) REFERENCES technology (tech), but lines 1140-1141 already define this inline.

🧹 Proposed fix
     efficiency  real,
     lifetime    integer,

-    FOREIGN KEY (tech) REFERENCES technology (tech),
     PRIMARY KEY (region, input_comm, tech, vintage, output_comm)
 );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/tutorial_assets/utopia.sql` around lines 1134 - 1151, The CREATE TABLE
myopic_efficiency has a redundant foreign key constraint: remove the duplicate
table-level clause "FOREIGN KEY (tech) REFERENCES technology (tech)" because the
column "tech TEXT REFERENCES technology (tech)" already defines the same FK;
update the myopic_efficiency definition to keep the inline REFERENCES on the
tech column and delete the separate FOREIGN KEY (tech) ... line.

1195-1217: ⚠️ Potential issue | 🟠 Major

Missing update: output_curtailment.season FK inconsistent with schema.

The schema file (temoa_schema_v4.sql line 711) changed output_curtailment.season to reference season_label(season), but this file still references time_period(period) at line 1203.

This FK is semantically incorrect — season values in this database are labels like 'inter', 'summer', 'winter' (see season_label inserts at lines 1435-1437), not period integers.

🐛 Proposed fix
     period      INTEGER
         REFERENCES time_period (period),
     season      TEXT
-        REFERENCES time_period (period),
+        REFERENCES season_label (season),
     tod         TEXT
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/tutorial_assets/utopia.sql` around lines 1195 - 1217, The
output_curtailment table's season column currently references
time_period(period) but should reference season_label(season); update the CREATE
TABLE for output_curtailment so the season column's FOREIGN KEY references
season_label(season) (retain the column type TEXT and other constraints),
ensuring the PRIMARY KEY and other references remain unchanged; target the
output_curtailment definition and change the REFERENCES clause for the season
column to season_label(season).

1291-1311: 🧹 Nitpick | 🔵 Trivial

Changes align with schema; remove redundant FK.

FK additions and nullability changes correctly mirror temoa_schema_v4.sql. Line 1309 duplicates the inline FK at lines 1301-1302.

🧹 Remove duplicate FK
     flow        REAL,

-    FOREIGN KEY (tech) REFERENCES technology (tech),
     PRIMARY KEY (scenario, region, period, input_comm, tech, vintage, output_comm)
 );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/tutorial_assets/utopia.sql` around lines 1291 - 1311, The CREATE TABLE
output_flow_out_summary contains a duplicated foreign key for technology: remove
the redundant FOREIGN KEY (tech) REFERENCES technology (tech) clause (the tech
column already has an inline REFERENCES technology (tech)), keeping the tech
column definition and the PRIMARY KEY intact; verify output_flow_out_summary
still matches temoa_schema_v4.sql after removing that duplicate FK.
temoa/db_schema/temoa_schema_v4.sql (2)

1091-1111: 🧹 Nitpick | 🔵 Trivial

Nullability changes support materials module; remove redundant FK.

The removal of NOT NULL constraints for scenario, region, input_comm, and flow correctly enables writing partial records for construction/EOL material flows per PR objectives.

However, line 1109 duplicates the FK constraint already declared inline at lines 1101-1102.

🧹 Remove duplicate FK
     flow        REAL,

-    FOREIGN KEY (tech) REFERENCES technology (tech),
     PRIMARY KEY (scenario, region, period, input_comm, tech, vintage, output_comm)
 );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/db_schema/temoa_schema_v4.sql` around lines 1091 - 1111, The table
output_flow_out_summary declares tech TEXT REFERENCES technology (tech) inline,
but also repeats the same FOREIGN KEY (tech) REFERENCES technology (tech) in the
table constraints; remove the duplicate table-level FOREIGN KEY constraint so
only the inline REFERENCES on the tech column remains (search for
output_flow_out_summary and the redundant FOREIGN KEY (tech) REFERENCES
technology (tech) and delete that table-constraint entry).

1074-1086: 🧹 Nitpick | 🔵 Trivial

Remove redundant FK constraint.

Line 1085 declares FOREIGN KEY (tech) REFERENCES technology (tech), but this constraint is already defined inline at lines 1076-1077. The duplicate is harmless but adds clutter.

🧹 Proposed fix
     efficiency  real,
     lifetime    integer,

-    FOREIGN KEY (tech) REFERENCES technology (tech),
     PRIMARY KEY (region, input_comm, tech, vintage, output_comm)
 );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@temoa/db_schema/temoa_schema_v4.sql` around lines 1074 - 1086, Remove the
redundant foreign-key constraint that repeats the inline reference on the tech
column: there is already an inline "tech TEXT REFERENCES technology (tech)"
definition, so delete the separate "FOREIGN KEY (tech) REFERENCES technology
(tech)" constraint to avoid duplication and clutter while keeping the inline
reference and the existing PRIMARY KEY (region, input_comm, tech, vintage,
output_comm) intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/test_material_results.py`:
- Around line 36-38: The test assigns into config.myopic_inputs unconditionally;
change it so these assignments only run when mode == "myopic" and, if mode ==
"myopic" but config.myopic_inputs is None, fail fast with a clear test error
(e.g., raise AssertionError or pytest.fail) explaining that MYOPIC mode requires
myopic_inputs; reference config.myopic_inputs and the test's mode variable and
ensure this check is added before the assignments (this mirrors the expectations
in HybridLoader.create_data_dict).

---

Outside diff comments:
In `@temoa/db_schema/temoa_schema_v4.sql`:
- Around line 1091-1111: The table output_flow_out_summary declares tech TEXT
REFERENCES technology (tech) inline, but also repeats the same FOREIGN KEY
(tech) REFERENCES technology (tech) in the table constraints; remove the
duplicate table-level FOREIGN KEY constraint so only the inline REFERENCES on
the tech column remains (search for output_flow_out_summary and the redundant
FOREIGN KEY (tech) REFERENCES technology (tech) and delete that table-constraint
entry).
- Around line 1074-1086: Remove the redundant foreign-key constraint that
repeats the inline reference on the tech column: there is already an inline
"tech TEXT REFERENCES technology (tech)" definition, so delete the separate
"FOREIGN KEY (tech) REFERENCES technology (tech)" constraint to avoid
duplication and clutter while keeping the inline reference and the existing
PRIMARY KEY (region, input_comm, tech, vintage, output_comm) intact.

In `@temoa/tutorial_assets/utopia.sql`:
- Around line 1134-1151: The CREATE TABLE myopic_efficiency has a redundant
foreign key constraint: remove the duplicate table-level clause "FOREIGN KEY
(tech) REFERENCES technology (tech)" because the column "tech TEXT REFERENCES
technology (tech)" already defines the same FK; update the myopic_efficiency
definition to keep the inline REFERENCES on the tech column and delete the
separate FOREIGN KEY (tech) ... line.
- Around line 1195-1217: The output_curtailment table's season column currently
references time_period(period) but should reference season_label(season); update
the CREATE TABLE for output_curtailment so the season column's FOREIGN KEY
references season_label(season) (retain the column type TEXT and other
constraints), ensuring the PRIMARY KEY and other references remain unchanged;
target the output_curtailment definition and change the REFERENCES clause for
the season column to season_label(season).
- Around line 1291-1311: The CREATE TABLE output_flow_out_summary contains a
duplicated foreign key for technology: remove the redundant FOREIGN KEY (tech)
REFERENCES technology (tech) clause (the tech column already has an inline
REFERENCES technology (tech)), keeping the tech column definition and the
PRIMARY KEY intact; verify output_flow_out_summary still matches
temoa_schema_v4.sql after removing that duplicate FK.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0ef5b092-3148-4d64-9015-a6ed3a285987

📥 Commits

Reviewing files that changed from the base of the PR and between 0531717 and 3f89bbd.

📒 Files selected for processing (3)
  • temoa/db_schema/temoa_schema_v4.sql
  • temoa/tutorial_assets/utopia.sql
  • tests/test_material_results.py

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/model_checking/network_model_data.py`:
- Around line 294-306: Remove the unnecessary f-string prefix from the query
(use a normal triple-quoted string for variable query) and rename the ambiguous
loop variable l to lifetime for clarity; update the for-loop unpacking to "for
r, tech, v, oc, lifetime in cur.execute(query,
(TemoaModel.default_lifetime_tech,)).fetchall():" and adjust the lookups access
to use lifetime (lookups['eol'][(r, tech, v, lifetime)].append(oc)), leaving
TemoaModel.default_lifetime_tech, query, cur.execute, and lookups['eol']
references unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: e97fc11f-6959-4874-8931-7eb36b0f751c

📥 Commits

Reviewing files that changed from the base of the PR and between 3f89bbd and 860fae8.

📒 Files selected for processing (5)
  • temoa/_internal/table_data_puller.py
  • temoa/components/capacity.py
  • temoa/db_schema/temoa_schema_v4.sql
  • temoa/model_checking/network_model_data.py
  • temoa/tutorial_assets/utopia.sql

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/test_network_model_data.py (1)

279-286: 🧹 Nitpick | 🔵 Trivial

Consider combining adjacent branches returning identical results.

Both the unlim_cap==1 and retire==1 branches return empty results. Combining them would reduce duplication.

♻️ Optional refactor
-        elif 'technology WHERE unlim_cap==1' in query:
-            m = MagicMock()
-            m.fetchall.return_value = []
-            return m
-        elif 'technology WHERE retire==1' in query:
+        elif 'technology WHERE unlim_cap==1' in query or 'technology WHERE retire==1' in query:
             m = MagicMock()
             m.fetchall.return_value = []
             return m

That said, keeping them separate is also reasonable for maintainability if these queries may need different mock data in future tests.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_network_model_data.py` around lines 279 - 286, The two adjacent
branches checking for 'technology WHERE unlim_cap==1' and 'technology WHERE
retire==1' both create a MagicMock with m.fetchall.return_value = [] and return
it; replace them with a single conditional that matches either substring (e.g.,
using an or or membership check) and returns the single MagicMock to remove
duplication while preserving behavior in the test helper that inspects the query
string.
♻️ Duplicate comments (1)
tests/test_network_model_data.py (1)

367-374: 🧹 Nitpick | 🔵 Trivial

Same optional refactor opportunity as above.

Consider combining with the adjacent retire==1 branch for consistency.

♻️ Optional refactor
-        elif 'technology WHERE unlim_cap==1' in query:
-            m = MagicMock()
-            m.fetchall.return_value = []
-            return m
-        elif 'technology WHERE retire==1' in query:
+        elif 'technology WHERE unlim_cap==1' in query or 'technology WHERE retire==1' in query:
             m = MagicMock()
             m.fetchall.return_value = []
             return m
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_network_model_data.py` around lines 367 - 374, The two separate
elif branches checking for 'technology WHERE unlim_cap==1' and 'technology WHERE
retire==1' both create the same MagicMock with an empty fetchall; replace them
with a single branch that detects either query (e.g., combine the two 'in'
checks with an or) and returns one MagicMock with m.fetchall.return_value = []
to avoid duplicated code; locate the existing elifs that reference the exact
strings 'technology WHERE unlim_cap==1' and 'technology WHERE retire==1' and
merge their logic accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@temoa/components/capacity.py`:
- Around line 644-648: The loop currently calls model.used_techs.add(t) for
every end_of_life_output row even when no retirement flows are created; restrict
marking techs as used to only those that actually produce a retirement flow by
moving the model.used_techs.add(t) call inside the if (r, t, v) in
model.retirement_periods block (i.e., after iterating and adding to
model.retirement_production_processes[(r, p, o)]), so used_techs is only updated
when retirement_production_processes entries are created.
- Around line 607-636: The new p0 EOL branch can add retirement periods for
non-existent existing_capacity rows; update the loop that builds
model.retirement_periods (the unique_rtv iteration handling is_p0_eol) to
require a positive existing_capacity before treating a process as p0 EOL: when
computing is_p0_eol, also check model.existing_capacity[(r,t,v)] (or use
model.existing_capacity.get((r,t,v), 0) if safe) and skip adding p0 retirement
periods if that value is zero or missing; keep all other checks (is_living,
is_natural_eol, is_early_retire, is_survival_curve) unchanged so only real
existing-vintage capacity triggers the p0 exact-EOL branch.

In `@temoa/model_checking/network_model_data.py`:
- Around line 443-448: The branch that sets is_p0_eol currently treats any
vintage whose lifetime ends at the first modeled period as an EOL even if there
was never any installed capacity; update the condition so is_p0_eol also
requires a positive install-base lookup for that (r, tech, v). Concretely,
modify the is_p0_eol assignment (in the loop over periods where is_p0_eol,
is_living, and living_rtv are used) to check the install-base/existing capacity
data structure (e.g., install_base[(r, tech, v)] or
existing_capacity_lookup.get((r,tech,v),0) > 0) before allowing a horizon-start
EOL edge. Ensure the rest of the logic still uses is_p0_eol and is_living
unchanged.
- Around line 357-361: res.demand_commodities is populated from the full payload
before you narrow periods for myopic_index, so in myopic mode you must filter
demand_commodities to the same period slice; after the existing narrowing of
periods (where periods is set from basic_data['periods'] and optionally filtered
by myopic_index.base_year and myopic_index.last_demand_year) remove any entries
in res.demand_commodities whose Period keys fall outside that inclusive range
(use the same Period comparison logic as the periods list filtering) so
demand_commodities aligns with the filtered periods.

---

Outside diff comments:
In `@tests/test_network_model_data.py`:
- Around line 279-286: The two adjacent branches checking for 'technology WHERE
unlim_cap==1' and 'technology WHERE retire==1' both create a MagicMock with
m.fetchall.return_value = [] and return it; replace them with a single
conditional that matches either substring (e.g., using an or or membership
check) and returns the single MagicMock to remove duplication while preserving
behavior in the test helper that inspects the query string.

---

Duplicate comments:
In `@tests/test_network_model_data.py`:
- Around line 367-374: The two separate elif branches checking for 'technology
WHERE unlim_cap==1' and 'technology WHERE retire==1' both create the same
MagicMock with an empty fetchall; replace them with a single branch that detects
either query (e.g., combine the two 'in' checks with an or) and returns one
MagicMock with m.fetchall.return_value = [] to avoid duplicated code; locate the
existing elifs that reference the exact strings 'technology WHERE unlim_cap==1'
and 'technology WHERE retire==1' and merge their logic accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5aebfb27-3df4-4542-9fdc-ab6774be436b

📥 Commits

Reviewing files that changed from the base of the PR and between 860fae8 and cba8975.

📒 Files selected for processing (5)
  • temoa/components/capacity.py
  • temoa/components/technology.py
  • temoa/core/model.py
  • temoa/model_checking/network_model_data.py
  • tests/test_network_model_data.py

@idelder idelder force-pushed the bugfix/materials_myopic branch from cd16db0 to fc0d408 Compare March 16, 2026 21:10
@idelder
Copy link
Author

idelder commented Mar 16, 2026

Well that was a lot more complicated than I expected

@idelder idelder closed this Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant