02. Generator-Level Outage Reasons

02. Generator-Level Outage Reasons#

Authors: Brian Nguyen · Claire Kim · Created: 2026-07-23 · Tags: #Experiment #Quanto #Outages

2. Objective & Input Framework#

Objective: test whether the report’s reason codes carry a usable weather signal:

  1. Classify the nature_of_work taxonomy and isolate the weather-driven categories.

  2. Build an hourly outage MW series split by reason, reconciled against the derived aggregate.

  3. Measure whether the temperature-driven series co-moves with real-time prices.

Data & Feature Catalog:

Dataset

Granularity

Role

caiso_curtailed_non_operational_generator_report

outage segment × daily edition

reason-coded generator outages

caiso_outages_aggregated_hourly (derived)

hourly, system-wide

validation target

caiso_lmp_real_time_15_min (SP15 hub)

15-min

price leg

 
import polars as pl
from bokeh.io import output_notebook, show
from bokeh.models import (
    ColorBar,
    ColumnDataSource,
    DatetimeTickFormatter,
    HoverTool,
    LabelSet,
    NumeralTickFormatter,
    Range1d,
)
from bokeh.plotting import figure
from bokeh.transform import linear_cmap
from bokeh.resources import INLINE

from markets.definitions.caiso import CAISO_CLOCK
from chart_theme import SECONDARY, SERIES, html_table, style
from settings import config

DATA_DIR = config("DATA_DIR")
OS_DIR = DATA_DIR / "opensource"

report = pl.read_parquet(
    OS_DIR / "caiso_curtailed_non_operational_generator_report.parquet"
)

output_notebook(INLINE, hide_banner=True)


def ts_figure(title: str, y_label: str):
    """One-line factory for the standard time-series figure frame."""
    return figure(
        title=title,
        x_axis_type="datetime",
        y_axis_label=y_label,
        height=380,
        sizing_mode="stretch_width",
        toolbar_location="above",
    )

3. Mathematical Model & Analytical Formulation#

The report arrives as outage segments, not an hourly series: continuous windows \([s_g, e_g]\) per generator, end null while ongoing, several overlapping segments per unit per day, and every edition republishing the prior trade day. One unit from the latest edition:

latest = report.filter(pl.col("publish_time_utc") == pl.col("publish_time_utc").max())
busiest = (
    latest.group_by("resource_id").agg(pl.len().alias("segments"))
    .sort("segments", descending=True)["resource_id"][0]
)
html_table(
    latest.filter(pl.col("resource_id") == busiest)
    .select(
        "resource_name", "outage_type", "nature_of_work",
        "curtailment_start_time_utc", "curtailment_end_time_utc", "curtailment_mw",
    )
    .sort("curtailment_start_time_utc")
)
49 rows × 6 columns
resource_name outage_type nature_of_work curtailment_start_time_utc curtailment_end_time_utc curtailment_mw
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 07:00 2026-08-25 08:00 34.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 07:00 2026-08-25 08:00 12.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 08:00 2026-08-25 09:00 35.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 08:00 2026-08-25 09:00 11.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 09:00 2026-08-25 10:00 36.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 09:00 2026-08-25 10:00 10.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 10:00 2026-08-25 11:00 36.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 10:00 2026-08-25 11:00 10.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 11:00 2026-08-25 12:00 36.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 11:00 2026-08-25 12:00 10.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 12:00 2026-08-25 13:00 36.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 12:00 2026-08-25 13:00 10.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 13:00 2026-08-25 14:00 36.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 13:00 2026-08-25 14:00 10.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 14:00 2026-08-25 15:00 35.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 14:00 2026-08-25 15:00 11.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 15:00 2026-08-25 16:00 34.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 15:00 2026-08-25 16:00 12.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 16:00 2026-08-25 17:00 32.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 16:00 2026-08-25 17:00 14.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 17:00 2026-08-25 18:00 31.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 17:00 2026-08-25 18:00 15.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 18:00 2026-08-25 19:00 35.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 18:00 2026-08-25 19:00 11.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 19:00 2026-08-25 20:00 34.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 19:00 2026-08-25 20:00 12.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 20:00 2026-08-25 21:00 33.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 20:00 2026-08-25 21:00 13.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 21:00 2026-08-25 22:00 31.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 21:00 2026-08-25 22:00 15.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 22:00 2026-08-25 23:00 31.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 22:00 2026-08-25 23:00 15.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-25 23:00 2026-08-26 00:00 31.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-25 23:00 2026-08-26 00:00 15.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 00:00 2026-08-26 01:00 31.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 00:00 2026-08-26 01:00 15.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 01:00 2026-08-26 02:00 32.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 01:00 2026-08-26 02:00 14.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 02:00 2026-08-26 03:00 34.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 02:00 2026-08-26 03:00 12.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 03:00 2026-08-26 04:00 30.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 03:00 2026-08-26 04:00 16.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 04:00 2026-08-26 05:00 31.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 04:00 2026-08-26 05:00 15.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 05:00 2026-08-26 06:00 32.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 05:00 2026-08-26 06:00 14.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 06:00 2026-08-26 06:59 33.00
La Paloma Generating Plant Unit #2 FORCED AMBIENT_DUE_TO_TEMP 2026-08-26 06:00 2026-08-26 06:59 13.00
La Paloma Generating Plant Unit #2 FORCED PLANT_TROUBLE 2026-08-26 06:59 — 46.00

A. Hourly active indicator. For Pacific trade day \(T\) and hour \(h\) with UTC bounds \([\underline h, \bar h)\), an outage row \(g\) from the edition covering \(T\) is active when its window overlaps the hour:

\[a_g(h) = \mathbf{1}\{ s_g \le \bar h \ \wedge\ (e_g = \varnothing \ \vee\ e_g \ge \underline h) \}\]

B. Category partitioning & fleet reconciliation.

\[W^{R}_h = \sum_{g \in R,\ \text{FORCED}} m_g \, a_g(h), \qquad \sum_{R} W^{R}_h = F_h\]

The four reason groups (temperature, fuel/steam/charge, hydro, other) are mutually exclusive and exhaustive, so summing active curtailed MW across them must reconstruct the derived forced aggregate \(F_h\) (forced_curtailment_mw) exactly; section 6.1 computes the check.

C. The quanto identity (pricing motivation).

\[E[XY] = E[X]\,E[Y] + \mathrm{Cov}(X,Y)\]

Conditioning power prices (\(Y\)) on the outage variable (\(X = W_h\)) adds value only if \(\mathrm{Cov}(W, P) > 0\).

Variable Definition Table

Symbol

Meaning

Type / Units

Bounds

Code name

\(\underline h, \bar h\)

hour interval start / end (UTC)

datetime

-

interval_start, interval_end

\(s_g, e_g\)

curtailment window start / end

datetime, end nullable

\(s_g \le e_g\)

out_start, out_end

\(m_g\)

curtailed capacity of row \(g\)

Float64, MW

\(> 0\)

curtailment_mw

\(W^{temp}_h\)

temp-derate forced MW

Float64, MW

\(\ge 0\)

temp_mw

\(W^{fuel}_h\)

fuel/steam/charge forced MW

Float64, MW

\(\ge 0\)

fuel_mw

\(W^{hydro}_h\)

hydro-condition forced MW

Float64, MW

\(\ge 0\)

hydro_mw

\(W^{other}_h\)

all other forced MW

Float64, MW

\(\ge 0\)

other_forced_mw

\(F_h\)

derived forced aggregate

Float64, MW

\(\ge 0\)

forced_curtailment_mw

\(P_d\)

daily max SP15 RT 15-min LMP

Float64, $/MWh

can be \(< 0\)

rt_max

4. Quantitative Data Exploration & Assumption Testing#

4.1 The nature_of_work Taxonomy#

Rows, resources, and curtailed MW per reason and outage type:

taxonomy = (
    report.group_by("nature_of_work", "outage_type")
    .agg(
        pl.len().alias("rows"),
        pl.col("resource_id").n_unique().alias("resources"),
        pl.col("curtailment_mw").sum().round(0).alias("total_mw"),
    )
    .sort("rows", descending=True)
)
html_table(taxonomy)
38 rows × 5 columns
nature_of_work outage_type rows resources total_mw
AMBIENT_DUE_TO_TEMP FORCED 1,899,371 335 23,441,905.00
AMBIENT_NOT_DUE_TO_TEMP FORCED 564,499 429 19,459,245.00
PLANT_TROUBLE FORCED 553,926 1,376 26,849,090.00
PLANT_MAINTENANCE FORCED 170,153 1,282 8,626,559.00
PLANT_MAINTENANCE PLANNED 111,913 967 12,549,905.00
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY FORCED 75,499 327 1,948,760.00
TRANSMISSION_INDUCED FORCED 59,084 1,167 4,126,193.00
NEW_GENERATOR_TEST_ENERGY FORCED 58,083 304 9,177,498.00
ENVIRONMENTAL_RESTRICTIONS FORCED 36,820 288 5,520,500.00
UNIT_TESTING FORCED 33,994 791 6,320,789.00
TRANSMISSION_INDUCED PLANNED 19,082 871 1,197,632.00
METERING_TELEMETRY FORCED 17,991 844 756,309.00
PLANT_TROUBLE PLANNED 11,797 125 782,834.00
TECHNICAL_LIMITATIONS_NOT_IN_MARKET_MODEL FORCED 6,863 118 225,641.00
NEW_GENERATOR_TEST_ENERGY PLANNED 6,438 71 471,671.00
MONTHLY_USE_LIMIT_REACHED FORCED 5,962 66 272,657.00
TRANSITIONAL_LIMITATION FORCED 4,166 96 1,444,488.00
UNIT_TESTING PLANNED 3,639 300 289,558.00
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY PLANNED 3,149 62 501,406.00
OTHER_USE_LIMIT_REACHED FORCED 2,680 75 215,848.00
SHORT_TERM_USE_LIMIT_REACHED FORCED 1,759 209 292,004.00
ENVIRONMENTAL_RESTRICTIONS PLANNED 1,670 15 409,816.00
ANNUAL_USE_LIMIT_REACHED FORCED 1,355 29 123,203.00
RIMS_OUTAGE FORCED 1,034 23 74,915.00
METERING_TELEMETRY PLANNED 954 150 356,759.00
TRANSITIONAL_LIMITATION PLANNED 691 5 183,100.00
RTU_RIG FORCED 675 176 26,029.00
AMBIENT_NOT_DUE_TO_TEMP PLANNED 568 19 77,035.00
UNIT_SUPPORTING_STARTUP FORCED 448 25 44,344.00
AMBIENT_DUE_TO_TEMP PLANNED 389 3 15,397.00
RIMS_OUTAGE PLANNED 311 9 33,174.00
ICCP FORCED 206 24 7,960.00
TECHNICAL_LIMITATIONS_NOT_IN_MARKET_MODEL PLANNED 143 3 14,132.00
RTU_RIG PLANNED 113 43 3,253.00
UNIT_SUPPORTING_STARTUP PLANNED 48 9 3,114.00
RIMS_TESTING FORCED 35 1 10,050.00
ICCP PLANNED 24 16 2,167.00
SHORT_TERM_USE_LIMIT_REACHED PLANNED 15 2 2,815.00

The same taxonomy as one bar per reason, with the forced/planned split. Bars count report rows: temperature derates are over half the report, while by curtailed MW the ranking shifts toward plant trouble.

taxonomy_wide = (
    taxonomy.pivot(on="outage_type", index="nature_of_work", values="rows")
    .fill_null(0)
    .with_columns((pl.col("FORCED") + pl.col("PLANNED")).alias("total"))
    .sort("total")
)
taxonomy_wide = taxonomy_wide.with_columns(
    (pl.col("total") / taxonomy_wide["total"].sum() * 100).round(1).alias("pct")
).with_columns(pl.format("{}%", pl.col("pct")).alias("pct_label"))

src = ColumnDataSource(taxonomy_wide.to_dict(as_series=False))
fig = figure(
    title="Report Rows by Outage Reason and Type",
    y_range=taxonomy_wide["nature_of_work"].to_list(),
    x_range=Range1d(0, taxonomy_wide["total"].max() * 1.12),
    x_axis_label="Report rows (outage segment-days)",
    height=460,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.hbar_stack(
    ["FORCED", "PLANNED"],
    y="nature_of_work",
    height=0.7,
    color=SERIES[:2],
    source=src,
    legend_label=["FORCED", "PLANNED"],
)
fig.add_layout(
    LabelSet(x="total", y="nature_of_work", text="pct_label", source=src,
             x_offset=5, text_baseline="middle", text_font_size="10px",
             text_color=SECONDARY)
)
fig.xaxis.formatter = NumeralTickFormatter(format="0.0a")
fig.add_tools(
    HoverTool(tooltips=[("reason", "@nature_of_work"), ("FORCED", "@FORCED{0,0}"),
              ("PLANNED", "@PLANNED{0,0}"), ("share of total", "@pct{0.0}%")])
)
fig.legend.location = "bottom_right"
show(style(fig))

4.2 Data Quality & Integrity Flags#

Null map. A null curtailment_end_time_utc means the outage was still ongoing at publish; the section-3 rule counts those rows as active through the end of the trade day.

NULL_MEANINGS = {
    "curtailment_end_time_utc": "outage still ongoing at publish",
    "net_qualifying_capacity_mw": "NQC not reported for some resources",
}
quality = (
    report.null_count()
    .transpose(include_header=True, header_name="column", column_names=["null rows"])
    .with_columns(
        (pl.col("null rows") / report.height * 100).round(1).alias("% of rows"),
        pl.col("column").replace_strict(NULL_MEANINGS, default="").alias("meaning"),
    )
)
html_table(quality)
11 rows × 4 columns
column null rows % of rows meaning
publish_time_utc 0 0.00
outage_mrid 0 0.00
resource_name 0 0.00
resource_id 0 0.00
outage_type 0 0.00
nature_of_work 0 0.00
curtailment_start_time_utc 0 0.00
curtailment_end_time_utc 620,593 17.00 outage still ongoing at publish
curtailment_mw 0 0.00
resource_pmax_mw 0 0.00
net_qualifying_capacity_mw 332,408 9.10 NQC not reported for some resources

4.3 Who Files Under the AMBIENT Categories#

What the three weather-linked AMBIENT reasons cover, by top filer:

AMBIENT = [
    "AMBIENT_DUE_TO_TEMP",
    "AMBIENT_NOT_DUE_TO_TEMP",
    "AMBIENT_DUE_TO_FUEL_INSUFFICIENCY",
]
top_filers = (
    report.filter(pl.col("nature_of_work").is_in(AMBIENT))
    .group_by("nature_of_work", "resource_name")
    .agg(pl.len().alias("rows"), pl.col("curtailment_mw").max().alias("max_mw"))
    .sort("nature_of_work", "rows", descending=[False, True])
    .group_by("nature_of_work", maintain_order=True)
    .head(5)
)
html_table(top_filers)
15 rows × 4 columns
nature_of_work resource_name rows max_mw
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY GEYSERS UNIT 14 (HEALDSBURG) 4,112 70.00
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY GEYSERS UNIT 11 (HEALDSBURG) 3,707 86.80
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY GEYSERS UNITS 7 & 8 AGGREGATE 3,686 59.80
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY GEYSERS UNIT 17 (HEALDSBURG) 3,589 75.50
AMBIENT_DUE_TO_FUEL_INSUFFICIENCY GEYSERS UNIT 16 (HEALDSBURG) 3,241 63.00
AMBIENT_DUE_TO_TEMP Pio Pico Unit 1 44,356 111.30
AMBIENT_DUE_TO_TEMP Pio Pico Unit 2 43,313 112.70
AMBIENT_DUE_TO_TEMP Pio Pico Unit 3 43,039 112.00
AMBIENT_DUE_TO_TEMP GATEWAY GENERATING STATION 40,571 585.00
AMBIENT_DUE_TO_TEMP Marsh Landing 3 40,292 208.96
AMBIENT_NOT_DUE_TO_TEMP PIT PH 7 UNIT 1 67,769 55.70
AMBIENT_NOT_DUE_TO_TEMP PIT PH 6 UNIT 2 61,887 40.00
AMBIENT_NOT_DUE_TO_TEMP PIT PH 7 UNIT 2 49,922 54.60
AMBIENT_NOT_DUE_TO_TEMP PIT PH 6 UNIT 1 49,222 39.00
AMBIENT_NOT_DUE_TO_TEMP HOOVER VEA 32,271 17.67

  • AMBIENT_DUE_TO_TEMP - gas peakers and CCGTs (Pio Pico, Gateway) derating in heat: the direct temperature signal and the primary quanto conditioning candidate.

  • AMBIENT_NOT_DUE_TO_TEMP - hydro (Pit River, Hoover): slow-moving water conditions, not spike-day heat.

  • AMBIENT_DUE_TO_FUEL_INSUFFICIENCY - Geysers steam-field limits, plus batteries at low state of charge below the top five: despite the name, not a cold-snap gas story in this sample.

5. Implementation & Simulation Logic#

Same recipe as derive_caiso_outages_aggregated_hourly, split by reason group instead of only forced/planned.

5.1 Trade-Date Mapping and the Hourly Grid#

Stamp each row to the trade day its edition covers, then build that day’s Pacific hourly grid (DST-correct, stored naive UTC).

report_stamped = report.with_columns(
    # The market's own clock does the conversion; the day back is the
    # reporting convention, since an edition published on one trade day
    # describes the day before it.
    trade_date=CAISO_CLOCK.trade_day(pl.col("publish_time_utc")).dt.offset_by("-1d"),
)

grid = (
    report_stamped.select("trade_date").unique().sort("trade_date")
    .with_columns(
        day_start=pl.col("trade_date").cast(pl.Datetime).dt.replace_time_zone("US/Pacific")
    )
    .with_columns(day_end=pl.col("day_start").dt.offset_by("1d"))
    .with_columns(
        hour=pl.datetime_ranges("day_start", "day_end", interval="1h", closed="left")
    )
    .explode("hour", empty_as_null=False)
    .with_columns(
        interval_start=pl.col("hour").dt.convert_time_zone("UTC").dt.replace_time_zone(None)
    )
    .with_columns(interval_end=pl.col("interval_start").dt.offset_by("1h"))
    .select("trade_date", "interval_start", "interval_end")
)

5.2 The Interval-Overlap Join#

Attach every segment to each hour of its trade day that its window overlaps - the activity indicator from section 3.

joined = grid.join_where(
    report_stamped.select(
        "trade_date", "outage_type", "nature_of_work", "curtailment_mw",
        pl.col("curtailment_start_time_utc").alias("out_start"),
        pl.col("curtailment_end_time_utc").alias("out_end"),
    ),
    pl.col("trade_date") == pl.col("trade_date_right"),
    pl.col("out_start") <= pl.col("interval_end"),
    pl.col("out_end").is_null() | (pl.col("out_end") >= pl.col("interval_start")),
)

5.3 Reason-Split Aggregation#

Sum active MW into the four reason groups per hour; daily means feed the charts in section 6. When the pricing stage needs this series, the builder graduates into src/pulls/ beside derive_caiso_outages_aggregated_hourly.py.

forced = pl.col("outage_type") == "FORCED"
mw = pl.col("curtailment_mw")
hourly = (
    joined.group_by("interval_start")
    .agg(
        temp_mw=mw.filter(forced & (pl.col("nature_of_work") == "AMBIENT_DUE_TO_TEMP")).sum(),
        fuel_mw=mw.filter(
            forced & (pl.col("nature_of_work") == "AMBIENT_DUE_TO_FUEL_INSUFFICIENCY")
        ).sum(),
        hydro_mw=mw.filter(
            forced & (pl.col("nature_of_work") == "AMBIENT_NOT_DUE_TO_TEMP")
        ).sum(),
        other_forced_mw=mw.filter(forced & ~pl.col("nature_of_work").is_in(AMBIENT)).sum(),
    )
    .sort("interval_start")
)

daily = (
    hourly.group_by(pl.col("interval_start").dt.truncate("1d").alias("day"))
    .agg(pl.col("temp_mw", "fuel_mw", "hydro_mw", "other_forced_mw").mean())
    .sort("day")
)

6. Quantitative Analysis & Validation#

6.1 Reconciliation Against the Derived Aggregate#

The check \(\sum_R W^R_h = F_h\) against forced_curtailment_mw, hourly:

derived = pl.read_parquet(OS_DIR / "caiso_outages_aggregated_hourly_derived.parquet")
check = (
    hourly.with_columns(
        total=pl.col("temp_mw") + pl.col("fuel_mw") + pl.col("hydro_mw")
        + pl.col("other_forced_mw"),
        interval_start=pl.col("interval_start").cast(pl.Datetime("us")),
    )
    .join(derived, left_on="interval_start", right_on="interval_start_utc")
    .select(
        (pl.col("total") - pl.col("forced_curtailment_mw"))
        .abs().max().alias("max abs diff vs derived (MW)")
    )
)
html_table(check)
1 rows × 1 columns
max abs diff vs derived (MW)
0.000

6.2 The Reason-Split Series#

The bands sum to the derived forced aggregate, so the top envelope is total forced MW and the seasonal mix reads in absolute scale.

LABELS = {
    "temp_mw": "Temp derates",
    "hydro_mw": "Hydro conditions",
    "fuel_mw": "Fuel/steam/charge",
    "other_forced_mw": "Other forced",
}
stack_cols = ["temp_mw", "hydro_mw", "fuel_mw", "other_forced_mw"]
area_src = ColumnDataSource(
    {"day": daily["day"].to_numpy(),
     **{c: daily[c].to_numpy() for c in stack_cols}}
)
fig = ts_figure("Daily Forced Outage MW by Reason Group (stacked)", "Outage (MW)")
fig.varea_stack(stack_cols, x="day", color=SERIES[:4], alpha=0.85,
                legend_label=[LABELS[c] for c in stack_cols], source=area_src)
fig.legend.location = "top_left"
show(style(fig))

6.3 Do Temp-Driven Outages Co-Move With Prices?#

A heat wave cuts thermal capacity through ambient derates while A/C demand rises, so the system tightens from both sides and the real-time price spikes. Daily mean temp-derate MW against the daily max SP15 RT 15-min LMP, with the non-weather forced series as the control:

rt_daily = (
    pl.scan_parquet(OS_DIR / "caiso_lmp_real_time_15_min.parquet")
    .filter(pl.col("location") == "TH_SP15_GEN-APND")
    .group_by(pl.col("interval_start_utc").dt.truncate("1d").alias("day"))
    .agg(pl.col("lmp").max().alias("rt_max"))
    .collect()
    .with_columns(pl.col("day").cast(pl.Datetime("us")))
)
market_data = (
    daily.with_columns(pl.col("day").cast(pl.Datetime("us")))
    .join(rt_daily, on="day")
)
correlations = market_data.select(
    pl.corr("temp_mw", "rt_max").alias("corr(temp derates, RT max)"),
    pl.corr("other_forced_mw", "rt_max").alias("corr(other forced, RT max)"),
)
html_table(correlations)
1 rows × 2 columns
corr(temp derates, RT max) corr(other forced, RT max)
0.318 −0.144 
# Sequential single-hue ramp anchored on the house blue: light = start of
# the sample, dark = most recent. Color carries the calendar, so the
# summer cluster shows up as a dark band in the right tail.
BLUE_RAMP = ["#d6e6f8", "#a7c8ee", "#76a8e3", "#4589d9", "#2a78d6", "#1d5cad", "#144484"]
day_ms = market_data["day"].dt.epoch(time_unit="ms").to_numpy()
color_by_day = linear_cmap("day_ms", BLUE_RAMP, day_ms.min(), day_ms.max())
scatter_src = ColumnDataSource(
    {"temp_mw": market_data["temp_mw"].to_numpy(), "rt_max": market_data["rt_max"].to_numpy(),
     "day_ms": day_ms}
)

fig = figure(
    title="Daily Max SP15 RT LMP vs Daily Mean Temp-Derate MW",
    x_axis_label="Temp-derate outage MW (daily mean)",
    y_axis_label="SP15 RT 15-min LMP daily max ($/MWh)",
    height=380,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.scatter("temp_mw", "rt_max", source=scatter_src, size=6,
            color=color_by_day, alpha=0.6)
fig.add_layout(
    ColorBar(color_mapper=color_by_day["transform"], width=8,
             formatter=DatetimeTickFormatter(months="%b %Y", days="%b %Y"),
             label_standoff=6, major_label_text_color=SECONDARY),
    "right",
)
show(style(fig))

Across the three-year sample the correlation of daily mean temp-derate MW with the SP15 RT daily max is \(r \approx 0.32\), against \(-0.14\) for the non-weather forced control. A single mild year measured 0.07 and read it as no relationship; the extended sample holds 8 days with an RT max above 1,000 dollars per MWh, reaching the 2,000 cap, and those days land at elevated derate levels often enough to turn the full-sample correlation positive - a tail relationship, not a tight line.

Notebook 03 replaces the outage proxy with actual station HDD/CDD and measures Cov(weather index, price) directly; an event study of the RT 5-min LMP around the top temp-derate days is the other follow-up.