CAISO Basis Risk Exploration

Authors: Claire Kim · Brian Nguyen · Created: 2026-07-31 · Tags: #Exploration #Basis-Risk #Weather

This notebook measures basis risk across the CAISO weather, market, fuel, load, outage, and generator datasets, whose objects live on different geographies and time conventions. The goal is to separate the variables that align cleanly from the ones that require controls.

import sys
from pathlib import Path
import zipfile

import numpy as np
import pandas as pd
import polars as pl
from bokeh.io import output_notebook, show
from bokeh.models import ColumnDataSource, HoverTool, LabelSet, NumeralTickFormatter
from bokeh.plotting import figure
from bokeh.resources import INLINE
from IPython.display import display

pd.set_option("display.max_columns", 160)
pd.set_option("display.width", 180)
pd.set_option("display.float_format", lambda value: f"{value:,.4f}")

PROJECT_ROOT = Path.cwd()
if PROJECT_ROOT.name == "notebooks":
    PROJECT_ROOT = PROJECT_ROOT.parent

SRC_DIR = PROJECT_ROOT / "src"
if str(SRC_DIR) not in sys.path:
    sys.path.insert(0, str(SRC_DIR))

from chart_theme import SECONDARY, SERIES, html_table, style

output_notebook(INLINE, hide_banner=True)

DATA_DIR = PROJECT_ROOT / "data"
PROCESSED_DIR = DATA_DIR / "processed"
MARKET_DIR = DATA_DIR / "raw" / "caiso_market"
ZIP_PATH = DATA_DIR / "drive-download-20260724T164008Z-1-001.zip"

if not MARKET_DIR.exists() or not list(MARKET_DIR.glob("*.parquet")):
    if not ZIP_PATH.exists():
        raise FileNotFoundError(f"Missing market parquet folder and zip: {MARKET_DIR}, {ZIP_PATH}")
    MARKET_DIR.mkdir(parents=True, exist_ok=True)
    with zipfile.ZipFile(ZIP_PATH) as zf:
        zf.extractall(MARKET_DIR)

ZONE_LOCATION_MAP = {
    "TH_NP15_GEN-APND": "NP15",
    "TH_SP15_GEN-APND": "SP15",
    "TH_ZP26_GEN-APND": "ZP26",
    "NP15": "NP15",
    "SP15": "SP15",
    "ZP26": "ZP26",
    "ALL_CAISO": "CAISO",
    "CAISO": "CAISO",
}

HUB_ORDER = ["NP15", "SP15", "ZP26"]


def to_utc_hour(series: pd.Series) -> pd.Series:
    ts = pd.to_datetime(series, errors="coerce")
    if getattr(ts.dt, "tz", None) is None:
        ts = ts.dt.tz_localize("UTC")
    else:
        ts = ts.dt.tz_convert("UTC")
    return ts.dt.floor("h").dt.tz_localize(None)


def read_market(name: str) -> pd.DataFrame:
    path = MARKET_DIR / name
    if not path.exists():
        raise FileNotFoundError(path)
    return pd.read_parquet(path)

print(PROJECT_ROOT)

.

1. Load the Relevant Tables

weather_region_d = pd.read_parquet(PROCESSED_DIR / "caiso_weather_daily_region.parquet")
weather_zone_d = pd.read_parquet(PROCESSED_DIR / "caiso_weather_daily_weather_zone.parquet")
weather_region_h = pd.read_parquet(PROCESSED_DIR / "caiso_weather_hourly_region.parquet")
weather_zone_crosswalk = pd.read_parquet(PROCESSED_DIR / "weather_zone_crosswalk.parquet")
basis_risk_table = pd.read_parquet(PROCESSED_DIR / "weather_market_zone_basis_risk.parquet")

imputed_plants_path = PROCESSED_DIR / "caiso_generator_weather_imputed.parquet"
plant_daily_path = PROCESSED_DIR / "caiso_generator_weather_daily_imputed.parquet"
imputed_plants = pd.read_parquet(imputed_plants_path) if imputed_plants_path.exists() else pd.DataFrame()
plant_daily = pd.read_parquet(plant_daily_path) if plant_daily_path.exists() else pd.DataFrame()

market = {
    "da_lmp": read_market("caiso_lmp_day_ahead_hourly.parquet"),
    "rt15_lmp": read_market("caiso_lmp_real_time_15_min.parquet"),
    "fuel_prices": read_market("caiso_fuel_prices.parquet"),
    "fuel_regions": read_market("caiso_fuel_regions.parquet"),
    "load_hourly_tac": read_market("caiso_load_hourly.parquet"),
    "load_5min_system": read_market("caiso_load.parquet"),
    "outages_hourly": read_market("caiso_outages_aggregated_hourly_derived.parquet"),
    "curtailment_detail": read_market("caiso_curtailed_non_operational_generator_report.parquet"),
}

inventory = []
for name, df in {
    "weather_region_daily": weather_region_d,
    "weather_zone_daily": weather_zone_d,
    "weather_zone_crosswalk": weather_zone_crosswalk,
    "basis_risk_table": basis_risk_table,
    "imputed_generator_weather": imputed_plants,
    **market,
}.items():
    row = {"table": name, "rows": len(df), "columns": len(df.columns)}
    for col in ["date", "interval_start_utc", "timestamp_utc"]:
        if col in df.columns:
            values = pd.to_datetime(df[col], errors="coerce")
            row["start"] = values.min()
            row["end"] = values.max()
            break
    inventory.append(row)

display(html_table(pl.from_pandas(pd.DataFrame(inventory))))

13 rows × 5 columns

table	rows	columns	start	end
weather_region_daily	8,032	29	2020-12-31 00:00	2026-06-30 00:00
weather_zone_daily	8,032	30	2020-12-31 00:00	2026-06-30 00:00
weather_zone_crosswalk	7	11	—	—
basis_risk_table	8	11	—	—
imputed_generator_weather	1,661	53	—	—
da_lmp	92,349	10	2023-05-11 07:00	2026-08-27 06:00
rt15_lmp	369,394	10	2023-05-11 07:00	2026-08-27 06:45
fuel_prices	10,876,916	4	2018-01-01 08:00	2026-08-27 06:00
fuel_regions	184	7	—	—
load_hourly_tac	1,944,693	4	2018-01-01 08:00	2026-08-27 06:00
load_5min_system	880,751	3	2018-04-10 07:00	2026-08-27 06:55
outages_hourly	45,479	4	2021-06-17 07:00	2026-08-26 06:00
curtailment_detail	3,655,547	11	—	—

Time coverage differs across tables, so every join below uses only overlapping observations.

2. Weather Geography Basis

crosswalk_summary = (
    weather_zone_crosswalk.groupby("region")
    .agg(
        proxy_locations=("location", lambda values: ", ".join(sorted(values))),
        weather_zone_count=("noaa_climate_division_proxy", "nunique"),
        weather_zones=("noaa_climate_division_proxy", lambda values: ", ".join(sorted(set(values)))),
        weight_sum=("prototype_caiso_weight", "sum"),
    )
    .reset_index()
)

display(html_table(pl.from_pandas(basis_risk_table), kind="reference"))
print("Point-to-zone crosswalk summary")
display(html_table(pl.from_pandas(crosswalk_summary), kind="reference"))

reference · 8 rows × 11 columns

basis_direction	market_region	weather_zone	location_count	locations	weather_zone_count	weather_zones	market_region_count	market_regions	basis_risk_flag	basis_risk_note
market_zone_to_weather_zones	CAISO	—	7	bakersfield, fresno, los_angeles, merced, sacramento, san_diego, san_jose	4	Central Coast Drainage, Sacramento Drainage, San Joaquin Drainage, South Coast Drainage	3	NP15, SP15, ZP26	True	All-CAISO index combines multiple market and weather-zone proxies; treat as a broad system exposure.
market_zone_to_weather_zones	NP15	—	2	sacramento, san_jose	2	Central Coast Drainage, Sacramento Drainage	1	NP15	True	Market zone mixes multiple weather-zone proxies; include dispersion or interaction terms.
market_zone_to_weather_zones	SP15	—	2	los_angeles, san_diego	1	South Coast Drainage	1	SP15	False	Market zone maps to one weather-zone proxy in this prototype.
market_zone_to_weather_zones	ZP26	—	3	bakersfield, fresno, merced	1	San Joaquin Drainage	1	ZP26	False	Market zone maps to one weather-zone proxy in this prototype.
weather_zone_to_market_zones	—	Central Coast Drainage	1	san_jose	1	Central Coast Drainage	1	NP15	False	Weather-zone proxy maps to one CAISO market zone in this prototype.
weather_zone_to_market_zones	—	Sacramento Drainage	1	sacramento	1	Sacramento Drainage	1	NP15	False	Weather-zone proxy maps to one CAISO market zone in this prototype.
weather_zone_to_market_zones	—	San Joaquin Drainage	3	bakersfield, fresno, merced	1	San Joaquin Drainage	1	ZP26	False	Weather-zone proxy maps to one CAISO market zone in this prototype.
weather_zone_to_market_zones	—	South Coast Drainage	2	los_angeles, san_diego	1	South Coast Drainage	1	SP15	False	Weather-zone proxy maps to one CAISO market zone in this prototype.

Point-to-zone crosswalk summary

reference · 3 rows × 5 columns

region	proxy_locations	weather_zone_count	weather_zones	weight_sum
NP15	sacramento, san_jose	2	Central Coast Drainage, Sacramento Drainage	0.360
SP15	los_angeles, san_diego	1	South Coast Drainage	0.410
ZP26	bakersfield, fresno, merced	1	San Joaquin Drainage	0.230

NP15 is split across Sacramento Drainage and Central Coast weather proxies, while SP15 and ZP26 map to one proxy each, so a single market-zone weather index can hide sub-zone exposure in northern California.

weather_region_d = weather_region_d.copy()
weather_region_d["date"] = pd.to_datetime(weather_region_d["date"])

weather_dispersion_summary = (
    weather_region_d.groupby("region")
    .agg(
        days=("date", "nunique"),
        avg_temp_dispersion_f=("temperature_f_std", "mean"),
        p95_temp_dispersion_f=("temperature_f_std", lambda values: values.quantile(0.95)),
        max_temp_dispersion_f=("temperature_f_std", "max"),
        avg_cdd65=("cdd65_from_weighted_temp", "mean"),
        p95_cdd65=("cdd65_from_weighted_temp", lambda values: values.quantile(0.95)),
        avg_hdd65=("hdd65_from_weighted_temp", "mean"),
        p95_hdd65=("hdd65_from_weighted_temp", lambda values: values.quantile(0.95)),
    )
    .round(3)
    .reset_index()
)

display(html_table(pl.from_pandas(weather_dispersion_summary)))

plot_df = weather_dispersion_summary[weather_dispersion_summary["region"].isin(["CAISO", "NP15", "SP15", "ZP26"])]
dispersion_src = ColumnDataSource(
    {
        "region": plot_df["region"].tolist(),
        "p95_temp_dispersion_f": plot_df["p95_temp_dispersion_f"].to_numpy(),
        "color": SERIES[: len(plot_df)],
    }
)
fig = figure(
    title="95th Percentile Temperature Dispersion by Weather Index",
    x_range=plot_df["region"].tolist(),
    y_axis_label="Temperature Dispersion (F)",
    height=340,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.vbar(x="region", top="p95_temp_dispersion_f", width=0.7, color="color", source=dispersion_src)
fig.add_tools(
    HoverTool(tooltips=[("region", "@region"), ("p95 dispersion (F)", "@p95_temp_dispersion_f{0.00}")])
)
show(style(fig))

4 rows × 9 columns

region	days	avg_temp_dispersion_f	p95_temp_dispersion_f	max_temp_dispersion_f	avg_cdd65	p95_cdd65	avg_hdd65	p95_hdd65
CAISO	2,008	4.633	9.532	12.84	3.678	14.12	5.308	16.61
NP15	2,008	2.337	5.649	9.30	2.956	13.19	6.572	19.39
SP15	2,008	1.310	3.290	7.77	2.420	10.37	4.067	13.50
ZP26	2,008	2.018	3.927	6.00	7.583	26.48	6.075	19.79

Within-zone temperature dispersion captures weather mismatch hidden by a regional average and is a candidate control for the outage model.

wz = weather_zone_d.copy()
wz["date"] = pd.to_datetime(wz["date"])
wz["market_region"] = wz["caiso_regions"].astype(str).str.split(", ")
wz_exploded = wz.explode("market_region")
wz_exploded = wz_exploded[wz_exploded["market_region"].isin(HUB_ORDER)]

weather_zone_spread = (
    wz_exploded.groupby(["date", "market_region"])
    .agg(
        weather_zone_count=("weather_zone", "nunique"),
        temp_zone_range_f=("temperature_f_mean_weighted", lambda values: values.max() - values.min()),
        cdd_zone_range=("cdd65_from_weighted_temp", lambda values: values.max() - values.min()),
        hdd_zone_range=("hdd65_from_weighted_temp", lambda values: values.max() - values.min()),
    )
    .reset_index()
)

zone_spread_summary = (
    weather_zone_spread.groupby("market_region")
    .agg(
        days=("date", "nunique"),
        avg_weather_zone_count=("weather_zone_count", "mean"),
        avg_temp_zone_range_f=("temp_zone_range_f", "mean"),
        p95_temp_zone_range_f=("temp_zone_range_f", lambda values: values.quantile(0.95)),
        max_temp_zone_range_f=("temp_zone_range_f", "max"),
        avg_cdd_zone_range=("cdd_zone_range", "mean"),
        p95_cdd_zone_range=("cdd_zone_range", lambda values: values.quantile(0.95)),
    )
    .round(3)
    .reset_index()
)

display(html_table(pl.from_pandas(zone_spread_summary)))

3 rows × 8 columns

market_region	days	avg_weather_zone_count	avg_temp_zone_range_f	p95_temp_zone_range_f	max_temp_zone_range_f	avg_cdd_zone_range	p95_cdd_zone_range
NP15	2,008	2.000	4.673	11.30	18.60	2.833	11.22
SP15	2,008	1.000	0.000	0.00	0.00	0.000	0.00
ZP26	2,008	1.000	0.000	0.00	0.00	0.000	0.00

plot_zone = zone_spread_summary.set_index("market_region").reindex(HUB_ORDER).reset_index()
zone_src = ColumnDataSource(
    {
        "market_region": plot_zone["market_region"].tolist(),
        "p95_temp_zone_range_f": plot_zone["p95_temp_zone_range_f"].to_numpy(),
        "label": [f"{value:.1f}" for value in plot_zone["p95_temp_zone_range_f"]],
    }
)
fig = figure(
    title="Weather-Zone Basis Inside CAISO Market Zones",
    x_range=plot_zone["market_region"].tolist(),
    x_axis_label="CAISO Market Zone",
    y_axis_label="p95 Daily Temp Range Across Weather Proxies (F)",
    height=340,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.vbar(x="market_region", top="p95_temp_zone_range_f", width=0.7, color=SERIES[0], alpha=0.85, source=zone_src)
fig.add_layout(
    LabelSet(x="market_region", y="p95_temp_zone_range_f", text="label", source=zone_src,
             y_offset=4, text_align="center", text_font_size="10px", text_color=SECONDARY)
)
show(style(fig))

NP15 has nonzero weather-zone spread because it blends inland and coastal proxy weather. SP15 and ZP26 each map to one NOAA proxy division, so their within-market spread is mechanically lower.

3. Locational Price Basis Across CAISO Hubs

da = market["da_lmp"].copy()
da["interval_start_utc"] = to_utc_hour(da["interval_start_utc"])
da["region"] = da["location"].map(ZONE_LOCATION_MAP)
da = da[da["region"].isin(HUB_ORDER)]
da_hub = da.groupby(["interval_start_utc", "region"], as_index=False).agg(da_lmp=("lmp", "mean"), da_congestion=("congestion", "mean"))

rt = market["rt15_lmp"].copy()
rt["interval_start_utc"] = to_utc_hour(rt["interval_start_utc"])
rt["region"] = rt["location"].map(ZONE_LOCATION_MAP)
rt = rt[rt["region"].isin(HUB_ORDER)]
rt_hub = rt.groupby(["interval_start_utc", "region"], as_index=False).agg(rt_lmp=("lmp", "mean"), rt_congestion=("congestion", "mean"))

price_hourly = da_hub.merge(rt_hub, on=["interval_start_utc", "region"], how="inner")
price_hourly["dart_da_minus_rt"] = price_hourly["da_lmp"] - price_hourly["rt_lmp"]
price_hourly["dart_rt_minus_da"] = price_hourly["rt_lmp"] - price_hourly["da_lmp"]

pivot_da = price_hourly.pivot(index="interval_start_utc", columns="region", values="da_lmp")
pivot_rt = price_hourly.pivot(index="interval_start_utc", columns="region", values="rt_lmp")
price_basis_hourly = pd.DataFrame(index=pivot_da.index)
price_basis_hourly["da_hub_range"] = pivot_da.max(axis=1) - pivot_da.min(axis=1)
price_basis_hourly["rt_hub_range"] = pivot_rt.max(axis=1) - pivot_rt.min(axis=1)
price_basis_hourly["da_sp15_minus_np15"] = pivot_da.get("SP15") - pivot_da.get("NP15")
price_basis_hourly["da_zp26_minus_sp15"] = pivot_da.get("ZP26") - pivot_da.get("SP15")
price_basis_hourly["rt_sp15_minus_np15"] = pivot_rt.get("SP15") - pivot_rt.get("NP15")
price_basis_hourly["rt_zp26_minus_sp15"] = pivot_rt.get("ZP26") - pivot_rt.get("SP15")
price_basis_hourly = price_basis_hourly.reset_index()

weather_h_caiso = weather_region_h[weather_region_h["region"].eq("CAISO")].copy()
weather_h_caiso["interval_start_utc"] = to_utc_hour(weather_h_caiso["timestamp_utc"])
weather_h_caiso = weather_h_caiso[["interval_start_utc", "temperature_f_weighted", "temperature_f_std", "cdh65_from_weighted_temp", "hdh65_from_weighted_temp"]]
price_basis_weather = price_basis_hourly.merge(weather_h_caiso, on="interval_start_utc", how="inner")

price_basis_summary = price_basis_hourly[["da_hub_range", "rt_hub_range", "da_sp15_minus_np15", "rt_sp15_minus_np15", "da_zp26_minus_sp15", "rt_zp26_minus_sp15"]].describe(percentiles=[0.5, 0.9, 0.95, 0.99]).T.round(3)
price_weather_corr = pd.DataFrame([
    {"basis_metric": metric, "corr_with_caiso_cdh65": price_basis_weather[metric].corr(price_basis_weather["cdh65_from_weighted_temp"]), "corr_with_temp_dispersion": price_basis_weather[metric].corr(price_basis_weather["temperature_f_std"])}
    for metric in ["da_hub_range", "rt_hub_range", "da_sp15_minus_np15", "rt_sp15_minus_np15", "da_zp26_minus_sp15", "rt_zp26_minus_sp15"]
]).round(4)

print("Hub price basis summary")
display(html_table(pl.from_pandas(price_basis_summary.reset_index())))
print("Hub price basis vs CAISO weather stress / dispersion")
display(html_table(pl.from_pandas(price_weather_corr)))

Hub price basis summary

6 rows × 10 columns

index	count	mean	std	min	50%	90%	95%	99%	max
da_hub_range	28,895.00	8.62	14.87	0.00	2.963	24.60	34.77	59.43	497.27
rt_hub_range	28,895.00	10.67	24.14	0.00	3.498	28.93	43.32	81.56	1,132.65
da_sp15_minus_np15	28,895.00	−5.14	15.17	−150.41	−1.677	1.93	4.43	23.46	453.21
rt_sp15_minus_np15	28,895.00	−6.15	24.04	−693.08	−1.680	2.67	6.93	28.71	1,049.83
da_zp26_minus_sp15	28,895.00	−0.89	10.36	−497.27	−0.152	2.66	5.49	14.69	52.92
rt_zp26_minus_sp15	28,895.00	−0.59	17.65	−1,132.65	0.000	4.82	8.59	21.15	101.67

Hub price basis vs CAISO weather stress / dispersion

6 rows × 3 columns

basis_metric	corr_with_caiso_cdh65	corr_with_temp_dispersion
da_hub_range	−0.179	−0.135
rt_hub_range	−0.125	−0.105
da_sp15_minus_np15	0.269	0.220
rt_sp15_minus_np15	0.191	0.149
da_zp26_minus_sp15	−0.084	−0.101
rt_zp26_minus_sp15	−0.062	−0.063

price_basis_daily = price_basis_weather.copy()
price_basis_daily["date"] = pd.to_datetime(price_basis_daily["interval_start_utc"]).dt.date
price_basis_daily = (
    price_basis_daily.groupby("date", as_index=False)
    .agg(
        da_hub_range=("da_hub_range", "mean"),
        rt_hub_range=("rt_hub_range", "mean"),
        caiso_cdh65=("cdh65_from_weighted_temp", "mean"),
        temp_dispersion=("temperature_f_std", "mean"),
    )
)

fig = figure(
    title="Hub Price Basis vs CAISO Cooling-Degree Stress",
    x_axis_label="Mean Hourly CDH65, CAISO Index",
    y_axis_label="Mean Hub LMP Range ($/MWh)",
    height=360,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.scatter(price_basis_daily["caiso_cdh65"].to_numpy(), price_basis_daily["da_hub_range"].to_numpy(),
            size=6, color=SERIES[0], alpha=0.55, legend_label="DA")
fig.scatter(price_basis_daily["caiso_cdh65"].to_numpy(), price_basis_daily["rt_hub_range"].to_numpy(),
            size=6, color=SERIES[1], alpha=0.45, legend_label="RT")
fig.legend.location = "top_left"
show(style(fig))

fig = figure(
    title="RT Hub Basis vs Weather Dispersion",
    x_axis_label="Mean Temp Dispersion (F)",
    y_axis_label="Mean RT Hub LMP Range ($/MWh)",
    height=360,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.scatter(price_basis_daily["temp_dispersion"].to_numpy(), price_basis_daily["rt_hub_range"].to_numpy(),
            size=6, color=SERIES[1], alpha=0.55)
show(style(fig))

Hub-price divergence determines whether weather stress translates into different financial outcomes across settlement locations.

4. DA/RT Temporal Basis: DART by Hub

dart_summary = (
    price_hourly.groupby("region")
    .agg(
        hours=("interval_start_utc", "nunique"),
        mean_da_minus_rt=("dart_da_minus_rt", "mean"),
        mean_abs_da_minus_rt=("dart_da_minus_rt", lambda values: values.abs().mean()),
        p95_abs_da_minus_rt=("dart_da_minus_rt", lambda values: values.abs().quantile(0.95)),
        min_da_minus_rt=("dart_da_minus_rt", "min"),
        max_da_minus_rt=("dart_da_minus_rt", "max"),
    )
    .round(3)
    .reset_index()
)

weather_region_h2 = weather_region_h.copy()
weather_region_h2["interval_start_utc"] = to_utc_hour(weather_region_h2["timestamp_utc"])
weather_region_h2 = weather_region_h2[weather_region_h2["region"].isin(HUB_ORDER)][["interval_start_utc", "region", "temperature_f_weighted", "temperature_f_std", "cdh65_from_weighted_temp", "hdh65_from_weighted_temp"]]
dart_weather = price_hourly.merge(weather_region_h2, on=["interval_start_utc", "region"], how="inner")
dart_corr = (
    dart_weather.groupby("region")
    .apply(lambda g: pd.Series({
        "corr_cdh65_dart_da_minus_rt": g["cdh65_from_weighted_temp"].corr(g["dart_da_minus_rt"]),
        "corr_temp_dispersion_dart": g["temperature_f_std"].corr(g["dart_da_minus_rt"]),
        "corr_temp_dart_da_minus_rt": g["temperature_f_weighted"].corr(g["dart_da_minus_rt"]),
    }), include_groups=False)
    .round(4)
    .reset_index()
)

display(html_table(pl.from_pandas(dart_summary)))
display(html_table(pl.from_pandas(dart_corr)))

3 rows × 7 columns

region	hours	mean_da_minus_rt	mean_abs_da_minus_rt	p95_abs_da_minus_rt	min_da_minus_rt	max_da_minus_rt
NP15	28,895	0.684	8.450	22.47	−982.46	681.68
SP15	28,895	1.695	8.243	21.03	−1,034.69	833.63
ZP26	28,895	1.391	7.753	20.86	−1,010.53	687.18

3 rows × 4 columns

region	corr_cdh65_dart_da_minus_rt	corr_temp_dispersion_dart	corr_temp_dart_da_minus_rt
NP15	0.016	0.006	0.012
SP15	0.008	0.002	0.011
ZP26	−0.003	−0.025	−0.004

dart_plot = dart_summary.set_index("region").reindex(HUB_ORDER).reset_index()
dart_src = ColumnDataSource(
    {
        "region": dart_plot["region"].tolist(),
        "p95_abs_da_minus_rt": dart_plot["p95_abs_da_minus_rt"].to_numpy(),
        "label": [f"{value:.1f}" for value in dart_plot["p95_abs_da_minus_rt"]],
        "color": SERIES[: len(dart_plot)],
    }
)
fig = figure(
    title="DA/RT Basis by Trading Hub",
    x_range=dart_plot["region"].tolist(),
    y_axis_label="p95 Absolute DA - RT ($/MWh)",
    height=340,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.vbar(x="region", top="p95_abs_da_minus_rt", width=0.7, color="color", alpha=0.85, source=dart_src)
fig.add_layout(
    LabelSet(x="region", y="p95_abs_da_minus_rt", text="label", source=dart_src,
             y_offset=4, text_align="center", text_font_size="10px", text_color=SECONDARY)
)
show(style(fig))

Day-ahead and real-time prices diverge at the same hub, so DART belongs in the model separately from price levels and temperature controls.

5. Fuel/Gas Basis: Fuel Regions vs Electric Hubs

fuel_prices = market["fuel_prices"].copy()
fuel_prices["interval_start_utc"] = to_utc_hour(fuel_prices["interval_start_utc"])
fuel_regions = market["fuel_regions"].copy()
fuel = fuel_prices.merge(fuel_regions, on="fuel_region_id", how="left")
fuel_ciso = fuel[fuel["balancing_authority"].eq("CISO")].copy()

fuel_region_summary = (
    fuel_regions.groupby("balancing_authority", dropna=False)
    .agg(
        fuel_regions=("fuel_region_id", "nunique"),
        pricing_hubs=("pricing_hub", "nunique"),
        missing_pricing_hub=("pricing_hub", lambda values: values.isna().sum()),
    )
    .sort_values("fuel_regions", ascending=False)
    .head(12)
    .reset_index()
)

if len(fuel_ciso) > 0:
    fuel_basis_hourly = (
        fuel_ciso.groupby("interval_start_utc")
        .agg(
            ciso_fuel_region_count=("fuel_region_id", "nunique"),
            ciso_pricing_hub_count=("pricing_hub", "nunique"),
            ciso_gas_price_mean=("price", "mean"),
            ciso_gas_price_std=("price", "std"),
            ciso_gas_price_range=("price", lambda values: values.max() - values.min()),
            ciso_gas_price_p90_minus_p10=("price", lambda values: values.quantile(0.9) - values.quantile(0.1)),
        )
        .reset_index()
    )
    fuel_basis_summary = fuel_basis_hourly[["ciso_fuel_region_count", "ciso_pricing_hub_count", "ciso_gas_price_mean", "ciso_gas_price_std", "ciso_gas_price_range", "ciso_gas_price_p90_minus_p10"]].describe(percentiles=[0.5, 0.9, 0.95, 0.99]).T.round(3)
else:
    fuel_basis_hourly = pd.DataFrame()
    fuel_basis_summary = pd.DataFrame()

print("Fuel region inventory by BA")
display(html_table(pl.from_pandas(fuel_region_summary)))
print("CISO fuel/gas price dispersion over time")
display(html_table(pl.from_pandas(fuel_basis_summary.reset_index())))

Fuel region inventory by BA

12 rows × 4 columns

balancing_authority	fuel_regions	pricing_hubs	missing_pricing_hub
CISO	57	5	1
PSEI	19	2	1
AZPS	14	5	1
SRP	9	1	1
TEP	8	2	1
PNM	7	3	1
BANCSMUD	7	1	1
AVA	5	1	1
WALC	5	4	1
LDWP	5	2	1
NEVP	5	3	1
BANC	5	1	1

CISO fuel/gas price dispersion over time

6 rows × 10 columns

index	count	mean	std	min	50%	90%	95%	99%	max
ciso_fuel_region_count	75,832.00	52.39	5.389	25.00	53.00	57.00	57.00	57.00	57.00
ciso_pricing_hub_count	75,832.00	4.58	0.526	3.00	5.00	5.00	5.00	5.00	5.00
ciso_gas_price_mean	75,832.00	6.11	5.637	2.49	5.06	9.09	10.95	23.11	112.67
ciso_gas_price_std	75,832.00	1.43	2.262	0.47	1.20	1.75	2.28	4.89	57.37
ciso_gas_price_range	75,832.00	5.07	6.239	1.47	4.46	6.49	8.25	17.35	151.51
ciso_gas_price_p90_minus_p10	75,832.00	3.84	5.482	1.22	3.37	4.70	5.64	11.05	140.00

if not fuel_basis_hourly.empty:
    fuel_basis_plot = fuel_basis_hourly.copy()
    fuel_basis_plot["date"] = pd.to_datetime(fuel_basis_plot["interval_start_utc"]).dt.date
    fuel_basis_daily = fuel_basis_plot.groupby("date", as_index=False).agg(
        ciso_gas_price_range=("ciso_gas_price_range", "mean"),
        ciso_gas_price_std=("ciso_gas_price_std", "mean"),
        ciso_pricing_hub_count=("ciso_pricing_hub_count", "max"),
    )
    fig = figure(
        title="CISO Fuel-Region Gas Price Basis Over Time",
        x_axis_type="datetime",
        x_axis_label="Date",
        y_axis_label="Daily Mean Max-Min Gas Price ($/MMBtu)",
        height=360,
        sizing_mode="stretch_width",
        toolbar_location="above",
    )
    fig.line(pd.to_datetime(fuel_basis_daily["date"]).to_numpy(), fuel_basis_daily["ciso_gas_price_range"].to_numpy(),
             color=SERIES[0], line_width=1.4)
    show(style(fig))
else:
    print("No CISO fuel price rows available for plotting.")

Power and gas legs use different geographic references, so the spark-spread specification must define the gas hub explicitly.

6. Load/TAC Geography Basis

load_tac = market["load_hourly_tac"].copy()
load_tac["interval_start_utc"] = to_utc_hour(load_tac["interval_start_utc"])
load_tac["date"] = pd.to_datetime(load_tac["interval_start_utc"]).dt.date

# `tac_area_name` mixes three hierarchy levels: the five CAISO TAC areas, the
# `CA ISO-TAC` total that already contains them, and about twenty-five
# non-CAISO WECC balancing authorities. Summing all of them double-counts
# CAISO and adds load that is not CAISO's, so a share off that denominator is
# not a CAISO share. Restrict to the five before any share is taken.
CAISO_TAC_AREAS = ["PGE-TAC", "SCE-TAC", "SDGE-TAC", "MWD-TAC", "VEA-TAC"]

load_tac_daily = (
    load_tac[load_tac["tac_area_name"].isin(CAISO_TAC_AREAS)]
    .groupby(["date", "tac_area_name"], as_index=False)
    .agg(load_mw_mean=("load", "mean"))
)
load_tac_daily["system_load_mw_mean_from_tac"] = load_tac_daily.groupby("date")["load_mw_mean"].transform("sum")
load_tac_daily["load_share"] = load_tac_daily["load_mw_mean"] / load_tac_daily["system_load_mw_mean_from_tac"]

top_tac_share = (
    load_tac_daily.groupby("tac_area_name")
    .agg(avg_load_mw=("load_mw_mean", "mean"), avg_load_share=("load_share", "mean"), p95_load_share=("load_share", lambda values: values.quantile(0.95)))
    .sort_values("avg_load_share", ascending=False)
    .head(15)
    .round(4)
    .reset_index()
)

load_concentration = (
    load_tac_daily.groupby("date")
    .agg(
        tac_count=("tac_area_name", "nunique"),
        max_tac_load_share=("load_share", "max"),
        top5_tac_load_share=("load_share", lambda values: values.nlargest(5).sum()),
        load_hhi=("load_share", lambda values: (values ** 2).sum()),
    )
    .reset_index()
)

load_concentration_summary = load_concentration[["tac_count", "max_tac_load_share", "top5_tac_load_share", "load_hhi"]].describe(percentiles=[0.5, 0.9, 0.95, 0.99]).T.round(4)

display(html_table(pl.from_pandas(top_tac_share)))
display(html_table(pl.from_pandas(load_concentration_summary.reset_index())))

5 rows × 4 columns

tac_area_name	avg_load_mw	avg_load_share	p95_load_share
SCE-TAC	11,724.83	0.458	0.493
PGE-TAC	11,342.53	0.446	0.473
SDGE-TAC	2,218.83	0.087	0.095
MWD-TAC	156.95	0.006	0.010
VEA-TAC	68.95	0.003	0.004

4 rows × 10 columns

index	count	mean	std	min	50%	90%	95%	99%	max
tac_count	3,131.00	4.972	0.164	4.000	5.000	5.000	5.000	5.000	5.000
max_tac_load_share	3,131.00	0.467	0.013	0.447	0.463	0.485	0.493	0.507	0.528
top5_tac_load_share	3,131.00	1.000	0.000	1.000	1.000	1.000	1.000	1.000	1.000
load_hhi	3,131.00	0.417	0.004	0.403	0.417	0.423	0.424	0.428	0.433

tac_sorted = top_tac_share.sort_values("avg_load_share")
fig = figure(
    title="Largest TAC Load Shares",
    y_range=tac_sorted["tac_area_name"].tolist(),
    x_axis_label="Average Share of Summed TAC Load",
    height=420,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.hbar(y=tac_sorted["tac_area_name"].tolist(), right=tac_sorted["avg_load_share"].to_numpy(),
         height=0.7, color=SERIES[0])
fig.xaxis.formatter = NumeralTickFormatter(format="0%")
show(style(fig))

hist_counts, hist_edges = np.histogram(load_concentration["max_tac_load_share"], bins=25)
fig = figure(
    title="Daily Concentration of Load Geography",
    x_axis_label="Largest TAC Load Share",
    y_axis_label="Days",
    height=360,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.quad(top=hist_counts, bottom=0, left=hist_edges[:-1], right=hist_edges[1:],
         color=SERIES[1], alpha=0.8, line_color="white")
show(style(fig))

CAISO load arrives across five TAC areas, two of which carry about ninety per cent of it. Without a TAC-to-hub/weather crosswalk, system load is a control rather than a locational exposure variable.

7. Outage Basis: System Aggregate vs Resource-Level Detail

outages = market["outages_hourly"].copy()
outages["interval_start_utc"] = to_utc_hour(outages["interval_start_utc"])
outages["date"] = pd.to_datetime(outages["interval_start_utc"]).dt.date
outage_daily = (
    outages.groupby("date", as_index=False)
    .agg(
        forced_curtailment_mw_mean=("forced_curtailment_mw", "mean"),
        forced_curtailment_mw_max=("forced_curtailment_mw", "max"),
        planned_curtailment_mw_mean=("planned_curtailment_mw", "mean"),
        planned_curtailment_mw_max=("planned_curtailment_mw", "max"),
    )
)

weather_caiso_daily = weather_region_d[weather_region_d["region"].eq("CAISO")].copy()
weather_caiso_daily["date"] = pd.to_datetime(weather_caiso_daily["date"]).dt.date
weather_outage = weather_caiso_daily.merge(outage_daily, on="date", how="inner")

outage_corr = pd.DataFrame([
    {
        "weather_metric": metric,
        "corr_forced_mean_mw": weather_outage[metric].corr(weather_outage["forced_curtailment_mw_mean"]),
        "corr_planned_mean_mw": weather_outage[metric].corr(weather_outage["planned_curtailment_mw_mean"]),
        "rows": len(weather_outage),
    }
    for metric in ["temperature_f_mean_weighted", "temperature_f_std", "cdd65_from_weighted_temp", "hdd65_from_weighted_temp"]
]).round(4)

curtail = market["curtailment_detail"].copy()
reason_summary = (
    curtail.groupby(["outage_type", "nature_of_work"], dropna=False)
    .agg(
        records=("outage_mrid", "count"),
        curtailment_mw_sum=("curtailment_mw", "sum"),
        curtailment_mw_mean=("curtailment_mw", "mean"),
        resources=("resource_id", "nunique"),
    )
    .sort_values("curtailment_mw_sum", ascending=False)
    .head(20)
    .round(2)
    .reset_index()
)

weather_reason_mask = curtail["nature_of_work"].astype(str).str.contains("AMBIENT|TEMP|WEATHER|FUEL", case=False, na=False)
weather_reason_summary = (
    curtail[weather_reason_mask]
    .groupby(["outage_type", "nature_of_work"], dropna=False)
    .agg(records=("outage_mrid", "count"), curtailment_mw_sum=("curtailment_mw", "sum"), resources=("resource_id", "nunique"))
    .sort_values("curtailment_mw_sum", ascending=False)
    .round(2)
    .reset_index()
)

print("System weather vs aggregate outage MW")
display(html_table(pl.from_pandas(outage_corr)))
print("Largest outage reason categories")
display(html_table(pl.from_pandas(reason_summary)))
print("Weather/ambient/fuel-adjacent reason categories")
display(html_table(pl.from_pandas(weather_reason_summary.head(20))))

System weather vs aggregate outage MW

4 rows × 4 columns

weather_metric	corr_forced_mean_mw	corr_planned_mean_mw	rows
temperature_f_mean_weighted	0.137	−0.472	1,840
temperature_f_std	0.066	−0.536	1,840
cdd65_from_weighted_temp	0.068	−0.568	1,840
hdd65_from_weighted_temp	−0.170	0.303	1,840

Largest outage reason categories

20 rows × 6 columns

outage_type	nature_of_work	records	curtailment_mw_sum	curtailment_mw_mean	resources
FORCED	PLANT_TROUBLE	553,926	26,849,090.07	48.47	1,376
FORCED	AMBIENT_DUE_TO_TEMP	1,899,371	23,441,905.10	12.34	335
FORCED	AMBIENT_NOT_DUE_TO_TEMP	564,499	19,459,244.56	34.47	429
PLANNED	PLANT_MAINTENANCE	111,913	12,549,905.10	112.14	967
FORCED	NEW_GENERATOR_TEST_ENERGY	58,083	9,177,498.15	158.01	304
FORCED	PLANT_MAINTENANCE	170,153	8,626,558.68	50.70	1,282
FORCED	UNIT_TESTING	33,994	6,320,789.49	185.94	791
FORCED	ENVIRONMENTAL_RESTRICTIONS	36,820	5,520,500.29	149.93	288
FORCED	TRANSMISSION_INDUCED	59,084	4,126,193.11	69.84	1,167
FORCED	AMBIENT_DUE_TO_FUEL_INSUFFICIENCY	75,499	1,948,760.49	25.81	327
FORCED	TRANSITIONAL_LIMITATION	4,166	1,444,488.49	346.73	96
PLANNED	TRANSMISSION_INDUCED	19,082	1,197,632.22	62.76	871
PLANNED	PLANT_TROUBLE	11,797	782,833.97	66.36	125
FORCED	METERING_TELEMETRY	17,991	756,309.31	42.04	844
PLANNED	AMBIENT_DUE_TO_FUEL_INSUFFICIENCY	3,149	501,405.52	159.23	62
PLANNED	NEW_GENERATOR_TEST_ENERGY	6,438	471,670.57	73.26	71
PLANNED	ENVIRONMENTAL_RESTRICTIONS	1,670	409,815.55	245.40	15
PLANNED	METERING_TELEMETRY	954	356,758.64	373.96	150
FORCED	SHORT_TERM_USE_LIMIT_REACHED	1,759	292,004.47	166.01	209
PLANNED	UNIT_TESTING	3,639	289,558.25	79.57	300

Weather/ambient/fuel-adjacent reason categories

6 rows × 5 columns

outage_type	nature_of_work	records	curtailment_mw_sum	resources
FORCED	AMBIENT_DUE_TO_TEMP	1,899,371	23,441,905.10	335
FORCED	AMBIENT_NOT_DUE_TO_TEMP	564,499	19,459,244.56	429
FORCED	AMBIENT_DUE_TO_FUEL_INSUFFICIENCY	75,499	1,948,760.49	327
PLANNED	AMBIENT_DUE_TO_FUEL_INSUFFICIENCY	3,149	501,405.52	62
PLANNED	AMBIENT_NOT_DUE_TO_TEMP	568	77,035.01	19
PLANNED	AMBIENT_DUE_TO_TEMP	389	15,397.30	3

fig = figure(
    title="Forced Outage MW vs CAISO CDD",
    x_axis_label="Daily CDD65, CAISO Index",
    y_axis_label="Mean Forced Curtailment MW",
    height=360,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.scatter(weather_outage["cdd65_from_weighted_temp"].to_numpy(),
            weather_outage["forced_curtailment_mw_mean"].to_numpy(),
            size=6, color=SERIES[0], alpha=0.55)
show(style(fig))

fig = figure(
    title="Planned Outage MW vs CAISO CDD",
    x_axis_label="Daily CDD65, CAISO Index",
    y_axis_label="Mean Planned Curtailment MW",
    height=360,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.scatter(weather_outage["cdd65_from_weighted_temp"].to_numpy(),
            weather_outage["planned_curtailment_mw_mean"].to_numpy(),
            size=6, color=SERIES[1], alpha=0.55)
show(style(fig))

The aggregate outage file is system-level MW and the detailed report is keyed by resource name/ID rather than EIA plant ID, so these correlations screen for a weather-outage mechanism rather than establish one.

8. Plant Weather Proxy Basis

if imputed_plants.empty:
    raise FileNotFoundError("Missing caiso_generator_weather_imputed.parquet. Run notebooks/caiso_generator_weather_map.ipynb first.")

plant_distance_summary = (
    imputed_plants.groupby("region")
    .agg(
        plants=("plant_id", "count"),
        capacity_mw=("nameplate_capacity_mw", "sum"),
        avg_proxy_distance_km=("distance_to_weather_proxy_km", "mean"),
        capacity_weighted_proxy_distance_km=("distance_to_weather_proxy_km", lambda values: np.average(values, weights=imputed_plants.loc[values.index, "nameplate_capacity_mw"])),
        p90_proxy_distance_km=("distance_to_weather_proxy_km", lambda values: values.quantile(0.9)),
        max_proxy_distance_km=("distance_to_weather_proxy_km", "max"),
        weather_proxy_count=("nearest_weather_location", "nunique"),
        weather_zone_count=("noaa_climate_division_proxy", "nunique"),
    )
    .round(2)
    .reset_index()
)

plant_capacity_by_weather_zone = (
    imputed_plants.groupby(["region", "noaa_climate_division_proxy"])
    .agg(plants=("plant_id", "count"), capacity_mw=("nameplate_capacity_mw", "sum"), avg_recent_cdd65=("recent_cdd65_mean", "mean"))
    .sort_values(["region", "capacity_mw"], ascending=[True, False])
    .round(2)
    .reset_index()
)

tech_heat_exposure = (
    imputed_plants.groupby(["primary_technology_group", "heat_stress_label"], observed=False)
    .agg(plants=("plant_id", "count"), capacity_mw=("nameplate_capacity_mw", "sum"))
    .sort_values("capacity_mw", ascending=False)
    .round(2)
    .reset_index()
    .head(20)
)

display(html_table(pl.from_pandas(plant_distance_summary)))
display(html_table(pl.from_pandas(plant_capacity_by_weather_zone)))
display(html_table(pl.from_pandas(tech_heat_exposure)))

3 rows × 9 columns

region	plants	capacity_mw	avg_proxy_distance_km	capacity_weighted_proxy_distance_km	p90_proxy_distance_km	max_proxy_distance_km	weather_proxy_count	weather_zone_count
NP15	433	20,064.90	90.04	86.55	211.11	346.55	2	2
SP15	616	36,891.90	80.82	113.61	145.51	311.42	2	1
ZP26	612	34,940.30	63.90	75.35	111.77	191.09	3	1

4 rows × 5 columns

region	noaa_climate_division_proxy	plants	capacity_mw	avg_recent_cdd65
NP15	Sacramento Drainage	266	13,024.40	11.76
NP15	Central Coast Drainage	167	7,040.50	4.24
SP15	South Coast Drainage	616	36,891.90	12.83
ZP26	San Joaquin Drainage	612	34,940.30	23.71

20 rows × 4 columns

primary_technology_group	heat_stress_label	plants	capacity_mw
Natural gas	Hot CDD	144	20,282.70
Solar	Extreme CDD	360	14,021.00
Solar	Hot CDD	396	13,570.50
Battery	Hot CDD	104	6,758.80
Natural gas	Extreme CDD	64	6,714.60
Natural gas	Warm CDD	60	5,398.90
Hydro	Extreme CDD	67	4,532.30
Hydro	Hot CDD	128	4,284.10
Wind	Extreme CDD	51	3,656.40
Battery	Extreme CDD	36	3,097.70
Wind	Hot CDD	45	2,347.80
Nuclear	Extreme CDD	1	2,323.00
Geothermal	Hot CDD	7	1,790.40
Other	Hot CDD	54	820.20
Battery	Warm CDD	8	673.10
Oil	Warm CDD	9	381.10
Geothermal	Extreme CDD	7	356.60
Wind	Warm CDD	14	339.60
Other	Extreme CDD	23	171.40
Solar	Warm CDD	56	169.80

capacity_pivot = plant_capacity_by_weather_zone.pivot(index="region", columns="noaa_climate_division_proxy", values="capacity_mw").fillna(0)
proxy_cols = capacity_pivot.columns.tolist()
capacity_src = ColumnDataSource(
    {
        "region": capacity_pivot.index.tolist(),
        **{col: capacity_pivot[col].to_numpy() for col in proxy_cols},
    }
)
fig = figure(
    title="Mapped Generator Capacity by Market Zone and Weather Proxy",
    x_range=capacity_pivot.index.tolist(),
    x_axis_label="CAISO Market Zone",
    y_axis_label="Nameplate MW",
    height=380,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.vbar_stack(proxy_cols, x="region", width=0.7, color=SERIES[: len(proxy_cols)],
               source=capacity_src, legend_label=proxy_cols)
fig.yaxis.formatter = NumeralTickFormatter(format="0,0")
fig.legend.location = "top_right"
fig.legend.title = "Weather Proxy"
show(style(fig))

fig = figure(
    title="Plant-Weather Proxy Distance by Market Zone",
    x_range=plant_distance_summary["region"].tolist(),
    x_axis_label="CAISO Market Zone",
    y_axis_label="Capacity-Weighted Distance (km)",
    height=380,
    sizing_mode="stretch_width",
    toolbar_location="above",
)
fig.vbar(x=plant_distance_summary["region"].tolist(),
         top=plant_distance_summary["capacity_weighted_proxy_distance_km"].to_numpy(),
         width=0.7, color=SERIES[0], alpha=0.85)
show(style(fig))

The plant layer carries coordinates, technology, capacity, both zone proxies, and weather exposure, so it is the best available basis for a resource-level outage model. The residual basis is proxy distance and the missing resource-to-plant mapping for outage records.

Takeaway

def safe_value(value, fmt="{:.2f}"):
    if pd.isna(value):
        return "n/a"
    return fmt.format(float(value))

np15_row = zone_spread_summary[zone_spread_summary["market_region"].eq("NP15")]
weather_metric = "NP15 p95 weather-zone temp range = " + safe_value(np15_row["p95_temp_zone_range_f"].iloc[0] if len(np15_row) else np.nan) + " F"
price_metric = "p95 DA hub range = " + safe_value(price_basis_summary.loc["da_hub_range", "95%"] if "da_hub_range" in price_basis_summary.index else np.nan) + " $/MWh"
dart_metric = "p95 abs DART = " + safe_value(dart_summary["p95_abs_da_minus_rt"].max() if len(dart_summary) else np.nan) + " $/MWh"
fuel_metric = "p95 CISO fuel price range = " + safe_value(fuel_basis_summary.loc["ciso_gas_price_range", "95%"] if "ciso_gas_price_range" in fuel_basis_summary.index else np.nan) + " $/MMBtu"
# The top-5 share over five areas is 100% by construction, so the register
# carries the largest single TAC's share instead.
load_metric = "median largest-TAC load share = " + safe_value(load_concentration_summary.loc["max_tac_load_share", "50%"] if "max_tac_load_share" in load_concentration_summary.index else np.nan, "{:.1%}")
outage_metric = "weather/outage corr rows = " + str(len(weather_outage))
plant_metric = "capacity-weighted proxy distance max by region = " + safe_value(plant_distance_summary["capacity_weighted_proxy_distance_km"].max() if len(plant_distance_summary) else np.nan) + " km"

basis_register = pd.DataFrame([
    {
        "basis_risk_source": "Weather zone vs market hub",
        "measured_evidence": weather_metric,
        "modeling_implication": "Use market-zone weather for hub joins, but include dispersion/weather-zone mismatch controls.",
        "negative_binomial_relevance": "CDD/HDD effects may differ by weather-zone mixture; include interactions or random effects if count data is zonal/resource-level.",
        "spark_spread_relevance": "Temperature payoff may not match the hub where power/gas spread is settled.",
    },
    {
        "basis_risk_source": "Locational power price basis",
        "measured_evidence": price_metric,
        "modeling_implication": "Keep NP15/SP15/ZP26 price legs separate instead of collapsing all CAISO prices.",
        "negative_binomial_relevance": "Price stress can condition outage/reporting behavior, but should be hub-specific.",
        "spark_spread_relevance": "Directly affects spread payoff because power leg settlement differs by hub.",
    },
    {
        "basis_risk_source": "Day-ahead vs real-time basis",
        "measured_evidence": dart_metric,
        "modeling_implication": "Model DART separately from level prices and test weather x DART interactions.",
        "negative_binomial_relevance": "RT scarcity/outage timing may matter more than daily average weather.",
        "spark_spread_relevance": "A DA spark spread and RT spread are different exposures.",
    },
    {
        "basis_risk_source": "Fuel region vs electric hub",
        "measured_evidence": fuel_metric,
        "modeling_implication": "Define the gas leg explicitly; avoid one generic gas price unless justified.",
        "negative_binomial_relevance": "Fuel insufficiency outage reasons may depend on local gas constraints.",
        "spark_spread_relevance": "Directly affects spark spread because gas basis changes heat-rate economics.",
    },
    {
        "basis_risk_source": "TAC load vs hub/weather geography",
        "measured_evidence": load_metric,
        "modeling_implication": "Use system load as control; build TAC-to-zone mapping later for locational demand exposure.",
        "negative_binomial_relevance": "Load is a key exposure/tightness control, but current geography is not hub-clean.",
        "spark_spread_relevance": "Load drives price, but location of load affects congestion and hub spreads.",
    },
    {
        "basis_risk_source": "Outage aggregate vs resource/plant geography",
        "measured_evidence": outage_metric,
        "modeling_implication": "Current outage/weather results are descriptive until resource-to-plant or node mapping exists.",
        "negative_binomial_relevance": "Final NB target should be counts/events with exposure offsets; current MW aggregate is a proxy.",
        "spark_spread_relevance": "Outages affect scarcity/prices, but we need location to price the spread impact cleanly.",
    },
    {
        "basis_risk_source": "Plant weather proxy distance",
        "measured_evidence": plant_metric,
        "modeling_implication": "Track plant-weather proxy assignment and proxy distance as quality/control variables.",
        "negative_binomial_relevance": "Plant-level weather exposure can support resource-level outage counts once IDs are mapped.",
        "spark_spread_relevance": "Generator availability and heat exposure can condition marginal fuel/price regimes.",
    },
])

display(html_table(pl.from_pandas(basis_register)))

7 rows × 5 columns

basis_risk_source	measured_evidence	modeling_implication	negative_binomial_relevance	spark_spread_relevance
Weather zone vs market hub	NP15 p95 weather-zone temp range = 11.30 F	Use market-zone weather for hub joins, but include dispersion/weather-zone mismatch controls.	CDD/HDD effects may differ by weather-zone mixture; include interactions or random effects if count data is zonal/resource-level.	Temperature payoff may not match the hub where power/gas spread is settled.
Locational power price basis	p95 DA hub range = 34.77 $/MWh	Keep NP15/SP15/ZP26 price legs separate instead of collapsing all CAISO prices.	Price stress can condition outage/reporting behavior, but should be hub-specific.	Directly affects spread payoff because power leg settlement differs by hub.
Day-ahead vs real-time basis	p95 abs DART = 22.47 $/MWh	Model DART separately from level prices and test weather x DART interactions.	RT scarcity/outage timing may matter more than daily average weather.	A DA spark spread and RT spread are different exposures.
Fuel region vs electric hub	p95 CISO fuel price range = 8.25 $/MMBtu	Define the gas leg explicitly; avoid one generic gas price unless justified.	Fuel insufficiency outage reasons may depend on local gas constraints.	Directly affects spark spread because gas basis changes heat-rate economics.
TAC load vs hub/weather geography	median largest-TAC load share = 46.3%	Use system load as control; build TAC-to-zone mapping later for locational demand exposure.	Load is a key exposure/tightness control, but current geography is not hub-clean.	Load drives price, but location of load affects congestion and hub spreads.
Outage aggregate vs resource/plant geography	weather/outage corr rows = 1840	Current outage/weather results are descriptive until resource-to-plant or node mapping exists.	Final NB target should be counts/events with exposure offsets; current MW aggregate is a proxy.	Outages affect scarcity/prices, but we need location to price the spread impact cleanly.
Plant weather proxy distance	capacity-weighted proxy distance max by region = 113.61 km	Track plant-weather proxy assignment and proxy distance as quality/control variables.	Plant-level weather exposure can support resource-level outage counts once IDs are mapped.	Generator availability and heat exposure can condition marginal fuel/price regimes.

The largest unresolved mismatches are geographic rather than statistical: weather proxies, load areas, fuel regions, market hubs, and outage records do not share a common spatial unit. Hub-specific prices and DA/RT timing should therefore remain explicit in downstream models. Plant-level weather mapping is the clearest path to improving the outage side of the workflow.