Return to Fire

Where this began

On October 8, 2017, the Tubbs Fire came over the ridge into Santa Rosa. By morning our house was a chimney and a set of stone steps.

That fire did not make me a data scientist. I already loved this work, the pull of a question you can actually check. But when I sat down in early 2020 to build my first real project, at the very start of my career, there was never a question about the subject. It had to be fires.

I found the biggest wildfire dataset I could: 1.88 million US fires in a SQLite database. I wrote R by hand. I wrote Python by hand. I made maps, bar charts, a random forest. I published a three-part series and was proud of it, and I am still proud of it. It is where the work started.

Here is the detail I did not dwell on at the time: that dataset ended in 2015. The fire that had just burned through my family's life was 22 months past the edge of the data. I was analyzing 1.88 million fires, and the one that mattered most to me could not, by construction, be one of them.

The database has since been updated through 2020, and a research team bolted 270 new columns onto every record: the weather at each ignition, the dryness of the fuels, the terrain, the people nearby. Somewhere in those 2.3 million rows is a row named TUBBS. We will get to it.

How this was rebuilt: five rounds, four exhibits

This page is the finished report, but it was not written in one pass. It was built the way an analysis team works (draft, audit, model, hostile review, final), and every stage shipped as its own page that is still up. Every claim below is meant to be checked rather than taken on trust.

TL;DR

What 2020-me built

Three parts, written by hand at the start of a career: an R/ggplot exploration, a leaflet map tour, and a scikit-learn random forest. The originals stay online, unedited.

Respect first, because it is deserved. 2020-me picked a hard dataset (1.88M rows in SQLite, not a tutorial CSV) and handled the database boundary correctly, closing connections and explaining why. Every figure got a paragraph of interpretation; nothing was dumped and abandoned. The maps used a proper Albers equal-area projection, which plenty of working analysts skip. And buried in Part III is one sentence of real instinct:

"I am going to drop a lot of the variables that are known after the fire. I want to see if we can predict the cause of fire by just the Latitude, Longitude, Month, year, and day of week."

That is leakage awareness, unprompted, self-taught, in 2020. The instinct was right. It was not applied everywhere, and the features it trusted were quietly broken. Which brings us to the bug.

df['DATE'] = pd.to_datetime(df['DISCOVERY_DATE'])
df['MONTH'] = pd.DatetimeIndex(df['DATE']).month
df['DAY_OF_WEEK'] = df['DATE'].dt.dayofweek

The other 2020 problems, briefly and specifically:

• The headline conclusion outran the analysis. "It does seem climate change is a strong cause of this" rested on one straight trend line through raw annual counts. No climate variable existed anywhere in the data, and (as we will see) raw counts partly measure paperwork.
• The 94.7% was never questioned. A random 70/30 split let the model memorize coordinates; campfires were rare enough that guessing "not campfire" scored ~93%. Two other models landed on exactly 0.9000 and the notebook celebrated rather than squinted.
• Model 1 leaked. Predicting cause with FIRE_SIZE and containment dates in the feature set is using the aftermath to predict the beginning.
• Every fire was the same dot. A 1,200-acre fire and a 300,000-acre fire rendered identically on every map.
• The Florida section had a header, a figure, and zero sentences. It gets its words this time.

None of this is disqualifying for a first project. It is a starting line. The rest of this page is the same questions, asked the way I would ask them now.

The data, upgraded

Same national fire registry, two major versions later, plus 270 new columns of physics, fuels, terrain, and people attached to every single fire.

The backbone is the Fire Program Analysis Fire-Occurrence Database (FPA-FOD), the interagency registry of every reported wildfire in the US. My 2020 project used its 2015 edition via Kaggle. The current 6th edition runs 1992–2020 with 2.3 million fires. On top of it, the FPA FOD-Attributes dataset (Pourmohamad et al., 2024; Zenodo 10.5281/zenodo.8381129, CC-BY) attaches ~270 attributes to each record: gridMET fire weather on the discovery day (temperature, humidity, wind, vapor-pressure deficit, ERC), 5-day antecedent windows, local climatological percentiles, NDVI and land cover, elevation/slope/aspect, population, social-vulnerability indices, distances to fire stations. It is 29 annual CSVs, 4.68 GB raw, 308 columns wide.

The 2021–2025 extension. FPA-FOD ends in 2020, which would leave this rebuild six years behind the present. For the large-fire context series only, I pull annual aggregates (counts and acres of 1,000+ acre wildfires) from NIFC's WFIGS perimeter service. It is a different reporting system with different completeness, so it is drawn as a dashed, visibly separate segment, never mixed into the models, and never used for trend tests across the seam.

Honest scope notes (corrected by the Round-2 audit): I originally wrote here that Alaska and Hawaii were absent from this dataset. The data audit proved that wrong. Their fire records are present (AK: 15,195 fires and 36.65M acres; HI: 9,970; PR: 22,202); what's absent is their attributes (weather is empty, terrain carries a 32767 nodata fill). All "CONUS" series on this page now genuinely exclude AK/HI/PR. Cause is missing or undetermined for 8.4% of fires nationally, and for 30.8% of California-plus-Florida fires, which constrains every cause model downstream.

The table itself got a full forensic cross-examination in Round 2, a 29-check streaming audit covering missingness geography, duplicate candidates, coordinate microscopy, and the cause-label mystery: The Data Expert's Cut.

The full look

Part I's job, done the 2026 way: before any model exists, characterize what the table actually delivers, including what is missing, and where the registry itself changes shape.

Six bets, placed before the verdicts

Each hypothesis was written down with a decision rule; the lab fills in the verdict chips from trend_lab_results.json. Not all of them survived, including two claims 2020-me treated as obvious.

California: not more fires, more fire

The state's large-fire count trend is statistically unremarkable. Everything else about its fire behavior changed.

The row

FOD_ID 400015986. In 2020 this record did not exist to be found. Here is what the database now holds for the night everything burned.

Read the weather columns again. Bone-dry air, fuels at a top-decile energy state, and a wind field the daily average can only hint at. The row is the Diablo-wind signature written into a database, the same pattern the seasonal map shows for nearly every autumn disaster on the California coast ranges.

You can print this row yourself, offline, in one command: python examples/find_tubbs.py.

Florida, which finally gets its paragraphs

In 2020 the Florida section was a header and an unexamined image. It deserved better: Florida is the clearest counterexample to every "fires are exploding everywhere" take.

Florida burns constantly and on purpose: it runs one of the country's largest prescribed-burn programs, and its ecosystems are built to burn frequently and lightly. Its large-fire count trend over 29 years is a flat line (IRR 0.99/decade, MK p=0.612); its burned-area trend is indistinguishable from noise (MK p=0.468); and its fire season, if anything, contracted (257 → 228 days era-over-era, not significant).

So H6, the 2020 claim that large wildfires were rising "across the U.S. and across California and Florida", is rejected. The rejection carries the finding: the American fire problem is not a uniform national trend. It is a Western drying story, and Florida's stability is the control group that proves the point.

The national picture, with the paperwork removed

2020-me fit one line through national counts and read climate into it. The honest version has to separate physics from reporting first.

The 2021–2025 extension adds current-events context without contaminating the statistics: the WFIGS years include the quiet 2022–2023 lull and the ferocious 2024; January 2025's Los Angeles fires (Palisades, Eaton) barely register in acres, and were among the costliest disasters in state history, a reminder this page returns to in Limits: burned area is not damage.

The model lab: a museum piece and an honest machine

First rebuild the 2020 random forest exactly as it was (dead features, leaked outcome, random split), then sit it next to models evaluated the way 2026 requires.

The museum piece

Faithfully reconstructed on 1992–2015 California large fires (the old 13 cause classes approximated from NWCG's new taxonomy): the same RandomForest settings, the same random 70/30 split, MONTH pinned to 1 and DAY_OF_WEEK to Thursday exactly as the Julian bug left them, and FIRE_SIZE left in the features exactly as the leak allowed. It scores 49.4% on the 13-class task (the 2020 notebook reported 61.3%) and 96.0% on the celebrated campfire binary (2020: 94.7%), against a guess-the-majority baseline of 95.7%. Re-scored under a temporal split, the 13-class museum model's macro-F1 is 0.14.

The honest machine

Gradient boosting (scikit-learn's HistGradientBoosting, which handles missing values and categoricals natively with no extra dependencies) on at-ignition features only: location, calendar (computed correctly this time), the fire weather of the day, fuel moisture, vegetation, terrain, and human presence. Named and excluded: FIRE_SIZE, size class, containment anything, agency, owner. If it is not knowable at ignition, it is not a feature. Cause labels are missing for 15.2% of CA fires overall, but not evenly: the Round-2 review found the exclusion is 10.1% in the training era and 41.7% in the 2018–2020 test era, because California's cause-labeling collapsed over time (see the data audit). Test-era scores are therefore measured on the shrinking subset of fires the state still labels, a selection caveat reported rather than hidden.

Protocol: train ≤2014, validate 2015–2017, test 2018–2020. The model never sees the era containing Tubbs. Inside the training era, 5-fold spatial-block cross-validation (1°×1° blocks) checks that scores survive when nearby fires can't leak across folds. Three baselines every model must beat: majority class, a groupby climatology (majority cause per 0.2° cell × month), and a linear model.

The 2026 operational model, the piece 2020 never had

The cause lab above answers a scientist's question. The modeling stage asks the one a duty officer has: of the ignitions reported, which few will become something? Its label is clean where cause is drifting (fire size is essentially never missing), and its protocol was fixed before any model ran: train ≤2014, calibrate on 2015–2017, test on 2018–2020, spatial-block CV inside the training era, test-set selection gated in code.

And because a stakeholder would ask: leaving the freak 2020 season out entirely, the pooled test score is 0.106, printed rather than buried; the model keeps 3.5× lift even in the emptiest population tercile, so it is not a remoteness lookup; and transferred to Florida it drops to 0.180 against 0.293 for a Florida-native model. It learned California, and says so. Every one of those numbers comes from the 35-finding hostile review, which is its own exhibit.

Is it overfit, and can it predict fires before they start?

Two questions worth answering directly. On the first, the model was scored on all three eras: average precision 0.538 on the training years it fitted, 0.099 on the 2015–2017 validation era, 0.124 on the 2018–2020 test era. The in-sample gap is large, which is what a 360-tree ensemble does with 200,967 rows, and it is why no score in this project is ever quoted from the training era. The part that matters is that the two held-out eras agree with each other: ROC-AUC 0.813 and 0.817, with the later, more distant era scoring slightly higher. Skill that decayed with distance from training would look different. Refit on shuffled labels, the same pipeline collapses to AP 0.0232 against a 0.0206 base rate, so the ranking is not an artifact of the evaluation code.

That 0.817 ROC-AUC deserves a caveat rather than a victory lap. Between the training era and the test era, ROC-AUC falls 14% while average precision falls 77%. On a 2% problem, ROC-AUC is dominated by the 97.9% of easy negatives and reads far more forgiving than the metric that governs a real response budget.

On the second question the answer is no, for a structural reason. This model estimates the probability that a reported fire escalates. Predicting ignition itself needs the places and days where nothing burned, and those rows exist in no cache here: 740 fire-prone 0.25° California cells across 10,593 days give 7.8 million cell-days, of which the data holds 251,882. The missing 7.6 million are the negative class an ignition model would need, and there is no gridded daily weather with which to describe them. The full accounting is in the modeling stage.

Defend it

Every headline claim on this page, against the strongest objection I could aim at it. Objections that partially land are conceded, in print.

The full adversarial review (35 findings, the four blockers, and every experiment it demanded) is published as its own section of the modeling stage.

Scoreboard

Six bets on the science, two on the craft. One number each.

Six years later

What the beginner got right, what the rebuild fixed, and what changed in between.

The deeper change is speed of thought. In 2020, every idea cost an evening of hand-written code, so ideas went untested and enthusiasm attached to whatever numbers appeared. In 2026 I can spend that same evening trying twelve ideas, discarding nine, stress-testing three. The thinking is still mine; the friction is gone. This rebuild took a fraction of the original's calendar time and did roughly a hundred times the work.

You can see the six years in the artifacts themselves. The 2020 series is three pandoc-default knitr renders: megabytes of base64-inlined screenshots, no design system, the mistakes baked in and invisible. The 2026 version is four hand-tokened, zero-JavaScript pages that share one stylesheet, a frozen first draft anyone can diff against this one, and a verifier that fails the build if a single number drifts from the computation, or if a retired one reappears. The 2026 pages can be checked against the computation behind them. The 2020 pages could only be believed.

The original trilogy stays online, unedited, exactly as a first project should: Part I · Part II · Part III. It was a first project, and it reads like one.

Limits, stated plainly

• Reported ignitions are not fire risk. This registry sees fires that got reported; its small-fire completeness varies by state and era, and several analyses here exist specifically to route around that.
• Association is not attribution. The extreme-fire-weather shift at ignition is consistent with a warming, drying West; formal climate attribution requires counterfactual modeling this project does not attempt.
• Burned area is not damage. January 2025's Los Angeles fires were small in acres and catastrophic in loss. Acres are the measurable proxy here, not the human cost.
• Lower 48 by choice, not by dataset. The Round-2 audit found AK/HI/PR fire records in the data (attribute-less); the CONUS series here exclude them deliberately. National acreage figures therefore understate the US total; Alaska alone adds ~37M acres over the period.
• Daily 4-km weather. gridMET cannot see nighttime wind events like the one that drove Tubbs; the row's 6.0 m/s daily mean is the floor of that night's truth, not the story.
• Cause labels are incomplete and reclassified. A third of CA/FL records carry no usable cause; the old-13 museum crosswalk is approximate by construction.
• One fire anchors the narrative. The Tubbs row is one record given personal weight; the statistics never depend on it.
• The escalation model is a retrospective prototype. It runs on reanalysis weather that does not exist at decision time; an operational version needs forecast or station feeds. The red team measured what that costs: a forecast-free variant restricted to yesterday-knowable weather loses essentially nothing. The distinction is stated here as well as there.

Reproduce it

Everything on this page, and on the audit, the modeling stage, and the red-team section, regenerates from public data with plain Python. One network step, offline forever after.

python -m venv .venv
.venv\Scripts\activate                  # Windows
pip install -r requirements.txt         # + lightgbm / optuna / shap for the modeling stage

python fetch_raw.py --yes               # ~5 GB, once (or --years 2015-2020 to start smaller)
python fetch_recent.py                  # WFIGS 2021-2025 aggregates (seconds)
python reduce_raw.py                    # 4.68 GB -> ~33 MB of hash-locked caches (~10 min)

python examples\find_tubbs.py           # the row, offline, instant
python examples\headline_trend.py       # the headline IRRs, offline, instant

python run_eda.py                       # dataset anatomy
python run_trend_lab.py                 # H1-H6 verdicts + trend figures
python run_ml_lab.py                    # museum replica + honest cause models (~20 min)
python run_maps.py                      # the three maps
python run_data_audit.py                # the 29-check corpus audit + data dictionary (~12 min)

python tuning.py --probe                # which of sklearn / optuna / lightgbm / shap import
python run_modeling.py                  # the escalation model: bake-off, zoo, calibration (~45 min)
python run_redteam.py                   # the 35-finding hostile review's experiments (~10 min)
python run_final_figures.py             # calibration figures re-rendered from the val-selected map

python verify_claims.py                 # every number on ALL THREE pages == the JSONs, or exit 1

Sources: FPA FOD-Attributes (CC-BY 4.0; Pourmohamad et al. 2024, ESSD) · FPA-FOD v6 (Short, USFS RDS-2013-0009.6) · NIFC WFIGS perimeters (public domain) · US Census cartographic boundaries (public domain).