Skip to content

Datasets and source contracts

Last updated View as MarkdownAgent index

StormTrust names a dataset only when its authority, license, version, time semantics, geographic coverage, and quality checks are known. A map layer is a view of a dataset. It is not the dataset itself.

For configured cadence, queue retries, persistence checkpoints, and recovery limits, see Polling and triggers.

Current inventory

Dataset Status Authority Use Persistence
NOAA MRMS MESH Implemented NOAA/NSSL Radar-derived maximum estimated hail size Raw GRIB2 and rendered PNG in R2; catalog and revisions in D1
NOAA MRMS lowest-altitude reflectivity Implemented NOAA/NSSL Storm context and movement Raw GRIB2 and rendered PNG in R2; catalog and revisions in D1
NWS SPC outlooks Discovery lead; ingestion not implemented NOAA outlook service ↗ Proposed Forecast preparation No accepted source contract or ingested issuance history
NWS warnings Implemented in the separate hail-alert app NWS Configured observed-hail warning eligibility in the Kansas City market App-owned warning decisions; not a statewide report trigger
Canvassing-area boundaries Source/review process proposed Named boundary provider or authorized supervisor-defined geometry Report starting areas No accepted area release or report workflow is implemented
Nominatim place search Implemented as an on-demand lookup OpenStreetMap contributors Search addresses and named geography; pan or fit the map Response is not a StormTrust parcel or boundary dataset
United States administrative boundaries Candidate source selected United States Census Bureau TIGER/Line Countries, states, counties, jurisdictions, and ZCTAs Catalog schema exists; no boundary release is ingested
United States parcels Source not selected County or licensed parcel provider Property geometry and stable parcel identity Catalog schema exists; no production parcel release is ingested
Structures (building footprints) Source not selected Microsoft Building Footprints ↗ (license ↗) or licensed alternative Intersect storm severity polygons with building geometry No structure-geometry catalog exists; no release is ingested
Property outcomes Source not selected Customer-authorized or licensed outcome source Calibrate and evaluate damage-likelihood models Evaluation schema exists; no outcome corpus is ingested

NOAA MRMS

The NOAA MRMS Open Data registry ↗ describes a two-minute update cycle and open public access. StormTrust ingests MESH_00.50 and MergedReflectivityAtLowestAltitude_00.50 from the public AWS bucket. The operational MRMS GRIB2 table ↗ defines MESH in millimeters at a two-minute cadence.

MRMS is radar-derived evidence. It is not a confirmed hail report, property damage determination, or statement of NOAA endorsement. Every layer preserves its source URL, source modification time, observation time, unit, legend, and caveat.

For the first ten minutes after UTC midnight, discovery checks both the current and previous date prefixes. This overlap captures late-published prior-day objects. Versioned idempotency absorbs repeated listings.

Place search, boundaries, and parcels

These three concerns have different contracts:

Concern Answers Required data
Place search “Where did the user mean?” Geocoder result with coordinates and optional bounds
Administrative boundary “What area does this named geography cover?” Versioned country, state, county, jurisdiction, or ZCTA geometry
Parcel “Which property geometry and stable identifier is this?” Versioned parcel release with provider lineage

Parcel data is not required to pan or zoom after a search. The geocoder already returns a coordinate and, when available, a bounding box. Parcel data becomes necessary when StormTrust intersects storm evidence with property geometry and returns property-level results.

The Census Bureau TIGER/Line files ↗ are suitable for geographic identifiers and many administrative boundaries. They do not provide parcel ownership, roof attributes, or Google Places data. A ZCTA is an approximate statistical representation of a USPS ZIP Code service area, not the service area itself.

Missouri source research

The Missouri documentation contains Census jurisdiction identities and a Jackson County discovery example. It does not change the parcel inventory into an approved source or an ingested release. Use the source-review workflow to establish interface, permission, native schema, and acceptance evidence.

Structures

Building-footprint geometry intersects storm severity polygons to estimate counts of exposed structures. No structure-geometry source is selected. A candidate source must publish per-building polygons with a documented release, coverage, and license before StormTrust treats it as a dataset under this contract.

One candidate is Microsoft Building Footprints ↗, which publishes dataset documentation, country and region coverage, and release artifacts. Its license terms are published at Microsoft Maps Platform APIs Terms of Use ↗, and Microsoft also publishes coverage and release notes on the Global ML Building Footprints repository ↗.

A building footprint is not a parcel. A footprint describes a rooftop outline; a parcel describes a property boundary and ownership identity. Moving from swath-level exposure to address-level exposure needs both.

Source onboarding contract

Before a new source reaches production, record:

  1. Provider, authority class, canonical documentation, and license.
  2. Dataset name, source version, retrieval time, and valid time.
  3. Geographic coverage, resolution, coordinate reference system, and units.
  4. Stable source identity, checksum, raw object location, and normalized object location.
  5. Missing-value semantics, correction behavior, and expected cadence.
  6. Quality thresholds, quarantine rules, retention class, and deletion rules.
  7. Personally identifiable information and consent classification.
  8. A fixture or benchmark that detects schema and scientific drift.

dataset_versions records release-level lineage. administrative_areas catalogs searchable geography. parcel_partitions catalogs immutable parcel artifacts without forcing nationwide geometry into D1. R2 stores the bulk geometry, such as GeoParquet or vector tiles.

Data hygiene invariants

  • Treat (source, sourceKey, sourceVersion) as canonical source identity and include eventId in event-processing identity. This lets the same source evidence support a rolling feed and a durable benchmark without collision.
  • Preserve observation, source modification, retrieval, publication, and processing times as separate fields.
  • Store raw inputs before deriving normalized artifacts when licensing permits.
  • Publish an immutable dataset version; do not mutate a published partition.
  • Reject unknown units, invalid bounds, impossible timestamps, and unbounded input sizes.
  • Keep source facts, derived estimates, model probabilities, and human outcomes in separate fields.
  • Pin every exposure run to its analysis scope or established storm revision, source and geometry releases, and rule/model version. Parcel and roof releases are required only for modes using them.
  • Record quality results and rejection reasons; do not silently drop records.