Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ jobs:
- name: Fail if an unexpected page reached the site
run: |
set -eu
allowed="404 authors index LICENSE LICENSE-text"
allowed="404 authors index LICENSE LICENSE-text NOTICE"

# An empty result set is not a pass. A loop over nothing exits 0, so a build
# that produced no pages at all would report "all declared" — the guard
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/update-citation-cff.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
- name: Update CITATION.cff
run: |
Rscript -e '
x <- cffr::cff_create()
x <- cffr::cff_create(authors_roles = c("aut", "cre", "cph"))
x$references <- NULL; x$doi <- NULL; x$repository <- NULL
x$identifiers <- Filter(function(i) !grepl("CRAN.package", i$value, fixed = TRUE), unclass(x$identifiers))
if (!length(x$identifiers)) x$identifiers <- NULL
Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ Configurable via `cd_seasons()`. Default: standard meteorological (DJF, MAM, JJA
- `scripts/rag_departure_framing_build.R` — ragnar store for departure framing
- Naming: `rag_<topic>_<verb>.R` (`noun_verb`, mirrors the `cd_*` function convention)
- `data-raw/make_hexsticker.R` — hex sticker (reads from DESCRIPTION)
- `data-raw/readme_figure.R` — the README figure and the `cd_summary()` output pasted under its example, from one live run; its marked block is the README example verbatim, so re-run it and repaste when the catalog gains a year (#142)
- `data-raw/example_*.R` — bundled AOIs + context geodata for vignettes

## Vignettes
Expand Down
18 changes: 10 additions & 8 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Package: cd
Title: Climate Departure Analysis from ERA5-Land Reanalysis
Title: How far any BC watershed's climate has shifted from your chosen baseline
Version: 0.6.12
Date: 2026-10-17
Authors@R: c(
Expand All @@ -8,13 +8,15 @@ Authors@R: c(
affiliation = "New Graph Environment Ltd.")),
person("New Graph Environment Ltd.", role = c("cph", "fnd"))
)
Description: Compute climate anomalies from ERA5-Land reanalysis data for
custom areas of interest. Producer functions download ERA5-Land
monthly means, derive variables (VPD, RH, soil moisture), and publish
Cloud-Optimized GeoTIFFs with a static STAC catalog. Consumer
functions query the catalog, crop to AOI, compute baselines and
anomalies for arbitrary reference periods, and run trend analysis
(Mann-Kendall and Theil-Sen).
Description: Measures how far the climate of any area in British Columbia
has moved from a chosen reference period: air temperature,
precipitation, humidity, soil moisture and snow, by season and year
since 1950, with trends (Mann-Kendall and Theil-Sen). Reads ERA5-Land
reanalysis from a public STAC catalog of Cloud-Optimized GeoTIFFs, and
daily air temperature at points. The baseline, anomaly and trend
functions take any annual series, such as streamflow. Includes the
functions that build and publish the catalog. Under active
development.
License: MIT + file LICENSE
URL: https://github.com/NewGraphEnvironment/cd,
https://newgraphenvironment.github.io/cd/
Expand Down
25 changes: 25 additions & 0 deletions NOTICE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# NOTICE

cd's climate data are derived from ERA5-Land, produced by ECMWF for the Copernicus Climate Change Service and distributed under the [Creative Commons Attribution 4.0 International licence](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0).

> Contains modified Copernicus Climate Change Service information.

ERA5-Land: Muñoz-Sabater, J., et al. (2021). ERA5-Land: a state-of-the-art global reanalysis dataset for land applications. *Earth System Science Data*, 13, 4349–4383. <https://doi.org/10.5194/essd-13-4349-2021>

cd reads ERA5-Land from the [DestinE Earth Data Hub](https://earthdatahub.destine.eu/), aggregates and derives variables from it (seasonal and annual means and totals, vapour pressure deficit, relative humidity, soil moisture, snow metrics, and daily local-day air temperature), and publishes the results to `s3://stac-era5-land`.

## Redistributed data

The example and vignette data in `inst/` are built from these sources (scripts in `data-raw/`):

| Source | Licence |
|---|---|
| ERA5-Land, as processed by cd | CC BY 4.0 (above) |
| BC Freshwater Atlas: watershed groups, lakes, rivers, streams | [Open Government Licence – British Columbia](https://www2.gov.bc.ca/gov/content/data/policy-standards/open-data/open-government-licence-bc) |
| Ecoregion Ecosystem Classification of British Columbia | Open Government Licence – British Columbia |
| BC Geographical Names (town points) | Not yet confirmed ([#151](https://github.com/NewGraphEnvironment/cd/issues/151)) |
| Digital Road Atlas (highways) | Not yet confirmed ([#151](https://github.com/NewGraphEnvironment/cd/issues/151)) |
| Fish & Wildlife Compensation Program Peace Region boundary, and the Peace layers clipped or masked to it | Not yet confirmed ([#151](https://github.com/NewGraphEnvironment/cd/issues/151)) |
| `bcgov/bc_climate_anomaly` (`inst/extdata/example_climate.tif`) | Not yet confirmed ([#151](https://github.com/NewGraphEnvironment/cd/issues/151)) |

> Contains information licensed under the Open Government Licence – British Columbia.
130 changes: 41 additions & 89 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,126 +1,78 @@
# cd <img src="man/figures/logo.png" align="right" height="139" />
# cd <img src="man/figures/logo.png" align="right" height="139" alt="cd hex sticker" />

Climate Departure Analysis from ERA5-Land Reanalysis
<!-- badges: start -->
[![Lifecycle: experimental](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental)
<!-- badges: end -->

## Overview
> How far any BC watershed's climate has shifted from your chosen baseline

cd fetches [ERA5-Land](https://www.ecmwf.int/en/era5-land) hourly reanalysis (1950–present, ~9 km native grid) from the [DestinE Earth Data Hub](https://earthdatahub.destine.eu/), subsets to British Columbia, aggregates to monthly, seasonal, and annual periods, derives additional climate variables (vapour pressure deficit and relative humidity from temperature + dewpoint via the Tetens equation; soil moisture as a 4-depth mean) and snow-pack variables (snow water equivalent, snowfall, snowmelt, snow cover, plus annual derived scalars: peak SWE, snowfall fraction, snowmelt 50% day-of-year, peak weekly melt rate), and writes Cloud-Optimized GeoTIFFs alongside a static SpatioTemporal Asset Catalog (STAC) in a public S3 bucket. A monthly GitHub Action keeps the catalog current. On the consumer side, R functions (`cd_catalog`, `cd_extract`, `cd_baseline`, `cd_anomaly`, `cd_trend`, `cd_compare`, `cd_summary`, `cd_plot_timeseries`, `cd_plot_comparison`) read the COGs directly via GDAL's `/vsicurl/` — no credentials, no tile server — crop to a user-supplied area of interest, compute baselines and anomalies for arbitrary reference periods, and run Mann-Kendall and Theil-Sen trend statistics. All baseline and comparison logic stays on the consumer side, so reference periods are not baked into the served data.
cd measures how much the climate of a watershed or any other area in British Columbia has changed: air temperature, precipitation, humidity, soil moisture and snow, by season and year since 1950, compared with whatever reference period you choose, with the trend and how much it has moved. It reads ERA5-Land reanalysis from a public catalog that needs no account, so restoration can be planned and monitored against the climate a site has now rather than the one it had. It also gives daily air temperature at any point in the province, for stream-temperature models and degree-day work, and its baseline and trend steps take any annual series, such as streamflow or stream temperature.

<img src="man/figures/readme-kotl-tmean.png" alt="Bar chart of annual mean air temperature in the Kootenay Lake watershed group from 1950 to 2025, as the difference from the 1951 to 1980 average, with a dashed trend line rising about 1.9 degrees" width="100%" />

*Annual mean air temperature in the Kootenay Lake watershed group, compared with its 1951–1980 average. Red years were warmer, blue years cooler; the dashed line is the trend since 1951.*

## Installation

```r
pak::pak("NewGraphEnvironment/cd")
```

## Quick start
cd is under active development and its functions change between releases (see [NEWS](https://newgraphenvironment.github.io/cd/news/index.html)). Pin a release for reproducible work: `pak::pak("NewGraphEnvironment/cd@vX.Y.Z")`.

The example below also needs sf, Kendall and zyp: `pak::pak(c("sf", "Kendall", "zyp"))`.

## Example

Extract a series for an area, compare it with a baseline, and test the trend. This is the run behind the figure above:

```r
library(cd)

# Load the live STAC catalog and an example area of interest
catalog <- cd_catalog()
aoi <- sf::st_read(
system.file("extdata", "example_aoi_kotl.gpkg", package = "cd"),
quiet = TRUE
)

# Extract zonal mean time series
ts <- cd_extract(catalog, aoi)

# Compute baseline and anomalies
bl <- cd_baseline(ts, baseline_years = 1951:1955)
ts <- cd_extract(catalog, aoi, variables = "tmean", periods = "annual")
bl <- cd_baseline(ts, baseline_years = 1951:1980)
ano <- cd_anomaly(ts, bl)

# Trend analysis
trn <- cd_trend(ano, trend_start = 1951)

# Reporting table
cd_summary(trn)

# Compare time windows directly
cd_compare(ts, window_a = 1956:1960, window_b = 1951:1955)
#> # A tibble: 1 × 7
#> Parameter Period Slope Years `Total Change` Unit `p-value`
#> <chr> <chr> <dbl> <int> <dbl> <chr> <dbl>
#> 1 Mean temperature Annual 0.025 75 1.9 °C 0
```

## Caching

`cd_extract()` and `cd_crop()` cache each COG on first read, so repeated
extractions, report renders, and vignette rebuilds pull each file from S3
**once** and read locally thereafter — turning recurring S3 egress into a
one-time cost. Caching is on by default (`cache = TRUE`).

```r
# First call downloads; later calls read the local cache.
ts <- cd_extract(catalog, aoi) # cache = TRUE by default

cd_cache_info() # where the cache lives + size
cd_cache_clear() # wipe it
cd_extract(catalog, aoi, cache = FALSE) # bypass the cache for one call
```
Files are cached after the first read and downloaded again only when they are republished. Extracted rows record which commit of cd built each file they came from.

Freshness is checked with a cheap HTTP HEAD (S3 ETag), so the monthly
catalog republish is picked up automatically; `cd_cache_fetch(href,
refresh = TRUE)` forces a re-download. For a fully-offline session set
`options(cd.cache_revalidate = FALSE)` to serve cached copies without any
network call.
## Learn more

**Stopgap without the cache.** If you read COGs through GDAL directly
(e.g. raw `terra::rast("/vsicurl/...")` outside `cd_crop()`), you can cut
repeat egress within a session by enabling GDAL's `/vsicurl/` cache:

```r
Sys.setenv(VSI_CACHE = "TRUE", VSI_CACHE_SIZE = "100000000") # 100 MB
Sys.setenv(GDAL_HTTP_MAX_RETRY = "3", GDAL_HTTP_RETRY_DELAY = "1")
```

This only persists within one R session; the `cd_*` cache above persists
across sessions, which is what kills recurring report-dev egress.

## Data

The producer pipeline fetches ERA5-Land hourly reanalysis from
[DestinE Earth Data Hub](https://earthdatahub.destine.eu/), derives
additional variables (VPD, RH, soil moisture), aggregates to monthly,
seasonal, and annual periods on a single EPSG:4326 BC grid, and writes
Cloud-Optimized GeoTIFFs alongside a static SpatioTemporal Asset
Catalog (STAC) in a public S3 bucket.

- Catalog (JSON): <https://stac-era5-land.s3.us-west-2.amazonaws.com/catalog.json>
- Region: BC (~48–60° N, 114–140° W), 1950–2025, ~9 km native grid
- Variables (15):
- **Core climate** (7): tmean, tmax, tmin, prcp, vpd, rh, soil_moisture
- **Snow monthly natives** (4): swe, snowfall, snowmelt, snow_cover
- **Snow annual derived** (4): swe_max, snowfall_fraction, snowmelt_doy_50, snowmelt_rate_peak
- Periods: seasonal (DJF/MAM/JJA/SON) and annual for monthly-native vars; annual only for snow_max / snowfall_fraction / snowmelt_doy_50 / snowmelt_rate_peak

The catalog is consumable directly outside R — for example, in QGIS via
the STAC plugin, in `gdalcubes`, or with any STAC-aware client. The
`cd_catalog()` consumer function reads the catalog URL by default.

## Architecture

cd has a strict producer / consumer split, kept separate so reference periods, baselines, and statistical choices are not baked into the published data:
- [Climate Departure for the FWCP Peace Region](https://newgraphenvironment.github.io/cd/articles/peace-fwcp.html): trends, snowpack and the spatial pattern across a large northern region, by ecoregion and watershed group.
- [Climate Departure for the Kootenay Lake Region](https://newgraphenvironment.github.io/cd/articles/kootenay-lake.html): the same report for the southern Kootenays, where precipitation changes sharply from east to west.
- [How the Weather Has Changed in Three Northern BC Watersheds](https://newgraphenvironment.github.io/cd/articles/northern-bc.html): a short, plain-language story for readers with no climate background.
- [Function reference](https://newgraphenvironment.github.io/cd/reference/)

| Side | Functions | Where it runs |
|---|---|---|
| **Producer** | `cd_fetch()`, `cd_derive()`, `cd_aggregate()`, `cd_cog_write()`, `cd_stac_catalog()`, `cd_stac_item()`, `cd_s3_push()`, plus `scripts/backfill_edh_*.py` and `scripts/pipeline_*_edh.R` | A monthly GitHub Action pulls fresh ERA5-Land data from DestinE EDH, derives variables, writes COGs, rebuilds the STAC catalog, and pushes to `s3://stac-era5-land`. Run by maintainers; users do not need to run it. |
| **Consumer** | `cd_catalog()`, `cd_extract()`, `cd_crop()`, `cd_baseline()`, `cd_anomaly()`, `cd_trend()`, `cd_compare()`, `cd_summary()`, `cd_plot_timeseries()`, `cd_plot_comparison()`, `cd_periods()`, `cd_seasons()`, `cd_cache_*()` | Reads COGs directly via `/vsicurl/` — no credentials, no tile server. All baseline selection and statistical work (Mann-Kendall, Theil-Sen, Welch's t, Mann-Whitney U) happen client-side on the cropped time series. |
## Ecosystem

Anyone can compute climate departure against any baseline period, in any AOI, with whatever statistical test they prefer — without re-publishing the underlying data.
| Package | Role |
|---|---|
| **cd** | Climate departure: baselines, anomalies and trends, and ERA5-Land climate (this package) |
| [wet](https://github.com/NewGraphEnvironment/wet) | Stream flow and water temperature for BC's Freshwater Atlas |

## Roadmap
cd computes the departures from wet's yearly station values.

- **BEC zone shift analysis** ([#59](https://github.com/NewGraphEnvironment/cd/issues/59)) — methodology lit review for biogeoclimatic ecosystem classification (BEC) shifts under climate change, alongside the existing temperature / precipitation+drying / interpretation-framing methodology stacks.
- **Vignette templates** — `peace-fwcp` and `kootenay-lake` are reference implementations of the regional reporting pattern. Future regional vignettes follow the same structure (trends → recent vs pre-warming → spatial → per-ecoregion → snowpack) so cross-region findings are directly comparable.
## Data and methods it builds on

The climate data are [ERA5-Land](https://www.ecmwf.int/en/era5-land) ([Muñoz-Sabater et al. 2021](https://doi.org/10.5194/essd-13-4349-2021)), produced by ECMWF for the Copernicus Climate Change Service and read from the [DestinE Earth Data Hub](https://earthdatahub.destine.eu/). The published layers contain modified Copernicus Climate Change Service information, under CC BY 4.0. Trends are Mann-Kendall tests with Theil-Sen slopes, from the [Kendall](https://cran.r-project.org/package=Kendall) and [zyp](https://cran.r-project.org/package=zyp) packages. cd's core climate variables and its ecoregion and watershed framing follow [`bcgov/bc_climate_anomaly`](https://github.com/bcgov/bc_climate_anomaly), a Shiny app by the Province of British Columbia (Aseem Sharma and contributors).

## Links
## Citation

- [Function reference](https://newgraphenvironment.github.io/cd/reference/)
- Vignettes:
- [Climate Departure for the FWCP Peace Region](https://newgraphenvironment.github.io/cd/articles/peace-fwcp.html)
- [Climate Departure for the Kootenay Lake Region](https://newgraphenvironment.github.io/cd/articles/kootenay-lake.html)
- [How the Weather Has Changed in Three Northern BC Watersheds](https://newgraphenvironment.github.io/cd/articles/northern-bc.html) — a short, plain-language story for readers with no climate background
Cite cd with `citation("cd")`.

## Acknowledgements
## License

This package is inspired by [`bcgov/bc_climate_anomaly`](https://github.com/bcgov/bc_climate_anomaly), a Shiny app developed by the Province of British Columbia (Aseem Sharma and contributors) that visualizes monthly, seasonal, and annual climate anomalies for temperature, precipitation, humidity, vapor pressure, and soil moisture across BC. The variable set and the eco-region / watershed framing in `cd` follow directly from that work. `bc_climate_anomaly` is licensed under the Apache License 2.0.
MIT © New Graph Environment Ltd. See [LICENSE](LICENSE.md). The ERA5-Land data it ships and publishes are CC BY 4.0; see [NOTICE](NOTICE.md).
42 changes: 42 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,45 @@ url: https://newgraphenvironment.github.io/cd/
template:
bootstrap: 5

reference:
- title: Read the data
desc: Climate series for an area, or daily air temperature at points.
contents:
- cd_catalog
- cd_catalog_default
- cd_extract
- cd_extract_daily
- cd_crop
- title: Departures and trends
desc: Compare a series with a baseline or between two windows, test its trend, and tabulate the result. These take any annual series in cd's long format.
contents:
- cd_baseline
- cd_anomaly
- cd_compare
- cd_trend
- cd_summary
- title: Plots
contents:
- cd_plot_timeseries
- cd_plot_comparison
- title: Variables, periods and seasons
contents:
- cd_variables
- cd_periods
- cd_seasons
- title: Cache
desc: Files are cached after the first read and downloaded again only when they are republished.
contents:
- starts_with("cd_cache_")
- title: Publishing the catalog
desc: For maintainers. A scheduled workflow uses these to write the published files and catalog; users do not need them.
contents:
- cd_aggregate
- cd_cog_write
- cd_stac_catalog
- cd_s3_push
- title: Downloading from the Copernicus Climate Data Store
desc: An alternative route to monthly ERA5-Land through the Copernicus Climate Data Store. The published catalog is not built this way.
contents:
- cd_fetch
- cd_derive
Loading
Loading