Authoritative on-disk locations of the datasets this repo processes, so you don't have to guess. All roots are absolute paths on the lab's /spray storage and are defined as module constants in code — make them overridable via CLI flags, never hardcode a new one inline (mirror the existing DEFAULT_*_ROOT pattern).

Conventions used throughout: - Bounding boxes are west south east north in degrees. Longitudes come in both [-180,180) and [0,360); normalize with the shared BBox/normalize_longitudes helpers rather than re-rolling. - Downloaders land raw files under raw/{collection}/YYYY/MM/DD/, processed under processed/…, with manifests/. Downloads are idempotent (skip existing unless an --overwrite-* flag is set). - Data files are not committed (.gitignore excludes *.nc *.csv *.png *.bin); write run outputs to tmp/ or a --out path.

Altimetry (SSH / SSHA)

SWOT L2 LR SSH Expert — /spray/swot/versionD

Constant: DEFAULT_SWOT_ROOT (swot/utils.py, multi_altimeters/*). - Per-pass granules: cycle_XXX/SWOT_L2_LR_SSH_Expert_<CYC>_<PASS>_<START>_<STOP>_PGD0_02.nc - Glob: cycle_*/SWOT_L2_LR_SSH_Expert_*.nc - Sample: /spray/swot/versionD/cycle_001/SWOT_L2_LR_SSH_Expert_001_134_20230726T000155_20230726T000155_PGD0_02.nc - Concatenated per cycle: combined/SWOT_L2_LR_SSH_Expert_cycle_<CYC>_combined.nc (7–33 GB; adds a leading num_passes dim + pass_number(num_passes)). - Structure: 2-D swath num_lines(9866) × num_pixels(69). SSHA vars ssha_karin, ssha_karin_2 (no ssha_noiseless here); coords latitude/longitude (2-D), time(num_lines) (seconds since 2000-01-01), nadir tracks latitude_nadir/longitude_nadir. Quality ssha_karin_2_qual, crossover height_cor_xover, internal tide internal_tide_hret, ocean mask depth_or_elevation < 0. - Longitude: [0, 360). Coverage: science phase from 2023-07-26 onward. - Other SWOT dirs under /spray/swot/ (expert/, expert_aviso/, regridded/, ssha_mean, pass_stats, subsets/, grids/, pixc/, orbits/) are derived/auxiliary — the primary L2 archive is versionD. PIXC downloads default to /spray/swot/pixc (DEFAULT_PIXC_ROOT). AVISO science-orbit shapefiles default to /spray/swot/orbits (DEFAULT_ORBIT_DIR: sph_science_nadir.zip, sph_science_swath.zip). - Domain gotcha: SWOT restarted cycle numbering between orbits. Science phase (21-day, 584 passes/cycle) numbers from 1; cal/val phase (1-day, early 2023, 28 passes/cycle) numbers ~474–577. cycle >= 474 is cal/val (swot.constants.CALVAL_CYCLE_MIN).

SWOT 1 km Unsmoothed OI — /spray/swot/regrid_1km

Constants: GRID_ROOT / OUTPUT_ROOT (swot/regrid_unsmoothed_to_1km.py). - Coordinate grids: {science,calval}/pass_NNN_grid_1km.nc (1 km merged L+R, ~131 × 19 731, lat/lon + cross/along-track metres). - Pass products: cycle_CCC/pass_PPP_regridded_1km.nc (ssha, uncert; two OI parameter sets on n_param by default). - QC: ssha_karin_2 + height_cor_xover, qual==0, ocean class==0, |ssha|≤3 m.

SWOT 250 m FFT+OI tiles — /spray/swot/statistics/oi_250m_fft4km

  • Per side: cycle_XXX/pass_YYY_{left,right}.h5 + sidecar .csv.
  • Groups seg_NNNN: ssha_native, ssha_fft, ssha_oi, lat/lon.
  • Job log: logs/process.log. Resume-safe (skip existing .h5).

SWOT tile diagnostics — /spray/swot/statistics/

  • Aggregated 40×40 tile stats: diagnostics/ (Parquet).
  • Raw 40×40 SSHA extracts: raw_tiles/cycle_XXX_tiles.h5.

RADS nadir altimetry — /spray/rads/data (+ /spray/rads/tables)

Constants: DEFAULT_RADS_ROOT / DEFAULT_RADS_TABLES_DIR (rads/utils.py). - Layout: <sat>/<phase>/c<CCC>/<sat>p<PPPP>c<CCC>.nc (sat ∈ 3a 3b 6a c2 e1 e2 g1 gs j1 j2 j3 n1 pn sa sw tx; phase ∈ a b c d). - Glob: */[abcd]/c[0-9][0-9][0-9]/*.nc - Sample: /spray/rads/data/j3/a/c104/j3p0100c104.nc - Structure: 1-D along-track, single dim time. SSHA var ssha (m, scale_factor 1e-4; variants ssha_mle3, ssha_adaptive, ssha_plrm); coords time (seconds since 1985-01-01), lat, lon. Aux: swh_ku, flags, topography topo_srtm15plus/topo_ace2, sea-ice seaice_conc*. - Longitude: [0, 360). ~170 vars/file. mission_name/cycle_number/pass_number in global attrs. - /spray/rads/data/sw holds the SWOT-nadir tracks used to spatially prefilter SWOT Expert passes (DEFAULT_RADS_SW_ROOT in swot/utils.py). - Pass-index CSV used by colocate: /spray/rads/data/lookup_tables/rads_pass_index.csv (rads.utils.DEFAULT_PASS_INDEX; odsl-colocate-refresh --targets rads). If that file is absent, odsl-colocate-swot-rads falls back to the cycle-repack copy /spray/rads/cycle_files/lookup_tables/rads_pass_index.csv (currently Jason-3).

SWOT × RADS pair files — /spray/swot/swot_rads

Constant: DEFAULT_OUT_ROOT (colocate/colocate_swot_rads.py). - One NetCDF4 file per SWOT Expert × RADS pass crossing (±1 h, 10 km default): swot_c<CYC>_p<PASS>_x_<mission>_c<CYC>_p<PASS>.nc - Groups: /swot (native Expert excerpt + collocated ssha), /rads (along-track excerpt), /profile (RADS line through the swath), /matchup (nearest pixel, distance_km, dt_seconds, swot_cross_track_km). A pair is kept only when the RADS profile crosses the KaRIn swath (≥ 60 km cross-track and both half-swaths by default).

AVISO DUACS SWOT products — /spray/aviso/swot_products

Constant: AVISO_L4_ROOT (multi_altimeters/utils.py). - L4 gridded maps (the only populated tree): l4_karin_nadir/v3_0/miost/dt_global_allsat_phy_l4_<YYYYMMDD>_<PROCDATE>.nc - Glob: l4_karin_nadir/v3_0/miost/dt_global_allsat_phy_l4_*.nc - Structure: regular grid time(1) × latitude(1441) × longitude(2880) (~0.125°). SSHA var sla (+ adt, ugosa/vgosa, ugos/vgos, relative_vorticity); coords latitude/longitude/time (days since 1950-01-01). - Longitude: [-180, 180) ← differs from RADS/SWOT. - Coverage: 2023-03-28 … 2023-04-20 (24 daily maps as of this writing). - L3 along-track l3_karin_nadir/l3_lr_ssh/v3_0/Basic/ — scaffold exists but is empty (0 files).

Sea-surface temperature (SST)

  • Suomi-NPP VIIRS L2P — /spray/snpp (raw/, processed/, regridded/, grids/, manifests/, footprints/snpp_granule_footprints.parquet). Raw: raw/VIIRS_NPP-STAR-L2P-v2.80/YYYY/MM/DD/*.nc.
  • GOES ABI L2P (geostationary GOES satellites, not the NASA GEOS model) — /spray/geos (raw/, processed/, regridded/, manifests/, footprints/geos_granule_footprints.parquet). DEFAULT_OUT_ROOT in geos/geos_sst.py.
  • SST processing convention: kelvin→sst_celsius, recommended_quality (quality_level >= 5), decode l2p_flags day/night → is_day/is_night.

Ocean colour

  • PACE OCI L2 — /spray/pace with per-product dirs PACE_OCI_L2_BGC/, PACE_OCI_L2_IOP/ (AOP excluded on disk), each YYYY/MM/DD/*.nc; plus footprints/, logs/. Note: a science backfill may be running here.

In-situ / auxiliary

  • Argo — /spray/argo/<dac>/<wmo>/<wmo>_prof.nc (dacs: aoml bodc coriolis csio csiro incois jma kiost kma meds nmdis …); processed index processed/argo_profile_index.parquet.
  • Drifters — /spray/drifters/ (drifter_hourly_qc/, drifter_6hour_qc/, derived means/variance/loopers; footprints/drifter_footprints.parquet).
  • Gliders — /spray/gliders/netcdf/ (+ catalog/, metadata/, footprints/glider_footprints.parquet). National IOOS DAC dump (odsl-glider-download). GCOOS-submitted deployments can also be filtered with --gcoos and optionally --layout tree.
  • GCOOS — /spray/gcoos (DEFAULT_GCOOS_ROOT in gcoos/utils.py). Sorted tree: moorings/{nrt,historical}/{network}/*.nc, ship_ctd/{campaign}/*.nc, grids/{family}/*.nc, and gliders/{operational,historical}/{operator}/…. Catalogs under catalog/. See docs/GCOOS_SYNC.md. Do not relocate /spray/gliders/netcdf.
  • ICESat-2 — /spray/icesat2 is release 007: ATL07/ (sea_ice_segments), ATL10/ (freeboard in per-beam group freeboard_segment — v007; the old freeboard_beam_segment yields 0 rows), plus derived thin/, polar-segments/, catalog/, footprints/. Beams gt1l gt1r gt2l gt2r gt3l gt3r.
  • In-situ wind — /spray/wind/insitu/ (ndbc/, oceansites/, ooi_erddap/, pmel_erddap/, samos/, tao_rama_pirata/, unified in unified/, footprints/insitu_wind_stations.parquet).
  • ECCO V4r4 observations — /spray/ecco (DEFAULT_ECCO_ROOT in ecco/utils.py). Official PO.DAAC observational constraints live under raw/ECCO_L4_ANCILLARY_DATA_V4R4/ (the data_constraints tar) and extract to data_constraints/{input_sla,input_sst,input_sss,input_bp,input_seaice_conc,input_insitu,input_other,data_error}. Existing 0.5° model output is in v4r4-0.5deg/ and is not this archive.

Persistent broad-phase indices used by colocate/ and the regional subsetter (odsl-super-subsetter). Build/inspect them all with odsl-colocate-refresh {status,refresh}; authoritative map in colocate/refresh_registration.py::_FALLBACK_INDEX.

Source Index parquet One row per
swot /spray/swot/versionD/footprints/swot_pass_segments_science.parquet pass segment
pace /spray/pace/footprints/pace_granule_footprints.parquet granule (auto-updated by odsl-pace-download from CMR bboxes for locally-present files; the subsetter falls back to a date-tree scan if it ever lags)
argo /spray/argo/processed/argo_profile_index.parquet profile (~922k rows)
glider /spray/gliders/footprints/glider_footprints.parquet deployment file
drifter /spray/drifters/footprints/drifter_footprints.parquet buoy file (6-hour product)
icesat2 /spray/icesat2/footprints/icesat2_granule_footprints.parquet polar-segments parquet granule (not the raw .h5)
snpp /spray/snpp/footprints/snpp_granule_footprints.parquet processed VIIRS L2P granule
geos /spray/geos/footprints/geos_granule_footprints.parquet processed GOES ABI L2P granule
rads /spray/rads/data/lookup_tables/rads_pass_index.csv nadir pass (time only; bbox at load)
aviso /spray/aviso/footprints/aviso_l4_daily.parquet L4 daily map (global bbox; cells sampled at load)
wind /spray/wind/insitu/footprints/insitu_wind_stations.parquet unified-parquet station

Build/refresh every table with odsl-colocate-refresh refresh. SNPP/GOES builders are incremental (skip paths already in the parquet).

Note on the editable install

import <package> / the odsl-* console scripts resolve against whatever is on sys.path. If /opt/miniforge3/.../site-packages/<package> is a stale copy (not an editable link), your source edits won't be seen and you'll get errors like ImportError: cannot import name 'AVISO_L4_ROOT'. Refresh with python -m pip install -e ".[all]", or run against the source tree with PYTHONPATH=<repo-root> python <dir>/<script>.py.