\n $\"No$ \n

Combining datasets - Elemental and Organic Carbon at Birkenes\u00b6

In\u00a0[27]:

import numpy as np\nimport xarray as xr\nimport matplotlib.pyplot as plt\nimport json\nimport requests\nimport os\nimport time\n

Loading data\u00b6

This notebook continues from the Search with ACTRIS Data Portal example. It reads the basket.json file saved there, which contains metadata and access URLs for a set of elemental carbon (EC) and organic carbon (OC) datasets from Birkenes, Norway.

The data can be accessed in two ways \u2014 set protocol in the cell below:

opendap \u2014 streams data directly from the THREDDS server (no local files needed)
http \u2014 downloads the NetCDF files to a local folder

In\u00a0[28]:

protocol = 'opendap' # choose 'http' or 'opendap'\n

In\u00a0[29]:

# Navigate to the Search and access folder where basket.json is stored\nfolder = os.getcwd().replace(\"  Analyse ACTRIS data\", \"   Search and access\")\nos.chdir(folder)\n

In\u00a0[30]:

with open('static/basket.json') as f:\n    md = json.load(f)\n\nif protocol == 'http':\n    os.makedirs(\"static/data_oc_ec\", exist_ok=True)\n    for entry in md.values():\n        r = requests.get(entry['downloadUrl'])\n        filename = r.url[-12:]\n        with open(os.path.join(\"static/data_oc_ec\", filename), mode=\"wb\") as file:\n            file.write(r.content)\n    print(f\"Downloaded {len(md)} files to static/data_oc_ec/\")\n

Data processing\u00b6

The ACTRIS In-Situ NetCDF files follow a standard naming convention where measurement variables start with v_. Each file may contain multiple carbon fractions (EC, OC, total carbon) and size fractions (PM10, PM2.5). Here we extract the PM10 arithmetic mean for each carbon type.

Documentation for the In-Situ NetCDF file format: https://ebas.pages.nilu.no/ebas-io/fileformat_netcdf/index.html

In\u00a0[31]:

ds_elemental_carbon = []\nds_organic_carbon = []\nds_total_carbon = []\n\nEXCLUDE = ('_uncertainty', '_d_Fraction', '_max_', '_pm25', '_corrected')\n\ndef pick_best(candidates, ds):\n    for suffix in ('_pm10_amean', '_pm10', '_amean', ''):\n        match = [v for v in candidates if v.endswith(suffix)]\n        if match:\n            return ds[match[0]]\n\ndef load_entry(entry):\n    # Skip datasets that require authentication\n    if requests.head(entry['downloadUrl'], timeout=10).status_code != 200:\n        return entry, None\n    url = entry['opendapUrl'] if protocol == 'opendap' else 'static/data_oc_ec/{}'.format(entry['downloadUrl'][-12:])\n    for attempt in range(3):\n        try:\n            return entry, xr.open_dataset(url)\n        except Exception:\n            time.sleep(3) if attempt < 2 else None\n    return entry, None\n\nfor entry in md.values():\n    _, ds = load_entry(entry)\n    if ds is None:\n        continue\n    candidates = [v for v in ds.data_vars if v.startswith('v_') and not any(x in v for x in EXCLUDE)]\n    ec = pick_best([v for v in candidates if 'elemental_carbon' in v], ds)\n    oc = pick_best([v for v in candidates if 'organic_carbon' in v], ds)\n    tc = pick_best([v for v in candidates if 'total_carbon' in v], ds)\n    if ec is not None: ds_elemental_carbon.append(ec)\n    if oc is not None: ds_organic_carbon.append(oc)\n    if tc is not None: ds_total_carbon.append(tc)\n\nprint(f\"Loaded: {len(ds_elemental_carbon)} EC, {len(ds_organic_carbon)} OC, {len(ds_total_carbon)} TC time series\")\n

Loaded: 14 EC, 16 OC, 16 TC time series\n

In\u00a0[32]:

# Sort each list chronologically before concatenating\nsort_key = lambda x: x.time.values[0] if len(x.time) > 0 else np.datetime64('NaT')\nds_elemental_carbon.sort(key=sort_key)\nds_organic_carbon.sort(key=sort_key)\nds_total_carbon.sort(key=sort_key)\n

In\u00a0[33]:

def concat_with_gaps(arrays):\n    \"\"\"Concatenate DataArrays with a NaN gap between segments so matplotlib\n    does not draw connecting lines across time gaps between files.\"\"\"\n    pieces = []\n    for i, da in enumerate(arrays):\n        pieces.append(da)\n        if i < len(arrays) - 1:\n            gap_time = da.time.values[-1] + (arrays[i+1].time.values[0] - da.time.values[-1]) / 2\n            pieces.append(xr.DataArray([np.nan], dims=['time'],\n                                       coords={'time': [gap_time]}, attrs=da.attrs))\n    return xr.concat(pieces, dim='time')\n\nec_ds = concat_with_gaps(ds_elemental_carbon)\noc_ds = concat_with_gaps(ds_organic_carbon)\ntc_ds = concat_with_gaps(ds_total_carbon)\n

Data visualisation\u00b6

The plot below shows the full time series of PM10 elemental carbon (EC) and organic carbon (OC) at Birkenes, combining all available annual files into a single continuous record. OC is consistently higher than EC, with both showing clear seasonal variability and episodic peaks associated with long-range transport events.

In\u00a0[34]:

plt.figure(figsize=(15, 5))\nplt.plot(ec_ds.time, ec_ds, label='Elemental Carbon')\nplt.plot(oc_ds.time, oc_ds, label='Organic Carbon')\n#plt.plot(tc_ds.time, tc_ds, label='Total Carbon')\nplt.legend()\nplt.ylabel('ug C/m3')\nplt.xlabel('Time')\nplt.title('Elemental Carbon and Organic Carbon pm10 at Birkenes in Norway')\nplt.show()\n

\n $\"No$ \n

The same approach can be applied to any variable or station available through the ACTRIS Data Portal. To also show total carbon, uncomment the tc_ds line in the plot cell. The OC/EC ratio can be computed as oc_ds / ec_ds and is commonly used as an indicator of secondary organic aerosol formation.