Skip to content

Commit c22118b

Browse files
Sam-BollingSam-Bolling
committed
Add Digitraffic Marine AIS publisher
1 parent 7b6c565 commit c22118b

6 files changed

Lines changed: 526 additions & 1 deletion

File tree

‎docs/research/finland-publisher-expansion/Finland_Source_Endpoint_Probe_2026-05-29.md‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,32 @@ Selected Phase 1 curated stations:
3535
| 4010 | vt12_Hollola_Hameenkoski | Central southern Finland |
3636
| 12091 | kt92_Inari_Naatamo | Lapland / Inari |
3737

38-
Implementation choice: use station-specific latest endpoints for the runtime publisher to keep each fetch bounded and easy to diagnose. Preserve the full source `sensorValues[]` payload as compact JSON in the CSAPI observation result while also lifting common operational fields such as air temperature, road-surface temperature, wind speed, wind direction, precipitation, road condition, and warning code.
38+
Implementation choice: use station-specific latest endpoints for the runtime publisher to keep each fetch bounded and easy to diagnose. Preserve the full source `sensorValues[]` payload as compact JSON in the CSAPI observation result while also lifting common operational fields such as air temperature, road-surface temperature, wind speed, wind direction, precipitation, road condition, and warning code.
39+
40+
## Digitraffic Marine AIS
41+
42+
Latest vessel locations endpoint:
43+
44+
`https://meri.digitraffic.fi/api/ais/v1/locations`
45+
46+
Result: live gzip-compressed GeoJSON `FeatureCollection`. Global sample during probe returned about 18,442 AIS location features with source `dataUpdatedTime` around 2026-05-30T06:46Z to 2026-05-30T07:02Z. Feature geometry is Point coordinates in lon/lat order. Feature properties include MMSI, speed over ground, course over ground, navigation status, heading, ROT, position accuracy flags, and source timestamps.
47+
48+
Vessel metadata endpoint:
49+
50+
`https://meri.digitraffic.fi/api/ais/v1/vessels`
51+
52+
Result: live gzip-compressed JSON array keyed by MMSI with optional vessel name, callsign, IMO, destination, draught, ship type, reference points, and metadata timestamps.
53+
54+
Selected Phase 4 demo window:
55+
56+
| Field | Value |
57+
| --- | --- |
58+
| Latitude min/max | 59.0 / 60.8 |
59+
| Longitude min/max | 22.5 / 28.7 |
60+
| Region | Gulf of Finland / Helsinki approaches |
61+
| Probe count | about 1,198 live vessels in bbox |
62+
| Publish cap | 60 vessels per cycle |
63+
64+
Implementation choice: publish one bounded feed-adapter system and one `digitrafficMarineAisPosition` datastream. Runtime observations lift MMSI, vessel metadata, lat/lon, SOG, COG, heading, navigation status, ship type, destination, source update time, and compact source payload JSON. AIS sentinel values are normalized before publishing: SOG >= 102.2, COG >= 360, and heading >= 511 become `"NaN"`.
65+
66+
Live smoke on 2026-05-30: bootstrapped procedure `04ig`, system `06702`, datastream `07hg2`, root deployment `06i02`, and feed deployment `06ig2` on the public SensorHub. The first retained full cycle used millisecond offsets from source snapshot time so the server stores all vessels from the same Digitraffic snapshot; public proxy verification returned 62 AIS observations, including the 60-record 2026-05-30T07:02:25Z cycle.

‎publishers/digitraffic_marine_ais/README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Digitraffic Marine AIS Publisher
2+
3+
Publishes live vessel positions from Fintraffic Digitraffic Marine AIS for a bounded Gulf of Finland demo window.
4+
5+
The publisher uses a single CSAPI feed-adapter system and one datastream. Each publish cycle filters the public AIS latest-location feed to the configured bbox, caps the cycle to `max_vessels_per_cycle`, enriches records with vessel metadata where available, and emits one observation per vessel.
6+
7+
```powershell
8+
py publishers\digitraffic_marine_ais\bootstrap_digitraffic_marine_ais.py --dry-run
9+
py publishers\digitraffic_marine_ais\digitraffic_marine_ais_publisher.py --dry-run --once
10+
```
11+
12+
Source endpoints:
13+
14+
- `https://meri.digitraffic.fi/api/ais/v1/locations`
15+
- `https://meri.digitraffic.fi/api/ais/v1/vessels`
16+
17+
Attribution follows Digitraffic terms of use.

‎publishers/digitraffic_marine_ais/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Digitraffic Marine AIS publisher package."""

‎publishers/digitraffic_marine_ais/bootstrap_digitraffic_marine_ais.py‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
#!/usr/bin/env python3
2+
"""Bootstrap Finnish Digitraffic Marine AIS CSAPI resources."""
3+
4+
import argparse
5+
import json
6+
import os
7+
import sys
8+
9+
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", ".."))
10+
from publishers.bootstrap_helpers import (
11+
get_config, _auth_header,
12+
ensure_procedure, ensure_system, ensure_datastream, ensure_deployment,
13+
clean_resource, add_bootstrap_args, print_summary,
14+
)
15+
16+
17+
VALID_TIME_START = "2026-01-01T00:00:00Z"
18+
PROC_UID = "urn:os4csapi:procedure:digitraffic-marine-ais:v1"
19+
SYSTEM_UID = "urn:os4csapi:system:digitraffic-marine-ais-feed:v1"
20+
DEPLOY_ROOT_UID = "urn:os4csapi:deployment:digitraffic-marine-ais-demo:v1"
21+
DEPLOY_FEED_UID = "urn:os4csapi:deployment:digitraffic-marine-ais-feed:v1"
22+
DS_OUTPUT_NAME = "digitrafficMarineAisPosition"
23+
24+
DIGITRAFFIC_MARINE_HOME = "https://www.digitraffic.fi/en/marine-traffic/"
25+
DIGITRAFFIC_MARINE_SWAGGER = "https://meri.digitraffic.fi/swagger/"
26+
DIGITRAFFIC_MARINE_AIS_LOCATIONS = "https://meri.digitraffic.fi/api/ais/v1/locations"
27+
DIGITRAFFIC_MARINE_AIS_VESSELS = "https://meri.digitraffic.fi/api/ais/v1/vessels"
28+
DIGITRAFFIC_TERMS = "https://www.digitraffic.fi/en/terms-of-use/"
29+
AIS_ANTENNA_IMAGE = "https://upload.wikimedia.org/wikipedia/commons/5/51/Compact_AIS_antenna.jpg"
30+
31+
32+
def _load_config() -> dict:
33+
here = os.path.dirname(os.path.abspath(__file__))
34+
with open(os.path.join(here, "config.json"), encoding="utf-8") as file:
35+
return json.load(file)["digitraffic_marine_ais"]
36+
37+
38+
def _bbox_center(config: dict) -> tuple[float, float]:
39+
bbox = config["bounding_box"]
40+
return ((bbox["lomin"] + bbox["lomax"]) / 2, (bbox["lamin"] + bbox["lamax"]) / 2)
41+
42+
43+
def _bbox_label(config: dict) -> str:
44+
bbox = config["bounding_box"]
45+
return f"lat {bbox['lamin']}-{bbox['lamax']}, lon {bbox['lomin']}-{bbox['lomax']}"
46+
47+
48+
def _procedure_stub() -> dict:
49+
return {"type": "Feature", "geometry": None, "properties": {"uid": PROC_UID, "featureType": "sosa:ObservingProcedure", "name": "Digitraffic Marine AIS Decoder v1", "description": "Publishes vessel position observations from Fintraffic Digitraffic Marine AIS latest-location data.", "validTime": [VALID_TIME_START, ".."]}}
50+
51+
52+
def _procedure_sml() -> dict:
53+
return {"type": "SimpleProcess", "id": PROC_UID, "uniqueId": PROC_UID, "definition": "sosa:ObservingProcedure", "label": "Digitraffic Marine AIS Decoder v1", "description": "Fetches the Fintraffic Digitraffic Marine AIS latest vessel-location feed, filters it to the configured Gulf of Finland demo window, enriches records with vessel metadata when available, and publishes one CSAPI observation per vessel state.", "keywords": ["Fintraffic", "Digitraffic", "Finland", "marine", "AIS", "vessel", "tracking", "Gulf of Finland"], "documents": [{"role": "http://dbpedia.org/resource/Web_page", "name": "Digitraffic Marine Traffic", "link": {"href": DIGITRAFFIC_MARINE_HOME, "type": "text/html"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "Digitraffic Marine Swagger", "link": {"href": DIGITRAFFIC_MARINE_SWAGGER, "type": "text/html"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "AIS Locations Endpoint", "link": {"href": DIGITRAFFIC_MARINE_AIS_LOCATIONS, "type": "application/json"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "Digitraffic Terms of Use", "link": {"href": DIGITRAFFIC_TERMS, "type": "text/html"}}], "contacts": [{"role": "operator", "organisationName": "Fintraffic / Digitraffic", "contactInfo": {"onlineResource": {"linkage": DIGITRAFFIC_MARINE_HOME}}}, {"role": "publisher", "organisationName": "OS4CSAPI", "contactInfo": {"onlineResource": {"linkage": "https://github.com/OS4CSAPI/OSHConnect-Python"}}}]}
54+
55+
56+
def _system_stub(config: dict) -> dict:
57+
lon, lat = _bbox_center(config)
58+
return {"type": "Feature", "geometry": {"type": "Point", "coordinates": [lon, lat]}, "properties": {"uid": SYSTEM_UID, "featureType": "sosa:Sensor", "name": "Digitraffic Marine AIS Feed - Gulf of Finland", "description": f"Feed-adapter system for Digitraffic Marine AIS vessel observations in the configured demo window ({_bbox_label(config)}).", "validTime": [VALID_TIME_START, ".."]}}
59+
60+
61+
def _system_sml(config: dict) -> dict:
62+
lon, lat = _bbox_center(config)
63+
return {"type": "PhysicalSystem", "id": SYSTEM_UID, "uniqueId": SYSTEM_UID, "definition": "sosa:System", "label": "Digitraffic Marine AIS Feed - Gulf of Finland", "description": f"Feed-adapter system representing a bounded Digitraffic Marine AIS query window over the Gulf of Finland ({_bbox_label(config)}). The system is not a single physical sensor; it republishes live AIS vessel states from Fintraffic's public marine traffic infrastructure.", "keywords": ["Fintraffic", "Digitraffic", "Finland", "marine", "AIS", "vessels", "Gulf of Finland", "feed adapter"], "identifiers": [{"definition": "http://sensorml.com/ont/swe/property/UniqueID", "label": "OS4CSAPI UID", "value": SYSTEM_UID}], "classifiers": [{"definition": "http://sensorml.com/ont/swe/property/SensorType", "label": "Source Type", "value": "Digitraffic Marine AIS feed adapter"}, {"definition": "http://sensorml.com/ont/swe/property/IntendedApplication", "label": "Intended Application", "value": "Maritime traffic situational awareness"}], "contacts": [{"role": "operator", "organisationName": "Fintraffic / Digitraffic", "contactInfo": {"onlineResource": {"linkage": DIGITRAFFIC_MARINE_HOME}}}], "documents": [{"role": "http://dbpedia.org/resource/Photograph", "name": "Actual AIS antenna hardware", "description": "Photograph of real compact AIS antenna hardware used as the representative sensor image for the AIS feed adapter. Source: Wikimedia Commons.", "link": {"href": AIS_ANTENNA_IMAGE, "type": "image/jpeg"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "Digitraffic Marine Traffic", "link": {"href": DIGITRAFFIC_MARINE_HOME, "type": "text/html"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "AIS Locations Endpoint", "link": {"href": DIGITRAFFIC_MARINE_AIS_LOCATIONS, "type": "application/json"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "AIS Vessels Endpoint", "link": {"href": DIGITRAFFIC_MARINE_AIS_VESSELS, "type": "application/json"}}, {"role": "http://dbpedia.org/resource/Web_page", "name": "Digitraffic Terms of Use", "link": {"href": DIGITRAFFIC_TERMS, "type": "text/html"}}], "characteristics": [{"label": "Feed Configuration", "characteristics": [{"type": "Text", "name": "coverage_bbox", "label": "Coverage BBox", "value": _bbox_label(config)}, {"type": "Quantity", "name": "max_vessels_per_cycle", "label": "Max Vessels Per Cycle", "uom": {"code": "1"}, "value": config.get("max_vessels_per_cycle", 60)}, {"type": "Text", "name": "license", "label": "License", "value": "Digitraffic terms of use and attribution"}]}], "capabilities": [{"definition": "http://www.w3.org/ns/ssn/systems/SystemCapability", "label": "Publisher Capabilities", "capabilities": [{"type": "Quantity", "name": "publish_interval", "definition": "http://qudt.org/vocab/quantitykind/Period", "label": "Publish Interval", "uom": {"code": "s"}, "value": config.get("cadence_seconds", 300)}]}], "position": {"type": "Point", "coordinates": [lon, lat], "srsName": "http://www.opengis.net/def/crs/EPSG/0/4326"}}
64+
65+
66+
def _datastream_schema() -> dict:
67+
fields = [
68+
{"type": "Text", "name": "mmsi", "label": "MMSI", "definition": "http://sensorml.com/ont/swe/property/MMSI"},
69+
{"type": "Text", "name": "vesselName", "label": "Vessel Name", "definition": "http://sensorml.com/ont/swe/property/Name"},
70+
{"type": "Text", "name": "callSign", "label": "Call Sign", "definition": "http://sensorml.com/ont/swe/property/CallSign"},
71+
{"type": "Text", "name": "imo", "label": "IMO Number", "definition": "http://sensorml.com/ont/swe/property/Identifier"},
72+
{"type": "Quantity", "name": "lat_deg", "label": "Latitude", "definition": "http://sensorml.com/ont/swe/property/GeodeticLatitude", "uom": {"code": "deg"}},
73+
{"type": "Quantity", "name": "lon_deg", "label": "Longitude", "definition": "http://sensorml.com/ont/swe/property/GeodeticLongitude", "uom": {"code": "deg"}},
74+
{"type": "Quantity", "name": "sog_kts", "label": "Speed Over Ground", "definition": "http://sensorml.com/ont/swe/property/SpeedOverGround", "uom": {"code": "[kn_i]"}},
75+
{"type": "Quantity", "name": "cog_deg", "label": "Course Over Ground", "definition": "http://sensorml.com/ont/swe/property/CourseOverGround", "uom": {"code": "deg"}},
76+
{"type": "Quantity", "name": "heading_deg", "label": "Heading", "definition": "http://sensorml.com/ont/swe/property/Heading", "uom": {"code": "deg"}},
77+
{"type": "Text", "name": "navStatus", "label": "Navigation Status", "definition": "http://sensorml.com/ont/swe/property/Status"},
78+
{"type": "Text", "name": "shipType", "label": "Ship Type", "definition": "http://sensorml.com/ont/swe/property/Type"},
79+
{"type": "Text", "name": "destination", "label": "Destination", "definition": "http://sensorml.com/ont/swe/property/Destination"},
80+
{"type": "Text", "name": "sourceDataUpdatedTime", "label": "Source Data Updated Time", "definition": "http://sensorml.com/ont/swe/property/UpdateTime"},
81+
{"type": "Text", "name": "sourcePayloadJson", "label": "Source Payload JSON", "definition": "http://sensorml.com/ont/swe/property/RawData"},
82+
]
83+
return {"uid": "urn:os4csapi:datastream:digitraffic-marine-ais:positions:v1", "outputName": DS_OUTPUT_NAME, "name": "Digitraffic Marine AIS Vessel Positions", "description": "Live AIS vessel position observations from Fintraffic Digitraffic Marine, filtered to the configured Gulf of Finland demo window.", "schema": {"obsFormat": "application/om+json", "resultSchema": {"type": "DataRecord", "label": "Digitraffic Marine AIS Vessel Position", "fields": fields}}}
84+
85+
86+
def _deploy_root(config: dict) -> dict:
87+
lon, lat = _bbox_center(config)
88+
return {"type": "Feature", "geometry": {"type": "Point", "coordinates": [lon, lat]}, "properties": {"uid": DEPLOY_ROOT_UID, "featureType": "sosa:Deployment", "name": "Digitraffic Marine AIS Demo", "description": "Top-level grouping for the Finnish Digitraffic Marine AIS feed-adapter demo.", "validTime": [VALID_TIME_START, ".."]}}
89+
90+
91+
def _deploy_feed(config: dict, system_server_id: str, base_url: str) -> dict:
92+
lon, lat = _bbox_center(config)
93+
return {"type": "Feature", "geometry": {"type": "Point", "coordinates": [lon, lat]}, "properties": {"uid": DEPLOY_FEED_UID, "featureType": "sosa:Deployment", "name": "Digitraffic Marine AIS Feed", "description": f"Deployment linking the Digitraffic Marine AIS feed adapter to the configured Gulf of Finland query window ({_bbox_label(config)}).", "validTime": [VALID_TIME_START, ".."], "platform@link": {"href": f"{base_url.rstrip('/')}/systems/{system_server_id}", "uid": SYSTEM_UID, "title": "Digitraffic Marine AIS Feed - Gulf of Finland"}}}
94+
95+
96+
def clean_all(base_url: str, auth: str, *, dry_run: bool, stats: dict):
97+
clean_resource(base_url, auth, "deployments", DEPLOY_FEED_UID, dry_run=dry_run, stats=stats, cascade=True)
98+
clean_resource(base_url, auth, "deployments", DEPLOY_ROOT_UID, dry_run=dry_run, stats=stats, cascade=True)
99+
clean_resource(base_url, auth, "systems", SYSTEM_UID, dry_run=dry_run, stats=stats, cascade=True)
100+
clean_resource(base_url, auth, "procedures", PROC_UID, dry_run=dry_run, stats=stats)
101+
102+
103+
def bootstrap(*, clean: bool = False, clean_only: bool = False, dry_run: bool = False, force_sml: bool = False):
104+
config = _load_config(); server = get_config(); base_url = server["base_url"]; auth = _auth_header(server["user"], server["password"]); stats: dict[str, int] = {}
105+
print("\n" + "=" * 70); print(" Digitraffic Marine AIS -- Bootstrap"); print("=" * 70); print(f" Server: {base_url}"); print(f" BBox: {_bbox_label(config)}"); print(f" Clean: {clean} Clean-only: {clean_only} Dry-run: {dry_run} Force-SML: {force_sml}\n")
106+
if clean or clean_only:
107+
clean_all(base_url, auth, dry_run=dry_run, stats=stats)
108+
if clean_only:
109+
print_summary(stats, dry_run); return
110+
print(" -- Procedure --"); ensure_procedure(base_url, auth, PROC_UID, _procedure_stub(), _procedure_sml(), dry_run=dry_run, stats=stats, force_sml=force_sml)
111+
print(" -- System + Datastream --"); system_id = ensure_system(base_url, auth, SYSTEM_UID, _system_stub(config), _system_sml(config), dry_run=dry_run, stats=stats, force_sml=force_sml)
112+
if system_id:
113+
ensure_datastream(base_url, auth, system_id, DS_OUTPUT_NAME, _datastream_schema(), dry_run=dry_run, stats=stats)
114+
print(" -- Deployments --"); ensure_deployment(base_url, auth, DEPLOY_ROOT_UID, _deploy_root(config), dry_run=dry_run, stats=stats)
115+
if system_id:
116+
ensure_deployment(base_url, auth, DEPLOY_FEED_UID, _deploy_feed(config, system_id, base_url), dry_run=dry_run, stats=stats)
117+
print_summary(stats, dry_run)
118+
119+
120+
def main():
121+
parser = argparse.ArgumentParser(description="Bootstrap Digitraffic Marine AIS CSAPI resources")
122+
add_bootstrap_args(parser)
123+
args = parser.parse_args()
124+
bootstrap(clean=args.clean, clean_only=args.clean_only, dry_run=args.dry_run, force_sml=args.force_sml)
125+
126+
127+
if __name__ == "__main__":
128+
main()

‎publishers/digitraffic_marine_ais/config.json‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"digitraffic_marine_ais": {
3+
"description": "Fintraffic Digitraffic Marine AIS vessel positions for a bounded Gulf of Finland demo window.",
4+
"bounding_box": {
5+
"description": "Gulf of Finland and Helsinki approaches",
6+
"lamin": 59.0,
7+
"lomin": 22.5,
8+
"lamax": 60.8,
9+
"lomax": 28.7
10+
},
11+
"cadence_seconds": 300,
12+
"max_vessels_per_cycle": 60,
13+
"locations_endpoint": "https://meri.digitraffic.fi/api/ais/v1/locations",
14+
"vessels_endpoint": "https://meri.digitraffic.fi/api/ais/v1/vessels",
15+
"auth": {
16+
"mode": "none",
17+
"note": "Public Digitraffic API; no key required."
18+
}
19+
}
20+
}

0 commit comments

Comments
 (0)