Skip to content

V1.0.0.1 release#663

Merged
jirhiker merged 47 commits into
productionfrom
staging
Apr 28, 2026
Merged

V1.0.0.1 release#663
jirhiker merged 47 commits into
productionfrom
staging

Conversation

@jirhiker
Copy link
Copy Markdown
Member

V1 release of the api

dependabot Bot and others added 30 commits April 13, 2026 23:55
Bumps [pillow](https://github.com/python-pillow/Pillow) from 12.1.1 to 12.2.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](python-pillow/Pillow@12.1.1...12.2.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-version: 12.2.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
feat: enhance water well details with site name, historic depth notes, and field event participants
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.22 to 0.0.26.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](Kludex/python-multipart@0.0.22...0.0.26)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.26
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…e handlers to sync and enhancing error handling
feat: implement API concurrency fix strategy by converting async rout…
…and sampler validation

- create or reuse field event participants for imported staff columns
- link imported samples to the measuring participant
- fail a row when measuring_person does not resolve to exactly one participant
- Keep `water_level_notes` as freeform notes
- Stop duplicating field staff and sampler into field event/activity notes
- Treat structured participants as the authoritative staff source
- Verify the well details payload exposes the measuring person on latest_field_event_sample.contact
- Verify the latest field event participants list includes imported staff
…tion

`Contact` rows are unique on `name + organization`, so the water-level importer now uses that same key when resolving field staff contacts. This avoids missing an existing contact with a different contact_type and attempting a duplicate insert.

Also adds regression coverage for reusing an existing same-name/same-organization contact during import.
… tests

Water-level importer tests were leaving participant contacts behind in the shared test database, which caused test_get_contacts to fail during full-suite runs.

Update fixture and CLI test cleanup so importer-created staff contacts are removed after tests finish, while keeping the contact API assertions strict.
…ipant creation

Normalize the fixed field_staff CSV columns into a single iterable shape after validation, so participant creation no longer hardcodes each slot in the importer.

This keeps the input CSV/schema unchanged while making the participant logic easier to read and easier to extend later if the staff-field shape changes.
Bumps [mako](https://github.com/sqlalchemy/mako) from 1.3.10 to 1.3.11.
- [Release notes](https://github.com/sqlalchemy/mako/releases)
- [Changelog](https://github.com/sqlalchemy/mako/blob/main/CHANGES)
- [Commits](https://github.com/sqlalchemy/mako/commits)

---
updated-dependencies:
- dependency-name: mako
  dependency-version: 1.3.11
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [authlib](https://github.com/authlib/authlib) from 1.6.9 to 1.6.11.
- [Release notes](https://github.com/authlib/authlib/releases)
- [Changelog](https://github.com/authlib/authlib/blob/v1.6.11/docs/changelog.rst)
- [Commits](authlib/authlib@v1.6.9...v1.6.11)

---
updated-dependencies:
- dependency-name: authlib
  dependency-version: 1.6.11
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…53-refactor-water-level-importer

kas-bdms-749-thru-753-refactor-water-level-importer
TylerAdamMartinez and others added 13 commits April 17, 2026 17:22
[BDMS-123]: Add UTC normalization for date_time and measurement_date_time in well inventory importer
…#661)

Bumps the gha-minor-and-patch group with 1 update: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv).


Updates `astral-sh/setup-uv` from 8.0.0 to 8.1.0
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](astral-sh/setup-uv@v8.0.0...v8.1.0)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: gha-minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…i-fix

[codex] Reshape well details response around field events
The field_events query returns the 25 most recent events sorted newest-first.
For wells with many visits the oldest event falls outside that window, making
it impossible to retrieve first-visit participants from the paged list.

Adds a separate targeted query that always fetches the single oldest field
event (with participants) and returns it as first_field_event alongside the
existing paged field_events list.
Add first_field_event to well details response
Copilot AI review requested due to automatic review settings April 21, 2026 20:14
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR prepares the API for a V1 release by normalizing datetime handling to UTC, improving well details payload structure (field events/participants), and updating dependency locks and workflows.

Changes:

  • Normalize CSV/import datetimes to UTC (timezone-naive treated as America/Denver; timezone-aware converted via provided offset).
  • Store water-level field staff as structured FieldEventParticipant records and expose them in well details payloads.
  • Convert many DB-backed FastAPI routes from async def to def to avoid blocking the event loop with synchronous SQLAlchemy sessions; update dependencies/workflows accordingly.

Reviewed changes

Copilot reviewed 65 out of 68 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
uv.lock Bumps pinned dependency artifacts (authlib, mako, pillow, python-multipart).
requirements.txt Updates pinned dependency versions/hashes to match lock updates.
pyproject.toml Updates dependency pins to match lock updates.
services/util.py Adds UTC-normalization helper; adjusts quad name helper return type.
schemas/well_inventory.py Normalizes key datetime fields to UTC using shared utility.
schemas/water_level_csv.py Normalizes water level CSV datetime fields to UTC; adjusts validators.
services/water_level_csv.py Creates/reuses field event participants/contacts; links samples to measuring participant; stops embedding staff in notes.
services/well_details_helper.py Reworks details payload to return field events/participants/activities/samples rather than “latest sample/recent obs” fields.
schemas/well_details.py Introduces new well-details response shapes for field events/participants/activities/samples/observations.
schemas/field.py Adds FieldEventParticipantResponse for structured participant output.
schemas/thing.py Adds site_name, historic/access note list fields, changes open_status to boolean; refactors well screen response base type.
schemas/location.py Adds release_status to geojson location response.
db/thing.py Adds site_name, historic_depth_to_water, well_location_note properties; converts open_status to boolean derivation.
db/notes.py Orders notes by created_at and sorts results in _get_notes().
api/thing.py Converts many routes to sync def; adds field_event_limit param for details; expands DB error handling.
api/sample.py Converts routes to sync def; expands DB error handling.
api/contact.py Cleans imports, converts routes to sync def; expands DB error handling.
api/sensor.py Converts routes to sync def.
api/search.py Converts route to sync def.
api/publication.py Converts route to sync def.
api/observation.py Converts many routes to sync def.
api/ngwmn.py Converts routes to sync def.
api/location.py Converts routes to sync def.
api/lexicon.py Converts routes to sync def.
api/group.py Converts routes to sync def.
api/geospatial.py Converts routes to sync def.
api/geochronology.py Converts routes to sync def.
api/author.py Converts route to sync def.
tests/test_well_inventory.py Updates assertions for ordered notes and boolean open status; updates UTC expectations.
tests/test_water_level_csv_service.py Adds coverage for participant creation, idempotency, ambiguity errors, and contact reuse.
tests/test_cli_commands.py Updates expectations around notes and cleans up participant-created contacts in teardown.
tests/test_thing.py Expands details payload assertions to include field events/participants and new well fields; adds tests for site name/open status.
tests/conftest.py Enhances fixture teardown to remove importer-created participant contacts safely.
tests/test_transfer_legacy_dates.py Updates migration wording and import ordering.
tests/test_nma_chemistry_lineage.py Makes driver-agnostic note about NOT NULL violations.
tests/integration/test_nma_legacy_relationships.py Makes driver-agnostic note and adjusts orphan id string.
tests/features/well-inventory-csv.feature Updates BDD spec for UTC normalization semantics and supports timezone-aware inputs.
tests/features/steps/well-inventory-csv.py Updates step implementations to validate/normalize timezone handling consistent with new semantics.
features/admin/README.md Updates project directory name references.
admin/views/init.py Reorders imports and updates package docstring naming.
admin/config.py Updates docstring naming; import ordering cleanup.
admin/auth.py Import ordering cleanup; updates docstring naming.
admin/init.py Updates package docstring naming.
admin/views/thing.py Updates docstring naming.
admin/views/surface_water.py Updates docstring naming.
admin/views/sensor.py Updates docstring naming.
admin/views/sample.py Updates docstring naming.
admin/views/parameter.py Updates docstring naming.
admin/views/observation.py Updates docstring naming.
admin/views/notes.py Updates docstring naming.
admin/views/location.py Updates docstring naming.
admin/views/lexicon.py Updates docstring naming.
admin/views/group.py Updates docstring naming.
admin/views/geologic_formation.py Updates docstring naming.
admin/views/field.py Updates docstring naming.
admin/views/deployment.py Updates docstring naming.
admin/views/data_provenance.py Updates docstring naming.
admin/views/contact.py Updates docstring naming.
admin/views/asset.py Updates docstring naming.
admin/views/aquifer_type.py Updates docstring naming.
admin/views/aquifer_system.py Updates docstring naming.
ADR2.md Adds ADR documenting sync DB + async route concurrency issue and remediation plan.
.gitignore Ignores .codex directory.
.github/workflows/tests.yml Updates setup-uv action version.
.github/workflows/jira_codex_pr.yml Updates pinned setup-uv action ref.
.github/workflows/CD_testing.yml Adds new testing deploy workflow (App Engine deploy + tagging).
.github/workflows/CD_staging.yml Updates setup-uv version and increases gunicorn workers.
.github/workflows/CD_production.yml Updates setup-uv version and increases gunicorn workers.

Comment on lines 239 to 243
assert (
thing.open_status == "Open"
thing.open_status is True
if file_content["is_open"].lower() == "true"
else "Closed"
else thing.open_status is False
)
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion is hard to read and uses a conditional expression with is True / is False. Since open_status is now boolean, consider asserting directly against the expected boolean (e.g., compare to (file_content['is_open'].lower() == 'true')) to make the intent clearer.

Copilot uses AI. Check for mistakes.
Comment thread db/notes.py
Comment on lines 128 to +130
def _get_notes(self, note_type: str) -> list[Notes]:
return [n for n in self.notes if n.note_type == note_type]
notes = [n for n in self.notes if n.note_type == note_type]
return sorted(notes, key=lambda n: n.created_at)
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relationship is already ordered by Notes.created_at, but _get_notes() sorts again. This duplicates work and can still be unstable when multiple notes share the same created_at. Consider relying on the relationship ordering (or sorting by (created_at, id) once) and avoid the extra sorted(...) here.

Copilot uses AI. Check for mistakes.
Comment on lines +17 to +67
- name: Check out source repository
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0

- name: Install uv in container
uses: astral-sh/setup-uv@v8.1.0
with:
version: "latest"

- name: Generate requirements.txt
run: |
uv export \
--format requirements-txt \
--no-emit-project \
--no-dev \
--output-file requirements.txt

- name: Authenticate to Google Cloud
uses: 'google-github-actions/auth@v3'
with:
credentials_json: ${{ secrets.CLOUD_DEPLOY_SERVICE_ACCOUNT_KEY }}

- name: Run Alembic migrations on staging database
env:
DB_DRIVER: "cloudsql"
CLOUD_SQL_INSTANCE_NAME: "${{ secrets.CLOUD_SQL_INSTANCE_NAME }}"
CLOUD_SQL_DATABASE: "${{ vars.CLOUD_SQL_DATABASE }}"
CLOUD_SQL_USER: "${{ secrets.CLOUD_SQL_USER }}"
CLOUD_SQL_IAM_AUTH: true
run: |
uv run alembic upgrade head

- name: Refresh materialized views on staging database
env:
DB_DRIVER: "cloudsql"
CLOUD_SQL_INSTANCE_NAME: "${{ secrets.CLOUD_SQL_INSTANCE_NAME }}"
CLOUD_SQL_DATABASE: "${{ vars.CLOUD_SQL_DATABASE }}"
CLOUD_SQL_USER: "${{ secrets.CLOUD_SQL_USER }}"
CLOUD_SQL_IAM_AUTH: true
run: |
uv run python -m cli.cli refresh-pygeoapi-materialized-views

- name: Ensure envsubst is available
run: |
if ! command -v envsubst >/dev/null 2>&1; then
sudo apt-get update
sudo apt-get install -y gettext-base
fi

- name: Render App Engine configs
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The steps: list is indented at the same level as its - name entries, which makes this workflow invalid YAML (list items must be indented under steps:). Adjust indentation so each step item is nested under steps: (matching the style used in other workflows).

Suggested change
- name: Check out source repository
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0
- name: Install uv in container
uses: astral-sh/setup-uv@v8.1.0
with:
version: "latest"
- name: Generate requirements.txt
run: |
uv export \
--format requirements-txt \
--no-emit-project \
--no-dev \
--output-file requirements.txt
- name: Authenticate to Google Cloud
uses: 'google-github-actions/auth@v3'
with:
credentials_json: ${{ secrets.CLOUD_DEPLOY_SERVICE_ACCOUNT_KEY }}
- name: Run Alembic migrations on staging database
env:
DB_DRIVER: "cloudsql"
CLOUD_SQL_INSTANCE_NAME: "${{ secrets.CLOUD_SQL_INSTANCE_NAME }}"
CLOUD_SQL_DATABASE: "${{ vars.CLOUD_SQL_DATABASE }}"
CLOUD_SQL_USER: "${{ secrets.CLOUD_SQL_USER }}"
CLOUD_SQL_IAM_AUTH: true
run: |
uv run alembic upgrade head
- name: Refresh materialized views on staging database
env:
DB_DRIVER: "cloudsql"
CLOUD_SQL_INSTANCE_NAME: "${{ secrets.CLOUD_SQL_INSTANCE_NAME }}"
CLOUD_SQL_DATABASE: "${{ vars.CLOUD_SQL_DATABASE }}"
CLOUD_SQL_USER: "${{ secrets.CLOUD_SQL_USER }}"
CLOUD_SQL_IAM_AUTH: true
run: |
uv run python -m cli.cli refresh-pygeoapi-materialized-views
- name: Ensure envsubst is available
run: |
if ! command -v envsubst >/dev/null 2>&1; then
sudo apt-get update
sudo apt-get install -y gettext-base
fi
- name: Render App Engine configs
- name: Check out source repository
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0
- name: Install uv in container
uses: astral-sh/setup-uv@v8.1.0
with:
version: "latest"
- name: Generate requirements.txt
run: |
uv export \
--format requirements-txt \
--no-emit-project \
--no-dev \
--output-file requirements.txt
- name: Authenticate to Google Cloud
uses: 'google-github-actions/auth@v3'
with:
credentials_json: ${{ secrets.CLOUD_DEPLOY_SERVICE_ACCOUNT_KEY }}
- name: Run Alembic migrations on staging database
env:
DB_DRIVER: "cloudsql"
CLOUD_SQL_INSTANCE_NAME: "${{ secrets.CLOUD_SQL_INSTANCE_NAME }}"
CLOUD_SQL_DATABASE: "${{ vars.CLOUD_SQL_DATABASE }}"
CLOUD_SQL_USER: "${{ secrets.CLOUD_SQL_USER }}"
CLOUD_SQL_IAM_AUTH: true
run: |
uv run alembic upgrade head
- name: Refresh materialized views on staging database
env:
DB_DRIVER: "cloudsql"
CLOUD_SQL_INSTANCE_NAME: "${{ secrets.CLOUD_SQL_INSTANCE_NAME }}"
CLOUD_SQL_DATABASE: "${{ vars.CLOUD_SQL_DATABASE }}"
CLOUD_SQL_USER: "${{ secrets.CLOUD_SQL_USER }}"
CLOUD_SQL_IAM_AUTH: true
run: |
uv run python -m cli.cli refresh-pygeoapi-materialized-views
- name: Ensure envsubst is available
run: |
if ! command -v envsubst >/dev/null 2>&1; then
sudo apt-get update
sudo apt-get install -y gettext-base
fi
- name: Render App Engine configs

Copilot uses AI. Check for mistakes.
Comment on lines +151 to +156
# ":" are not alloed in git tags, so replace with "-"
- name: Tag commit
run: |
git tag -a "testing-deploy-$(date -u +%Y-%m-%d)T$(date -u +%H-%M-%S%z)" -m "testing gcloud deployment: $
(date
-u +%Y-%m-%d)T$(date -u +%H:%M:%S%z)"
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tag creation command is split across multiple lines inside a single quoted -m argument, which will break the shell command and likely fail the workflow. Consider building the tag message in a variable (or using a single line) and fix the typo in the comment (“allowed”).

Suggested change
# ":" are not alloed in git tags, so replace with "-"
- name: Tag commit
run: |
git tag -a "testing-deploy-$(date -u +%Y-%m-%d)T$(date -u +%H-%M-%S%z)" -m "testing gcloud deployment: $
(date
-u +%Y-%m-%d)T$(date -u +%H:%M:%S%z)"
# ":" are not allowed in git tags, so replace with "-"
- name: Tag commit
run: |
TAG_TIMESTAMP="$(date -u +%Y-%m-%d)T$(date -u +%H-%M-%S%z)"
MESSAGE_TIMESTAMP="$(date -u +%Y-%m-%d)T$(date -u +%H:%M:%S%z)"
TAG_MESSAGE="testing gcloud deployment: $MESSAGE_TIMESTAMP"
git tag -a "testing-deploy-$TAG_TIMESTAMP" -m "$TAG_MESSAGE"

Copilot uses AI. Check for mistakes.
Comment thread api/thing.py
Comment on lines +133 to +139
else:
detail = {
"loc": ["body"],
"msg": error_message,
"type": "value_error",
"input": {},
}
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fallback branch returns the raw database error message to clients. This can leak internal schema/constraint details and produces inconsistent API errors. Consider returning a generic message (and logging the full DB error server-side), or map additional known constraints explicitly.

Copilot uses AI. Check for mistakes.
Comment thread api/sample.py
Comment on lines +81 to +87
else:
detail = {
"loc": ["body"],
"msg": error_message,
"type": "value_error",
"input": {},
}
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as in api/thing.py: the fallback handler returns raw DB error strings to clients, which can leak internal details. Prefer a generic client message and log the original error, or explicitly handle additional expected constraints.

Copilot uses AI. Check for mistakes.
Comment thread api/contact.py
Comment on lines +109 to +115
else:
detail = {
"loc": ["body"],
"msg": error_message,
"type": "value_error",
"input": {},
}
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as in api/thing.py: the fallback branch returns the raw DB error string to clients. Consider returning a generic validation message (and logging the DB error) to avoid leaking constraint/table details.

Copilot uses AI. Check for mistakes.
Comment thread schemas/thing.py
Comment on lines 248 to 281
@@ -263,7 +265,7 @@ class WellResponse(BaseThingResponse):
well_pump_depth: float | None
well_pump_depth_unit: str = "ft"
well_status: str | None
open_status: str | None
open_status: bool | None
datalogger_suitability_status: str | None
measuring_point_height: float | None
measuring_point_height_unit: str = "ft"
@@ -275,6 +277,7 @@ class WellResponse(BaseThingResponse):
permissions: list[PermissionHistoryResponse]
formation_completion_code: FormationCode | None
nma_formation_zone: str | None
well_location_note: list[str] = []

Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

historic_depth_to_water and well_location_note use mutable list literals as defaults. This can lead to shared state across model instances; use Field(default_factory=list) instead (consistent with other response schemas in this repo).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c66d10d27e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread db/notes.py
Comment on lines 128 to +129
def _get_notes(self, note_type: str) -> list[Notes]:
return [n for n in self.notes if n.note_type == note_type]
notes = [n for n in self.notes if n.note_type == note_type]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard note sorting against unpersisted timestamps

_get_notes now sorts by created_at, but created_at is DB-populated and is None on newly added Notes objects until flush/commit. Python sorting with None keys raises TypeError, so calling note-backed properties (for example Thing.historic_depth_to_water/well_location_note) on an object with in-memory notes can fail before persistence. This is a runtime regression introduced by the new unconditional sort.

Useful? React with 👍 / 👎.

order_sort_filter now walks every filter JSON in the request (optional filters
list plus legacy single filter_). _apply_json_filter_clause supports contains,
ncontains, startswith, endswith, eq, ne, comparison operators, null checks, and
in, in addition to the old contains-only behavior. Any list or report that
sends filter= more than once or uses operators other than contains will now
get real SQL instead of silent no-ops.
get_db_things accepts filters as a list of JSON strings (merged with legacy
single filter_) and passes them to order_sort_filter. Adds name_contains to
narrow rows with Thing.name ILIKE for substring search without changing the
full-text query path.
GET /thing/water-well, GET /thing/spring, and GET /thing now bind filter as a
list so multiple Refine filters are forwarded. Same routes accept optional
name_contains for substring matching on thing name alongside existing query.
@jirhiker jirhiker changed the title V1 release V1.0.0.1 release Apr 27, 2026
…st-filtering

BDMS 774  Fix WELL List filters
@jirhiker jirhiker merged commit 2da4c85 into production Apr 28, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants