payments: count total with date filters

[0xjc65eth]

Summary

• Make QueryPayments TotalCount honor CreationDateStart and CreationDateEnd when CountTotal is requested.
• Keep IncludeIncomplete=false aligned with returned-payment semantics by counting only succeeded payments.
• Apply the fix to both KV and SQL backends.
• Keep the KV count path lightweight by reading query metadata/status without loading full HTLC attempt payloads/routes.
• Add regression coverage for date-window counts, incomplete payment filtering, and a mixed settled+unresolved in-flight payment.
• Add a release note entry for the ListPayments count fix.

Fixes #8530.

Commit structure

• paymentsdb: filter KV total count by query
• sqldb: filter payment total count by query
• paymentsdb: test filtered total count semantics
• docs: add listpayments count release note

Tests

• go test ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1
• go test -tags test_db_sqlite ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1
• go test ./payments/db -count=1
• (cd sqldb && go test ./sqlc -count=1)
• git diff --check origin/master...HEAD

Benchmark

Local SQLite micro-benchmark on Apple M2, benchtime=20x, count=3, 250 synthetic payments, QueryPayments with CountTotal=true, date window, and IncludeIncomplete=false:

• origin/master: avg 5.75 ms/op (5.42, 6.03, 5.80 ms/op)
• this branch: avg 6.53 ms/op (5.63, 7.73, 6.24 ms/op)
• delta: about +13.6% for the corrected filtered/status-aware count path

The benchmark is local-only and intentionally not committed; it was used to sanity-check the added filtered count work against the previous global count behavior.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request ensures that the TotalCount field in payment queries accurately reflects the date-filtered results. By applying consistent filtering logic across both KV and SQL backends, the system now correctly reports the total number of payments within a specified time range, even when pagination is applied to the result set.

Highlights

• Payment Filtering: Updated the QueryPayments total count logic to respect CreationDateStart and CreationDateEnd filters.
• Backend Support: Implemented the filtered count logic for both KV and SQL storage backends, including new sqlc query definitions.
• Testing: Added a new test case in TestQueryPayments to verify that TotalCount correctly reflects the number of payments within a specific date window.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces support for counting total payments with date filters in both the KV and SQL database backends. In the KV store, a new matchesCreationDate helper is used to filter payments, and countFn is updated to apply these filters when counting total payments. In the SQL store, a new CountFilteredPayments query is added to perform the filtered count. However, feedback highlights a performance issue in the KV store implementation: calling fetchPaymentWithSequenceNumber inside countFn for every payment is highly inefficient as it fully deserializes the entire payment and its HTLC attempts. It is recommended to optimize this by only fetching the creation time.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[github-actions]

PR Severity: HIGH. sqldb/* files (querier.go, payments.sql) drive HIGH classification. payments/* files are MEDIUM. 4 non-excluded files, ~120 non-excluded lines. No bump triggered. <!-- pr-severity-bot -->

[0xjc65eth]

Thanks for the review. I updated the KV counting path so date-filtered CountTotal no longer calls fetchPaymentWithSequenceNumber. It now uses a lightweight fetchCreationTimeWithSequenceNumber helper that only reads PaymentCreationInfo for the matching sequence number, including legacy duplicate-payment buckets, and avoids loading/deserializing HTLC attempts.\n\nI also added the release note entry and refreshed the PR body.\n\nRe-ran:\n- go test ./payments/db -run TestQueryPayments -count=1\n- go test -tags test_db_sqlite ./payments/db -run TestQueryPayments -count=1\n- (cd sqldb && go test ./sqlc -count=1)

[murraystewart96]

Looks good! A few small comments but other than that this works well. It's worth getting a second opinion about whether the payment count should exclude failed payments unless the IncludeIncomplete is provided.

This would require also updating the DB query to do a status check. If we add this we should also then add a test so that we are testing both (count w creation window and IncludeIncomplete) & (count w creation window and !IncludeIncomplete)

[0xjc65eth]

Thanks, that makes sense. I updated the branch so TotalCount now mirrors the query result semantics for IncludeIncomplete:

• KV: the count path now fetches lightweight query metadata (creation time + status) without loading/deserializing all HTLC attempts. It applies the status filter whenever IncludeIncomplete=false, with or without date filters.
• SQL: CountFilteredPayments now accepts include_incomplete; when false it only counts payments with a settled HTLC resolution (resolution_type = 1).
• Tests: split/extended TestQueryPayments coverage for both date-window cases: IncludeIncomplete=true counts all payments in the window, while IncludeIncomplete=false counts only the succeeded payment in the window. I also updated the existing count-total/incomplete case to expect the succeeded-only count.

Re-ran:

• go test ./payments/db -run TestQueryPayments -count=1
• go test -tags test_db_sqlite ./payments/db -run TestQueryPayments -count=1
• (cd sqldb && go test ./sqlc -count=1)
• git diff --check

[murraystewart96]

Changes look good! Just a few other comments i've left if you have any opinions on them?

[0xjc65eth]

Thanks again for the review. I pushed one more update to make the IncludeIncomplete=false count path mirror the existing payment-status semantics more tightly.

What changed:

• KV: fetchPaymentQueryStatus now builds minimal HTLC status data and delegates to decidePaymentStatus, so a payment with one settled HTLC plus another unresolved HTLC remains in-flight for count purposes. It still avoids loading full HTLC attempt payloads/routes.
• SQL: CountFilteredPayments now requires both a settled HTLC and no unresolved HTLC attempts when include_incomplete=false.
• Tests: added TestQueryPaymentsCountTotalMixedInflight to cover the settled+unresolved case explicitly.

Re-ran:

• go test ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1
• go test -tags test_db_sqlite ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1
• go test ./payments/db -count=1
• (cd sqldb && go test ./sqlc -count=1)
• git diff --check

[yyforyongyu]

Nice! Could you split the single commit into at least three? one for kvdb, one for sql, one for release notes? Then maybe update the PR description with a benchmark result?

[0xjc65eth]

Done, thanks for the nudge.

I split the branch into four commits:

• KV count path
• SQL/sqlc count path
• tests
• release note

I also updated the PR description with a local SQLite micro-benchmark comparing this branch against origin/master for QueryPayments with CountTotal=true, a date window, and IncludeIncomplete=false.

[0xjc65eth]

Follow-up for the failed CI run: I pushed df69fbdc to address the kvdb_postgres rollback failure and the lint line-length failures.

What changed:

• The KV CountTotal path now first copies payment index entries out of the index cursor, then fetches the lightweight payment query metadata afterward. This avoids nested payment-bucket reads while the SQL-backed KVDB index cursor is still open, which reproduced locally as the conn busy rollback error in kvdb_postgres.
• Reflowed the non-generated line-length issues.

Re-ran locally:

• go test -tags kvdb_postgres ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1\n- go test -tags kvdb_sqlite ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1\n- go test ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1\n- go test ./payments/db -count=1\n- go test -tags test_db_sqlite ./payments/db -run 'TestQueryPayments|TestQueryPaymentsCountTotalMixedInflight' -count=1\n- (cd sqldb && go test ./sqlc -count=1)\n- git diff --check\n\nLocal notes: make lint and test_db_postgres still require Docker on this machine, but the previously failing kvdb_postgres payments/db case now passes locally.