Skip to content

Per 10435 storage purchase webhook#675

Merged
liam-lloyd merged 1 commit intomainfrom
per-10435_storage_purchase_webhook
Mar 24, 2026
Merged

Per 10435 storage purchase webhook#675
liam-lloyd merged 1 commit intomainfrom
per-10435_storage_purchase_webhook

Conversation

@liam-lloyd
Copy link
Copy Markdown
Member

@liam-lloyd liam-lloyd commented Mar 5, 2026

Testing Instructions:

  1. Install the Stripe CLI
  2. Run stripe listen --forward-to local.permanent.org/api/v2/storage-purchases/stripe/webhook
  3. Set the STRIPE_WEBHOOK_SECRET in your .env to the secret output in step 2.
  4. In your local database, set stripecustomerid to the ID of a customer in the stripe test environment for one of your test users
  5. Run stripe trigger payment_intent.succeeded --override payment_intent:customer=<CUSTOMER_ID_FROM_STEP_4>
  6. Check ledger_financial to confirm that storage was issued to your test account

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.32%. Comparing base (88c12d1) to head (26e34d7).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #675   +/-   ##
=======================================
  Coverage   98.31%   98.32%           
=======================================
  Files         107      107           
  Lines        2670     2744   +74     
  Branches      449      464   +15     
=======================================
+ Hits         2625     2698   +73     
- Misses         41       42    +1     
  Partials        4        4           
Flag Coverage Δ
api 98.32% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Member

@cecilia-donnelly cecilia-donnelly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of the first commit don't seem to match the commit message -- looks like we had a rebase mixup maybe?

You mentioned that the Stripe docs were a little opaque. Can you add a link to the instructions you were following for handling a webhook?

It would also be helpful to write a PR message. I don't think this is testable yet but might be wrong about that - perhaps there's a curl that could be constructed? Overall I think some documentation here is warranted, since someone will need to write the web-app counterpart. Since you've gathered up the pieces from Stripe to understand the intended (ha) flow, can you write down what you've learned?

Comment thread .env.template Outdated
ARCHIVEMATICA_PROCESSING_WORKFLOW="local_<dev_name>"

STRIPE_SECRET_KEY="<Stripe secret key here>"
STRIPE_SECRET_KEY=<Stripe secret key here>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the quotes because in my lambda meandering one of the corridors was an error that I ran into where I couldn't source my .env because of the unquoted angle brackets. Probably we should fill in the secrets for the most part, but in cases where we have some left unfilled it's nice to not cause errors that aren't needed.

@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch from acdfcd6 to c3dc8a0 Compare March 9, 2026 22:31
@liam-lloyd
Copy link
Copy Markdown
Member Author

I added links to the Stripe documentation pages I found most valuable to a Confluence Page. I suspect we'll add more links here as we proceed with the client migrations, as I am less sure what documentation pages will be most useful for that.

@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch 3 times, most recently from c01e4ba to cb840e5 Compare March 11, 2026 17:42
Copy link
Copy Markdown
Member

@cecilia-donnelly cecilia-donnelly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the first commit is still not quite right. I tried to run the test you wrote up in the PR description (thank you!) but got a 404 with no https and then an error because my local cert isn't real when I explicitly used https. Did you run into that?

Comment thread .env.template Outdated
@liam-lloyd
Copy link
Copy Markdown
Member Author

liam-lloyd commented Mar 12, 2026

It looks like the first commit is still not quite right. I tried to run the test you wrote up in the PR description (thank you!) but got a 404 with no https and then an error because my local cert isn't real when I explicitly used https. Did you run into that?

I do get the certificate error if I use https in the webhook URL, but it's working for me without that. And the first commit shouldn't have been part of this PR, it's already merged (although it does look like I missed a test coverage warning on that one)

@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch 2 times, most recently from 6e68fd1 to 3167709 Compare March 12, 2026 18:31
@slifty slifty requested a review from Copilot March 16, 2026 18:45
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for handling Stripe “storage purchase” webhooks in the API, crediting storage via the legacy backend, and wiring required secrets through Terraform for dev/staging/prod.

Changes:

  • Added /api/v2/storage-purchases/stripe/webhook endpoint with raw-body parsing and Stripe signature verification.
  • Implemented webhook handler that looks up the account by Stripe customer ID and calls the legacy backend to credit storage (plus optional Slack notification).
  • Added Terraform variables/secrets/env wiring for Stripe webhook secret and legacy credit-storage secret (and Slack webhook in prod), plus local env/docs updates.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
terraform/test_cluster/variables.tf Adds dev/staging variables for legacy credit-storage secret and Stripe webhook secrets
terraform/test_cluster/secrets.tf Adds new secret keys to dev/staging Kubernetes secrets
terraform/test_cluster/stela_dev_deployment.tf Injects legacy credit-storage secret + Stripe webhook secret into dev deployment
terraform/test_cluster/stela_staging_deployment.tf Injects legacy credit-storage secret + Stripe webhook secret into staging deployment
terraform/prod_cluster/variables.tf Adds prod variables for legacy credit-storage secret, Stripe webhook secret, and Slack webhook URL
terraform/prod_cluster/secrets.tf Adds new secret keys to prod Kubernetes secret
terraform/prod_cluster/stela_prod_deployment.tf Injects legacy credit-storage secret + Stripe webhook secret + Slack webhook URL into prod deployment
packages/api/src/app.ts Registers express.raw() for the Stripe webhook route before express.json()
packages/api/src/storage_purchase/controller.ts Adds the Stripe webhook route handler
packages/api/src/storage_purchase/validators.ts Adds validation for webhook raw-body payload
packages/api/src/storage_purchase/service.ts Implements webhook processing, legacy credit call, and Slack notification
packages/api/src/storage_purchase/queries/get_account_by_stripe_customer_id.sql Adds query to map Stripe customer ID to account
packages/api/src/storage_purchase/controller.test.ts Adds endpoint tests covering webhook behaviors
packages/api/src/legacy_client.ts Adds creditStorage legacy backend call
packages/api/src/legacy_client.test.ts Adds unit test for creditStorage request shape
README.md Documents new legacy/Stripe webhook env vars (partial)
.env.template Adds placeholders for new legacy/Stripe webhook env vars (partial)
Comments suppressed due to low confidence (1)

.env.template:50

  • .env.template includes the new Stripe/legacy secrets, but it’s missing SLACK_WEBHOOK_URL, which is now read by the API for purchase notifications. Consider adding it (with an empty/default placeholder) so local environments can opt in consistently.

STRIPE_SECRET_KEY="<Stripe secret key here>"
STRIPE_WEBHOOK_SECRET="<webhook secret here>"


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread README.md Outdated
Comment thread terraform/test_cluster/secrets.tf Outdated
Comment thread packages/api/src/storage_purchase/controller.ts
Comment thread packages/api/src/storage_purchase/service.ts
Comment thread packages/api/src/storage_purchase/service.ts Outdated
Comment thread packages/api/src/legacy_client.ts
@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch from 3167709 to 2e5438b Compare March 16, 2026 19:10
Copy link
Copy Markdown
Collaborator

@slifty slifty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't know if any these comments warrant action so I'm leaving this as a neutral review! We probably should be explicit about handling the other event types at least (or making it clear we're having a "success, failure, and everything else" path

Comment thread packages/api/src/legacy_client.ts
signature: string | string[],
): Promise<void> => {
const {
env: { STRIPE_WEBHOOK_SECRET: webhookSecret },
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other places we tend to load envs at the top of a given module (which then fails right away / flags the issue as part of the deployment rather than on API access). One could argue for either approach, but we should probably aim to be consistent. I do prefer the "fail on deploy" since that's easier to detect and recover from.

Comment thread packages/api/src/legacy_client.ts
Comment thread packages/api/src/storage_purchase/service.ts Outdated
logger.info("Stripe payment failed", {
paymentIntentId: paymentIntent.id,
});
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there are a bunch of other types of event that could happen (which will all translate to a silent success response without any crediting OR log entry)

I think this is a link to the top of the list: https://docs.stripe.com/api/events/types?q=payment_intent.payment_failed#event_types-payment_intent.amount_capturable_updated

I don't know if we need to handle them individually, but it also might be reasonable to use a switch block instead of if statements, and to either be exhaustive in listing the cases or just make the default behavior more explicit.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to handle any other messages, and I don't want to include cases for all the events we're not handling because that would be many lines of code that don't do anything, and also it would imply that the list in our code is exhaustive, which it might we stop being as Stripe adds events. Using a switch and commenting in the default case that this must be an event we don't care about seems like a good idea though.

Comment thread terraform/test_cluster/variables.tf
@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch from 2e5438b to 2168ad5 Compare March 16, 2026 19:58
@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch from 2168ad5 to b3cdcc0 Compare March 17, 2026 21:07
Copy link
Copy Markdown
Collaborator

@slifty slifty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay! Looks good to me.

@cecilia-donnelly cecilia-donnelly dismissed their stale review March 23, 2026 18:46

@liam-lloyd addressed my concerns.

We are working on migrating functionality from donations-firebase to
this repo. As part of that effort, this commit adds a Stripe webhook to
trigger storage credits upon receiving payment_intent.success events.
@liam-lloyd liam-lloyd force-pushed the per-10435_storage_purchase_webhook branch from b3cdcc0 to 26e34d7 Compare March 24, 2026 23:21
@liam-lloyd liam-lloyd merged commit f022771 into main Mar 24, 2026
34 checks passed
@liam-lloyd liam-lloyd deleted the per-10435_storage_purchase_webhook branch March 24, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants