Skip to content

Support MinionContext AuthProvider in Minion executors#17405

Merged
Jackie-Jiang merged 4 commits intoapache:masterfrom
cypherean:shreyaa/minion_auth
Jan 28, 2026
Merged

Support MinionContext AuthProvider in Minion executors#17405
Jackie-Jiang merged 4 commits intoapache:masterfrom
cypherean:shreyaa/minion_auth

Conversation

@cypherean
Copy link
Copy Markdown
Contributor

@cypherean cypherean commented Dec 22, 2025

Minion’s executors previously rebuilt auth from a static task token, forcing the Authorization header and preventing per-request token rotation. This broke short‑lived tokens and custom header requirements.

  • The change makes executors use the runtime AuthProvider from MinionContext for every outbound HTTP call, preserving custom header names/values and enabling token rotation.
  • We keep a strict fallback to the legacy task-token path when no runtime provider is configured, preserving backward compatibility.
  • Tests verify runtime-provider preference and legacy fallback; manual validation confirms getRequestHeaders() is called for requests and headers are attached as expected on every request.

@cypherean cypherean changed the title [pinot-minion] Support MinionContext AuthProvider in Minion executors Support MinionContext AuthProvider in Minion executors Dec 22, 2025
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Dec 22, 2025

Codecov Report

❌ Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.26%. Comparing base (95d43c0) to head (14dea67).
⚠️ Report is 90 commits behind head on master.

Files with missing lines Patch % Lines
.../tasks/BaseMultipleSegmentsConversionExecutor.java 33.33% 2 Missing ⚠️
...ion/tasks/BaseSingleSegmentConversionExecutor.java 0.00% 1 Missing ⚠️
...he/pinot/plugin/minion/tasks/BaseTaskExecutor.java 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17405      +/-   ##
============================================
- Coverage     63.28%   63.26%   -0.02%     
- Complexity     1474     1480       +6     
============================================
  Files          3154     3170      +16     
  Lines        188007   189512    +1505     
  Branches      28782    28998     +216     
============================================
+ Hits         118977   119900     +923     
- Misses        59807    60305     +498     
- Partials       9223     9307      +84     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.19% <63.63%> (-0.08%) ⬇️
java-21 63.23% <63.63%> (+0.01%) ⬆️
temurin 63.26% <63.63%> (-0.02%) ⬇️
unittests 63.26% <63.63%> (-0.02%) ⬇️
unittests1 55.57% <ø> (-0.11%) ⬇️
unittests2 34.07% <63.63%> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables Minion task executors to use a dynamic runtime AuthProvider from MinionContext instead of relying solely on static task tokens. This allows per-request token rotation, custom authorization headers, and better support for short-lived authentication tokens.

Key Changes:

  • Executors now check MinionContext.getTaskAuthProvider() before falling back to the legacy AUTH_TOKEN configuration
  • Three locations updated: BaseSingleSegmentConversionExecutor, BaseMultipleSegmentsConversionExecutor (two call sites), and SegmentUploadContext
  • Tests verify runtime provider preference and legacy fallback behavior

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
BaseMultipleSegmentsConversionExecutorTest.java Added test teardown cleanup and two tests verifying runtime provider preference and task token fallback
BaseSingleSegmentConversionExecutor.java Updated executeTask to prefer runtime AuthProvider over static token with NullAuthProvider check
BaseMultipleSegmentsConversionExecutor.java Applied runtime provider logic in preProcess, executeTask, and SegmentUploadContext constructor

@cypherean cypherean force-pushed the shreyaa/minion_auth branch from 31ce6f0 to 1ab277b Compare January 5, 2026 18:21
@cypherean
Copy link
Copy Markdown
Contributor Author

@tarun11Mavani @swaminathanmanish @xiangfu0 can you ptal?

Copy link
Copy Markdown
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd do it in a different way, where each task can override the auth token if necessary.

Currently auth token is set when starting a task (in TaskFactoryRegistry.runInternal()). Instead, we shouldn't set it, but check if that is explicitly provided. When explicitly provided, use it as the auth provider; if not, fall back to the minion global one.

@Jackie-Jiang Jackie-Jiang added enhancement Improvement to existing functionality security Related to security hardening minion Related to Pinot Minion task framework labels Jan 14, 2026
@cypherean cypherean force-pushed the shreyaa/minion_auth branch 2 times, most recently from 59bd7e0 to c35cd90 Compare January 20, 2026 08:36
@cypherean cypherean force-pushed the shreyaa/minion_auth branch from c35cd90 to 3a9a543 Compare January 20, 2026 09:15
@cypherean
Copy link
Copy Markdown
Contributor Author

I'd do it in a different way, where each task can override the auth token if necessary.

Currently auth token is set when starting a task (in TaskFactoryRegistry.runInternal()). Instead, we shouldn't set it, but check if that is explicitly provided. When explicitly provided, use it as the auth provider; if not, fall back to the minion global one.

Good catch, thanks. Fixed it to check for explicit token first.

@Jackie-Jiang Jackie-Jiang merged commit fc81e8e into apache:master Jan 28, 2026
18 checks passed
@xiangfu0 xiangfu0 added the auth Related to authentication and authorization label Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auth Related to authentication and authorization enhancement Improvement to existing functionality minion Related to Pinot Minion task framework security Related to security hardening

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants