feat(uploads): add createFromBuffer for programmatic GridFS storage#3290
Merged
PierreBrisorgueil merged 3 commits intomasterfrom Mar 24, 2026
Merged
feat(uploads): add createFromBuffer for programmatic GridFS storage#3290PierreBrisorgueil merged 3 commits intomasterfrom
PierreBrisorgueil merged 3 commits intomasterfrom
Conversation
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughAdds programmatic buffer upload support: a GridFS helper Changes
Sequence DiagramsequenceDiagram
participant Client as Service Consumer
participant UploadSvc as Uploads Service
participant Config as Configuration
participant GridFS as GridFS Service
participant Stream as Node Stream
Client->>UploadSvc: createFromBuffer(buffer, contentType, kind, metadata)
UploadSvc->>Config: load kind config
alt kind missing
UploadSvc-->>Client: AppError 422
else kind found
UploadSvc->>Config: check contentType in formats
alt disallowed contentType
UploadSvc-->>Client: AppError 422
else allowed
UploadSvc->>UploadSvc: check file size vs limits
alt exceeds limit
UploadSvc-->>Client: AppError 422
else within limit
UploadSvc->>UploadSvc: generate filename
UploadSvc->>GridFS: createFromBuffer(buffer, filename, contentType, metadata)
GridFS->>Stream: create upload stream with ObjectId
GridFS->>Stream: pipe Readable(buffer) into upload stream
Stream-->>GridFS: finish event
GridFS->>GridFS: query Uploads collection for document
GridFS-->>UploadSvc: return stored document
UploadSvc-->>Client: return stored document
end
end
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR extends the uploads module to support programmatic creation of GridFS-backed uploads from an in-memory Buffer, enabling internal services/workers to persist generated assets without going through multipart HTTP upload.
Changes:
- Added
createFromBufferhelper tolib/services/gridfs.jsto write buffers into theuploadsGridFS bucket. - Added a new
snapshotupload kind to uploads configuration with allowed MIME types and size limits. - Added
UploadsService.createFromBuffer(...)plus unit tests for the new service path.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| lib/services/gridfs.js | Adds buffer-to-GridFS upload helper and returns stored upload document. |
| modules/uploads/config/config.uploads.js | Adds snapshot upload kind config (formats + size limit). |
| modules/uploads/services/uploads.service.js | Adds service method to validate kind/content-type/size and delegate to GridFS helper. |
| modules/uploads/tests/uploads.createFromBuffer.unit.tests.js | Adds unit tests for new service method behavior (success + some validation cases). |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@modules/uploads/services/uploads.service.js`:
- Around line 81-83: Extract the inline MIME_TO_EXT map out of the function into
a module-level constant (e.g., MIME_TO_EXT) or into a shared utility module and
import it into uploads.service.js, then replace the inline definition with a
reference to that constant where ext is computed (the current code using
MIME_TO_EXT[contentType] || 'bin' that feeds into filename =
`${crypto.randomBytes(32).toString('hex')}.${ext}`); ensure the constant is
exported if moved to a shared util and update any imports accordingly.
- Around line 69-79: The createFromBuffer function assumes buffer is a valid
Buffer but can throw a TypeError when buffer is null/undefined or not a Buffer;
add an explicit validation at the top of createFromBuffer using
Buffer.isBuffer(buffer) (or equivalent) and throw an AppError with code
'SERVICE_ERROR' and status 422 if the check fails, and only access buffer.length
after this validation; update any relevant tests or callers if needed to ensure
they pass a proper Buffer to createFromBuffer.
- Around line 69-87: The createFromBuffer function currently assumes
kindConfig.formats exists and calls .includes, which can throw a TypeError if
formats is missing; add a defensive check that verifies kindConfig.formats is an
Array (or non-empty) before using .includes and throw the same AppError used for
invalid content types (e.g., `Upload: content type ... not allowed for kind
...`) when formats is missing or not an array; locate the check around the
existing use of kindConfig.formats in createFromBuffer and use kindConfig (from
config.uploads?.[kind]) to validate formats first, then proceed to the existing
includes check and subsequent logic.
In `@modules/uploads/tests/uploads.createFromBuffer.unit.tests.js`:
- Around line 73-130: Add tests that assert UploadsService.createFromBuffer
rejects when buffer input is null, undefined, or an empty Buffer; call
UploadsService.createFromBuffer with null, undefined, and Buffer.alloc(0) using
a valid contentType and kind, assert each invocation rejects (using
rejects.toThrow or similar), and verify mockGridfs.createFromBuffer was not
called; place them alongside the existing tests so they cover input validation
for UploadsService.createFromBuffer and reference mockGridfs.createFromBuffer to
assert no upstream storage call occurs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 500a8755-18e2-412b-9e88-db4ba9a54d1c
📒 Files selected for processing (4)
lib/services/gridfs.jsmodules/uploads/config/config.uploads.jsmodules/uploads/services/uploads.service.jsmodules/uploads/tests/uploads.createFromBuffer.unit.tests.js
…ty, tests - Fix Readable.from(buffer) → Readable.from([buffer]) to avoid byte-by-byte iteration - Add readable error handler for reliable Promise rejection on stream failures - Add Buffer.isBuffer validation with clear AppError (422) for invalid inputs - Add defensive Array.isArray check for kindConfig.formats - Extract MIME_TO_EXT map to module-level constant - Add tests for null/undefined buffer, non-Buffer input, and missing formats
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@modules/uploads/tests/uploads.createFromBuffer.unit.tests.js`:
- Around line 144-150: Add a unit test for UploadsService.createFromBuffer to
cover an empty Buffer case: call createFromBuffer with Buffer.alloc(0), the MIME
and type strings, then assert mockGridfs.createFromBuffer was called once and
that the buffer argument passed into mockGridfs.createFromBuffer has length 0
(verifying the empty buffer is forwarded/stored). This ensures
UploadsService.createFromBuffer and mockGridfs.createFromBuffer handle 0-byte
buffers as expected.
- Around line 152-160: Add a concise comment above the runtime mutation in the
test 'should throw error when kind has no formats configured' explaining that
mockConfig.uploads is mutated at runtime because UploadsService.reads config
dynamically; e.g., clarify that setting mockConfig.uploads.broken = { kind:
'broken', limits: { fileSize: 1024 } } happens after import so the service will
pick up the change and the test then calls UploadsService.createFromBuffer and
asserts mockGridfs.createFromBuffer is not invoked.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5b965882-a4c6-4df6-b38d-aab6b429f52a
📒 Files selected for processing (3)
lib/services/gridfs.jsmodules/uploads/services/uploads.service.jsmodules/uploads/tests/uploads.createFromBuffer.unit.tests.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
createFromBuffermethod to the uploads module, enabling programmatic file storage in GridFS without requiring an HTTP multipart upload. Closes #3289.lib/services/gridfs.js— newuploadFromBuffer(buffer, filename, bucketName)helper that opens a GridFS upload stream and writes a Buffer directlymodules/uploads/config/config.uploads.js— addscreateFromBufferconfiguration (allowed MIME types, max file size) alongside existing upload configmodules/uploads/services/uploads.service.js— newcreateFromBuffer({ buffer, filename, contentType, metadata })service method that validates inputs, calls the GridFS helper, and returns the stored file documentmodules/uploads/tests/uploads.createFromBuffer.unit.tests.js— comprehensive unit tests covering success path, missing-buffer/filename validation, MIME-type rejection, and file-size limit enforcementTest plan
npm run lintpasses (0 errors)npm testpasses (269 tests, including the new createFromBuffer tests)Summary by CodeRabbit
New Features
Tests