SCIM MongoDB query already fixed - no changes needed

[Copilot]

The reported 16MB MongoDB aggregation error when fetching groups with 20k+ members has already been resolved in the current codebase (v6.0.6+).

Analysis

The issue was caused by a LINQ join that MongoDB translated into a single aggregate command, exceeding the 16MB BSON limit:

// Problematic pattern (no longer in codebase)
var filteredRepresentationsWithAttributes = from a in filteredRepresentations
                                            join b in _scimDbContext.SCIMRepresentationAttributeLst.AsQueryable() 
                                            on a.Id equals b.RepresentationId into Attributes
                                            select new { Representation = a, Attributes = Attributes };

Current Implementation

File: src/Scim/SimpleIdServer.Scim.Persistence.MongoDB/SCIMRepresentationQueryRepository.cs (lines 97-111)

The fix uses separate queries combined in memory:

// 1. Fetch representations
var representationsList = await filteredRepresentations.ToListAsync(cancellationToken);
var representationIds = representationsList.Select(r => r.Id).ToList();

// 2. Fetch attributes separately (uses RepresentationId index)
var attributes = await _scimDbContext.SCIMRepresentationAttributeLst.AsQueryable()
    .Where(a => representationIds.Contains(a.RepresentationId))
    .ToListAsync(cancellationToken);

// 3. Group and combine in memory
var attributesByRepId = attributes.GroupBy(a => a.RepresentationId)
    .ToDictionary(g => g.Key, g => g.ToList());

This avoids MongoDB's aggregation limit while maintaining performance through indexed queries.

Verification

• Build: ✅ Success
• Tests: ✅ 7/7 passed
• Indexes: ✅ Present on RepresentationId (ServiceProviderExtensions.cs:78)

No code changes required - closing as already fixed.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[SCIM] Get group error "Size must be between 0 and 16793600(16MB)"</issue_title>
<issue_description>Hello guys,

we have this issue when the group has thousands of users inside:

Groups?filter=displayName+eq+"GLOBAL-READ"

The error is:
{
"schemas": [
"urn:ietf:params:scim:api:messages:2.0:Error"
],
"status": "500",
"scimType": "internalServerError",
"detail": "Command aggregate failed: PlanExecutor error during aggregation :: caused by :: BSONObj size: 76105902 (0x48948AE) is invalid. Size must be between 0 and 16793600(16MB) First element: Representation: { _id: "876c7a69-d1be-4e05-aa65-d8406f0bda79", _t: [ "SCIMRepresentation", "SCIMRepresentationModel" ], ExternalId: null, ResourceType: "Group", Version: 6731, DisplayName: "GLOBAL-READ", RealmName: "", Created: new Date(1752069237447), LastModified: new Date(1753778133279), SchemaRefs: [ { $ref: "scimSchemas", $id: "urn:ietf:params:scim:schemas:core:2.0:Group" } ], AttributeRefs: null }."
}

This Group has around 20,000 members.

Can you please check?

Regards.
Gabriele</issue_description>

<agent_instructions>We have about 20k users in the same group, the get it takes more than 13 minutes</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@simpleidserver Hello,

There is indeed an issue in the FindSCIMRepresentations function in the SCIMRepresentationQueryRepository class.
When a group contains a large number of attributes, the following query throws an exception. The LINQ query is translated into a MongoDB aggregate command, and the resulting data size exceeds 16 MB.

var filteredRepresentationsWithAttributes = from a in filteredRepresentations
                                            join b in _scimDbContext.SCIMRepresentationAttributeLst.AsQueryable() on a.Id equals b.RepresentationId into Attributes
                                            select new
                                            {
                                                Representation = a,
                                                Attributes = Attributes
                                            };
											

This issue should be fixed in the release/v6.0.4 branch. Could you please try again and confirm?

KR,

SID

• Fixes [SCIM] Get group error "Size must be between 0 and 16793600(16MB)" #909

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.