0 votes
in MongoDB by
Can $in handle an array of 10,000,000 entries in MongoDB?

1 Answer

0 votes
by

Look, if you’re planning to “query” your posts for all users in a set of 10,000,000 Users you are well past the stage of “query”. You say yourself that each User has 1,000s of posts so you’re talking about a query for “Users with Posts who live in London” returning 100Ms of records.

100M records isn’t a query, that’s a dataset!

If you’re worried about breaking the $in command, then I highly suggest that you use map/reduce. The Mongo Map/Reduce will create a new collection for you. You can then trim down or summarize this dataset as you see fit.

$in can handle 100,000 entries. I’ve never tried 10,000,000 entries but the query (a query is also a document) has to be smaller than 4mb (like every document) so 10,0000,0000 entries isn’t possible.

Why don’t you include the user and its town in the Posts collection in MongoDB? You can index this town because you can index properties of embedded entities. You no longer have to simulate a join because you can query the Posts on the towns of its embedded users.

Related questions

0 votes
asked Jun 3, 2020 in MongoDB by Robindeniel
0 votes
asked May 21, 2019 in Mathematics by sheetalkhandelwal
...