Oct 20, 2022
Yes, this test was run with 100 million records.
Given than 62,500 records take 1 MB of space at application level, 100 million would take about 1.5 GB of memory. The idea to join on application level relies that we define our data models efficiently such that only the identifiers are needed to join.
With the above effieciency the application layer can be stateless and scaled.