NULL
|
Claude
|
Claude
|
NULL
|
|
Firefox
|
Firefox
|
NULL
|
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
|
Use lazyById() instead of cursor(): * Laravel' Use lazyById() instead of cursor(): * Laravel's lazyById(250) fetches records in chunks under the hood (allowing for eager loading of relations without N+1) but yields them as a flat generator to the foreach loop. This gives you the clean syntax of a generator with the safety of chunked eager loading.
Dispatch to ElasticSearch Inside the Loop: * Do not wait until the end of the method to return massive collections.
Keep a counter. When $documentsToUpdate reaches a certain threshold (e.g., 500 documents), dispatch them to ElasticSearch, empty the SimpleCollection, and continue the loop.
Group Sentry Exceptions:
Consider implementing a circuit breaker or batching error logs so that a sudden spike in formatting errors doesn't result in thousands of synchronous API calls to Sentry....
|
NULL
|
NULL
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
|
A. The "Infinite Accumulation" Memory Leak A. The "Infinite Accumulation" Memory Leak
The primary goal of this refactor was memory optimization, but the implementation defeats its own purpose.
The Issue: The code iterates over the database cursor and manually unsets $entityModel to free memory. However, it simultaneously instantiates new Document objects and adds them to $documentsToUpdate (and IDs to $documentsToDelete).
The Bottleneck: These collections are not flushed or processed within the loop; they are accumulated and returned at the end of the method. If this script processes 500,000 records, $documentsToUpdate will hold 500,000 ElasticSearch Document objects in RAM at once, inevitably leading to an Out Of Memory (OOM) fatal error.
B. Laravel cursor() and N+1 Query Problems
Replacing chunkByIdDesc() with cursor() introduces a severe database performance risk.
The Issue: Laravel's cursor() executes a single query and uses a PHP generator to yield results one by one via a PDO cursor. Because it does n......
|
NULL
|
NULL
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
|
A. The "Infinite Accumulation" Memory Leak A. The "Infinite Accumulation" Memory Leak
The primary goal of this refactor was memory optimization, but the implementation defeats its own purpose.
The Issue: The code iterates over the database cursor and manually unsets $entityModel to free memory. However, it simultaneously instantiates new Document objects and adds them to $documentsToUpdate (and IDs to $documentsToDelete).
The Bottleneck: These collections are not flushed or processed within the loop; they are accumulated and returned at the end of the method. If this script processes 500,000 records, $documentsToUpdate will hold 500,000 ElasticSearch Document objects in RAM at once, inevitably leading to an Out Of Memory (OOM) fatal error.
B. Laravel cursor() and N+1 Query Problems
Replacing chunkByIdDesc() with cursor() introduces a severe database performance risk.
The Issue: Laravel's cursor() executes a single query and uses a PHP generator to yield results one by one via a PDO cursor. Because it does n......
|
NULL
|
NULL
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
|
Claude
|
Claude
|
NULL
|
|
|
What
|
Claude
|
Claude
|
NULL
|
|
do you think about these comments
|
Claude
|
Claude
|
NULL
|
NULL
|
Claude
|
Claude
|
NULL
|
|
A. The "Infinite Accumulation" Memory Leak A. The "Infinite Accumulation" Memory Leak
The primary goal of this refactor was memory optimization, but the implementation defeats its own purpose.
The Issue: The code iterates over the database cursor and manually unsets $entityModel to free memory. However, it simultaneously instantiates new Document objects and adds them to $documentsToUpdate (and IDs to $documentsToDelete).
The Bottleneck: These collections are not flushed or processed within the loop; they are accumulated and returned at the end of the method. If this script processes 500,000 records, $documentsToUpdate will hold 500,000 ElasticSearch Document objects in RAM at once, inevitably leading to an Out Of Memory (OOM) fatal error.
B. Laravel cursor() and N+1 Query Problems
Replacing chunkByIdDesc() with cursor() introduces a severe database performance risk.
The Issue: Laravel's cursor() executes a single query and uses a PHP generator to yield results one by one via a PDO cursor. Because it does n......
|
NULL
|
NULL
|
NULL
|
|
|
Claude
|
Claude
|
NULL
|
|
and
|
Claude
|
Claude
|
NULL
|
|
sugg
|
Claude
|
Claude
|
NULL
|
|
s
|
Claude
|
Claude
|
NULL
|
|
e
|
Claude
|
Claude
|
NULL
|
|
stions
|
Claude
|
Claude
|
NULL
|
NULL
|
Claude
|
Claude
|
NULL
|
|
A. The "Infinite Accumulation" Memory Leak A. The "Infinite Accumulation" Memory Leak
The primary goal of this refactor was memory optimization, but the implementation defeats its own purpose.
The Issue: The code iterates over the database cursor and manually unsets $entityModel to free memory. However, it simultaneously instantiates new Document objects and adds them to $documentsToUpdate (and IDs to $documentsToDelete).
The Bottleneck: These collections are not flushed or processed within the loop; they are accumulated and returned at the end of the method. If this script processes 500,000 records, $documentsToUpdate will hold 500,000 ElasticSearch Document objects in RAM at once, inevitably leading to an Out Of Memory (OOM) fatal error.
B. Laravel cursor() and N+1 Query Problems
Replacing chunkByIdDesc() with cursor() introduces a severe database performance risk.
The Issue: Laravel's cursor() executes a single query and uses a PHP generator to yield results one by one via a PDO cursor. Because it does n......
|
NULL
|
NULL
|
NULL
|
|
|
Claude
|
Claude
|
NULL
|
|
|
Claude
|
Claude
|
NULL
|
NULL
|
Claude
|
Claude
|
NULL
|
|
Firefox
|
Firefox
|
NULL
|
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
|
Claude
|
Claude
|
NULL
|
|
NULL
|
Claude
|
Claude
|
NULL
|
|
Firefox
|
Firefox
|
NULL
|
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|
|
Claude
|
Claude
|
NULL
|
|
NULL
|
Claude
|
Claude
|
NULL
|
NULL
|
Claude
|
Claude
|
NULL
|
NULL
|
Claude
|
Claude
|
NULL
|
|
Slack
|
Slack
|
NULL
|
|
NULL
|
Slack
|
Vasil Vasilev (DM) - Jiminny Inc - 5 new items - S Vasil Vasilev (DM) - Jiminny Inc - 5 new items - Slack...
|
NULL
|
NULL
|
Slack
|
Vasil Vasilev (DM) - Jiminny Inc - 5 new items - S Vasil Vasilev (DM) - Jiminny Inc - 5 new items - Slack...
|
NULL
|
|
Firefox
|
Firefox
|
NULL
|
This comment was marked as outdated. Hide comment
|
NULL
|
Firefox
|
Jy 20820 es reindex stream model hydration by Vasi Jy 20820 es reindex stream model hydration by Vasil-Jiminny · Pull Request #12059 · jiminny/app — Work...
|
NULL
|