Issue involves understanding and modifying deferred insert behavior with unclear setup requirements.
The issue involves modifying the behavior of deferred inserts in Frappe, specifically addressing the arbitrary limit of 500 records per queue and the resetting of document creation times. It requires understanding of Frappe's caching mechanism and potential architectural changes to ensure persistence. The setup and testing requirements are not fully clear.
save_to_db only dumps a maximum of 500 records per queue into the Database hourly.
Restricting to 500 (arbitrary & very low number) leads to starvation when you're pushing records in higher volumes. We could easily just let the job flush as many records as it can instead?
Document creation is reset
Since doc.insert is used within the deferred_insert API, the creation time is reset during DB insert. Essentially we end up re-writing the original information.
Claim this issue to let others know you're working on it. You'll earn 20 points when you complete it!