GH-123516: Improve JIT memory consumption by invalidating cold executors#124443
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR succeeds #123402 and reworks the approach to use the eval breaker for the invalidation call instead of executor creation or gc (thanks @markshannon!). In experimenting, I tried a couple of different thresholds of 10k, 100k, and 1 million runs. The benchmarks for 100k and 1 million were most promising. Here are some relevant stats for quick reference:
100k
- -2.4% memory
- Roughly the same performance-wise
1 million
After chatting with @brandtbucher, I've opted to open this PR with the 100k threshold. One thing to note is that we are potentially a little too liberal in invalidating executors with this threshold, but with the lack of movement in performance and a more substantial decrease in memory usage, it seemed justified. We can continue to iterate here and consider making this tunable in the future.