From one of our clients, I’m curious if y’all have any suggestions?
- Phoenix has this nifty little feature where it creates a temp file (in /var/tmp with the prefix ResultSpooler) to store query results if the query results are too big to fit in memory.
- The default threshold for the size of the query result is 20mb, meaning results under 20mb should use memory and results over 20mb should use the temp file.
- In theory, this temp file should get cleaned up, regardless of whether it was used to store the result or not
- Our problem is that we are getting tons of 53mb temp files that are never getting cleaned up, which is killing our client
- Phoenix seems to be having issues following the threshold for the result size, and having issues with cleaning up the temp files
As far as I've been able to tell while Googling this problem, the Apache folks fixed this in version 3.0 and 4.0 of Phoenix (we're using 4.0), but we're using the HWX Phoenix jar, and they aren't being particularly helpful with potential solutions.
We have tried:
- Modifying the Phoenix jar on the client so that it only creates temp files when absolutely necessary, and modifying the threshold (unsuccessful)
- Using the most recent Apache Phoenix jar on both client and server (serious compatibility issues with the HWX HDP stuff, so we rolled back)
Sent from Alex Newman on a small glass box.