phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Newman" <alex.new...@wandisco.com>
Subject Constant temp offloading to disk
Date Tue, 04 Nov 2014 15:47:57 GMT
>From one of our clients, I’m curious if y’all have any suggestions?


---
- Phoenix has this nifty little feature where it creates a temp file (in /var/tmp with the
prefix ResultSpooler) to store query results if the query results are too big to fit in memory.
- The default threshold for the size of the query result is 20mb, meaning results under 20mb
should use memory and results over 20mb should use the temp file.
- In theory, this temp file should get cleaned up, regardless of whether it was used to store
the result or not
- Our problem is that we are getting tons of 53mb temp files that are never getting cleaned
up, which is killing our client
- Phoenix seems to be having issues following the threshold for the result size, and having
issues with cleaning up the temp files


As far as I've been able to tell while Googling this problem, the Apache folks fixed this
in version 3.0 and 4.0 of Phoenix (we're using 4.0), but we're using the HWX Phoenix jar,
and they aren't being particularly helpful with potential solutions.


We have tried:
- Modifying the Phoenix jar on the client so that it only creates temp files when absolutely
necessary, and modifying the threshold (unsuccessful)
- Using the most recent Apache Phoenix jar on both client and server (serious compatibility
issues with the HWX HDP stuff, so we rolled back)


-Alex

Sent from Alex Newman on a small glass box.
404.507.6749
-- 


Listed on the London Stock Exchange: WAND 
<http://www.bloomberg.com/quote/WAND:LN>

THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY BE 
PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its 
subsidiaries, ("WANdisco") does not waive any confidentiality or privilege. 
 If you are not the intended recipient, please notify us immediately and 
destroy the message without disclosing its contents to anyone.  Any 
distribution, use or copying of this e-mail or the information it contains 
by other than an intended recipient is unauthorized.  The views and 
opinions expressed in this e-mail message are the author's own and may not 
reflect the views and opinions of WANdisco, unless the author is authorized 
by WANdisco to express such views or opinions on its behalf.  All email 
sent to or from this address is subject to electronic storage and review by 
WANdisco.  Although WANdisco operates anti-virus programs, it does not 
accept responsibility for any damage whatsoever caused by viruses being 
passed.

Mime
View raw message