phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: Phoenix and NodeJS
Date Mon, 18 May 2015 16:34:31 GMT
Hi Isart,
That code isn't Phoenix code. This sounds like a Node JS issue. Vaclav
has done a lot with Node JS, so he may be able to give you some tips.
Thanks,
James

On Mon, May 18, 2015 at 9:06 AM, Isart Montane <isart.montane@gmail.com> wrote:
> Hi Eli,
>
> thanks a lot for your comments. I think you are right. I found the client
> code that's causing the issue. Do you have an example I can use to patch it?
> is that the recommended way to access phoenix? I've seen on the web that
> there's also a query server available, is it worth a try?
>
>
> public String[] query(String sql)
>   {
>     List<String> lsResults = new ArrayList();
>     Connection conn = null;
>     try
>     {
>       conn = this.dataSource.getConnection();
>       ResultSet rs = conn.createStatement().executeQuery(sql);
>       ResultSetMetaData data = rs.getMetaData();
>       int numberOfColumns = data.getColumnCount();
>       List<String> lsRows = new ArrayList();
>       for (int i = 1; i <= numberOfColumns; i++) {
>         lsRows.add(data.getColumnName(i));
>       }
>       lsResults.add(join("\t", lsRows));
>       lsRows.clear();
>       while (rs.next())
>       {
>         for (int i = 1; i <= numberOfColumns; i++) {
>           lsRows.add(rs.getString(i));
>         }
>         lsResults.add(join("\t", lsRows));
>         lsRows.clear();
>       }
>       rs.close();
>       conn.close();
>     }
>     catch (Exception e)
>     {
>       e.printStackTrace();
>       return null;
>     }
>     return (String[])lsResults.toArray(new String[lsResults.size()]);
>   }
>
> On Mon, May 18, 2015 at 5:43 PM, Eli Levine <elilevine@gmail.com> wrote:
>>
>> I don't have info on what your app does with results from Phoenix. If the
>> app is constructing some sort of object representations from Phoenix results
>> and holding on to them, I would look at what the memory footprint of that
>> is. I know this isn't very helpful but at this point I would try to dig
>> deeper into your app and the NodeJS driver rather than Phoenix, since you
>> mentioned the same queries run fine in sqlline.
>>
>> On Mon, May 18, 2015 at 7:30 AM, Isart Montane <isart.montane@gmail.com>
>> wrote:
>>>
>>> Hi Eli,
>>>
>>> thanks a lot for your answer. That might be a workaround but I was hoping
>>> to get a more generic answer I can apply to the driver/phoenix since that
>>> will require me lots of changes to the code.
>>>
>>> Any clue on why it works with sqline but not trough the node driver?
>>>
>>> On Mon, May 18, 2015 at 4:20 PM, Eli Levine <elilevine@gmail.com> wrote:
>>>>
>>>> Have you looked at paging [1] using Phoenix's row-value constructors
>>>> together with the LIMIT clause? That might be what you are looking for.
>>>>
>>>> [1] http://phoenix.apache.org/paged.html
>>>>
>>>> Eli
>>>>
>>>>
>>>> On Mon, May 18, 2015 at 6:46 AM, Isart Montane <isart.montane@gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> the company I work for is performing some tests on Phoenix with NodeJS.
>>>>> For simple queries I didn't had any problem, but as soon as I start to
use
>>>>> our app I'm getting "process out of memory" errors on the client when
I runs
>>>>> queries that return a big number of rows (i.e. 400k) . I think the problem
>>>>> is that the client tries to buffer all the results in RAM and that kills
it.
>>>>> The same query runs fine when I run it with sqline.
>>>>>
>>>>> So, is there a way to tell the client to stream the results (or batch
>>>>> them) instead of buffering them all? is raising the client memory the
only
>>>>> solution?
>>>>>
>>>>> I'm using phoenix-4.3.1 and https://github.com/gaodazhu/phoenix-client
>>>>> as the NodeJS driver
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Isart Montane
>>>>
>>>>
>>>
>>
>

Mime
View raw message