phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Isart Montane <isart.mont...@gmail.com>
Subject Re: Phoenix and NodeJS
Date Mon, 18 May 2015 19:00:52 GMT
Thanks James.

That code is from the node driver, I will try to get some advice from it's
developer.

Thanks,


On Mon, May 18, 2015 at 6:34 PM, James Taylor <jamestaylor@apache.org>
wrote:

> Hi Isart,
> That code isn't Phoenix code. This sounds like a Node JS issue. Vaclav
> has done a lot with Node JS, so he may be able to give you some tips.
> Thanks,
> James
>
> On Mon, May 18, 2015 at 9:06 AM, Isart Montane <isart.montane@gmail.com>
> wrote:
> > Hi Eli,
> >
> > thanks a lot for your comments. I think you are right. I found the client
> > code that's causing the issue. Do you have an example I can use to patch
> it?
> > is that the recommended way to access phoenix? I've seen on the web that
> > there's also a query server available, is it worth a try?
> >
> >
> > public String[] query(String sql)
> >   {
> >     List<String> lsResults = new ArrayList();
> >     Connection conn = null;
> >     try
> >     {
> >       conn = this.dataSource.getConnection();
> >       ResultSet rs = conn.createStatement().executeQuery(sql);
> >       ResultSetMetaData data = rs.getMetaData();
> >       int numberOfColumns = data.getColumnCount();
> >       List<String> lsRows = new ArrayList();
> >       for (int i = 1; i <= numberOfColumns; i++) {
> >         lsRows.add(data.getColumnName(i));
> >       }
> >       lsResults.add(join("\t", lsRows));
> >       lsRows.clear();
> >       while (rs.next())
> >       {
> >         for (int i = 1; i <= numberOfColumns; i++) {
> >           lsRows.add(rs.getString(i));
> >         }
> >         lsResults.add(join("\t", lsRows));
> >         lsRows.clear();
> >       }
> >       rs.close();
> >       conn.close();
> >     }
> >     catch (Exception e)
> >     {
> >       e.printStackTrace();
> >       return null;
> >     }
> >     return (String[])lsResults.toArray(new String[lsResults.size()]);
> >   }
> >
> > On Mon, May 18, 2015 at 5:43 PM, Eli Levine <elilevine@gmail.com> wrote:
> >>
> >> I don't have info on what your app does with results from Phoenix. If
> the
> >> app is constructing some sort of object representations from Phoenix
> results
> >> and holding on to them, I would look at what the memory footprint of
> that
> >> is. I know this isn't very helpful but at this point I would try to dig
> >> deeper into your app and the NodeJS driver rather than Phoenix, since
> you
> >> mentioned the same queries run fine in sqlline.
> >>
> >> On Mon, May 18, 2015 at 7:30 AM, Isart Montane <isart.montane@gmail.com
> >
> >> wrote:
> >>>
> >>> Hi Eli,
> >>>
> >>> thanks a lot for your answer. That might be a workaround but I was
> hoping
> >>> to get a more generic answer I can apply to the driver/phoenix since
> that
> >>> will require me lots of changes to the code.
> >>>
> >>> Any clue on why it works with sqline but not trough the node driver?
> >>>
> >>> On Mon, May 18, 2015 at 4:20 PM, Eli Levine <elilevine@gmail.com>
> wrote:
> >>>>
> >>>> Have you looked at paging [1] using Phoenix's row-value constructors
> >>>> together with the LIMIT clause? That might be what you are looking
> for.
> >>>>
> >>>> [1] http://phoenix.apache.org/paged.html
> >>>>
> >>>> Eli
> >>>>
> >>>>
> >>>> On Mon, May 18, 2015 at 6:46 AM, Isart Montane <
> isart.montane@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> the company I work for is performing some tests on Phoenix with
> NodeJS.
> >>>>> For simple queries I didn't had any problem, but as soon as I start
> to use
> >>>>> our app I'm getting "process out of memory" errors on the client
> when I runs
> >>>>> queries that return a big number of rows (i.e. 400k) . I think the
> problem
> >>>>> is that the client tries to buffer all the results in RAM and that
> kills it.
> >>>>> The same query runs fine when I run it with sqline.
> >>>>>
> >>>>> So, is there a way to tell the client to stream the results (or
batch
> >>>>> them) instead of buffering them all? is raising the client memory
> the only
> >>>>> solution?
> >>>>>
> >>>>> I'm using phoenix-4.3.1 and
> https://github.com/gaodazhu/phoenix-client
> >>>>> as the NodeJS driver
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Isart Montane
> >>>>
> >>>>
> >>>
> >>
> >
>

Mime
View raw message