phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: Phoenix Query Server query logging
Date Fri, 03 Mar 2017 16:33:14 GMT
https://github.com/apache/calcite/blob/master/avatica/server/src/main/java/org/apache/calcite/avatica/jdbc/JdbcMeta.java

This is ultimately where the requests from the client using the thin 
JDBC driver get executed inside of PQS. The API's aren't 100%, but it 
should be obvious what is getting invoked with you're familiar with the 
JDBC APIs.

Michael Young wrote:
> Josh,
> I am interested in looking at adding this to Avatica myself, although
> I'm not familiar with that code base.
>
> Can you point me to where in the avatica code I should look at to add
> this logging?
>
> Cheers
>
>
> On Tue, Feb 28, 2017 at 4:15 AM, Josh Elser <elserj@apache.org
> <mailto:elserj@apache.org>> wrote:
>
>     No, I don't believe there is any log4j logging done in PQS that
>     would show queries being executed.
>
>     Ideally, we would have a "query log" in Phoenix which would present
>     an interface to this data and it wouldn't require anything special
>     in PQS. However, I wouldn't be opposed to some trivial additions to
>     PQS (Avatica, really) to add a simple logging as a stopgap.
>
>
>     On Feb 27, 2017 20:49, "Michael Young" <yomaiquin@gmail.com
>     <mailto:yomaiquin@gmail.com>> wrote:
>
>         I hadn't seen a reply to my earlier question.
>
>         We have business analysts running queries using BI tools (like
>         Tableau) which connect via the Phoenix Query Server.
>
>         How can we log all SELECT queries (raw query, start time, end
>         time, etc...)?
>
>         Any way to tweak log4j or other properties to get this?  The
>         TRACE logging I tried (mentioned in my post above) was way too
>         dense to be useful for reporting usage, and doesn't seem to show
>         the full SQL query params and query start/end times.  Also, it
>         logs every UPSERT during data load (which overwhelms the log
>         files).  We really just need SELECTS logged.
>
>
>
>         On Tue, Jan 31, 2017 at 5:10 PM, Michael Young
>         <yomaiquin@gmail.com <mailto:yomaiquin@gmail.com>> wrote:
>
>             Does the Phoenix Query Server have an option to log the SQL
>             statements which are executed?
>
>             We see there are ways to get various PQS trace logs
>             modifying the log4j settings used by the queryserver.py:
>
>             log4j.org.apache.phoenix.jdbc.PhoenixStatementFactory=TRACE
>             (or DEBUG)
>             log4j.org.apache.phoenix.jdbc.PhoenixStatement=TRACE
>             log4j.logger.org.apache.calcite.avatica=TRACE
>             log4j.logger.org.apache.phoenix.queryserver.server=TRACE
>             etc...
>
>             but the data in the trace logs (which show SQL statements)
>             are not particularly user friendly.  And it does not seem
>             straightforward to get to end-to-end query execution times.
>
>             Any suggestions how to get simple SQL logs (raw query,
>             execution time, ...)?  The idea is to monitor user activity
>             and take action if query times are slow, or timeout.
>
>             Thanks,
>             Michael
>
>
>
>

Mime
View raw message