livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsch, Tim" <>
Subject How to tune Livy for fast queries
Date Wed, 01 Aug 2018 00:04:31 GMT

I have a Livy application that I'm trying to tune as I'm seeing some performance issue when
the queries are fast queries.  I've wrapped my queries with a timer that logs the time taken.
 The spark code executed typically takes 50ms to 150ms.  I'm querying Livy every 500ms looking
for my response, and generally it doesn't succeed until the third check.   It seems Livy itself
is spending up to an extra 1000ms.  Where is Livy spending this time?  Are there any tuning
parameters I can adjust?

Also, I am having difficulty changing any of the settings in livy-client.conf.  I placed the
file in /etc/hadoop/conf and livy/conf folder but my settings seem to get ignored.



View raw message