phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: Upsert is EXTREMELY slow
Date Thu, 12 Jul 2018 12:49:41 GMT
HBase must grab a lock for the row which is being updated. Normally, for 
a batch of updates sent to a region server, the RS will grab as many row 
locks as it can at once. If you only send one row to update at a time, 
you obviously get no amortization.

It's just the normal semantics of batching which you are completely 
missing out on. There are multiple manifestations of this. Row-locks are 
just one (network overhead, serialization, and rpc scheduling/execution 
are three others I can easily see)

On 7/11/18 4:10 PM, alchemist wrote:
> Josh Elser-2 wrote
>> Josh thanks so much for all your help.  I do not understand why you
>> "However, you are still fighting yourself when you have threads all trying
>> to grab the same lock to write their data."  My understanding is if we
>> have many processes running many threads, the data will be logged into WAL
>> as they come then flushed to the disk.  Where is the contention in this
>> case? Do we have one write per region/region server?
>>
>>
>>
>> On 7/11/18 11:33 AM, alchemist wrote:
>>> Thanks so much Josh!  I am unable to understand why performance is
>>> extremely
>>> slow.
>>>
>>> 1.  If I perform update using PreparedStatement addBatch and executeBatch
>>> then I get nearly 6000 transactions per minute.
>>>
>>> 2.  But in our case we need to save each transaction so cannot perform
>>> update batch,  so I am using PreparedStatement executeQuery and commit()
>>> getting nearly 100 transactions per minute.
>>>
>>> These numbers seems extremely slow,  therefore I am wondering I am doing
>>> something very incorrect.
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-phoenix-user-list.1124778.n5.nabble.com/
>>>
> 
> 
> 
> 
> 
> --
> Sent from: http://apache-phoenix-user-list.1124778.n5.nabble.com/
> 

Mime
View raw message