phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Conradie <willem.conra...@pbtgroup.co.za>
Subject Telco HBase POC
Date Fri, 15 Jan 2016 09:12:23 GMT

Hi,



I am currently consulting at a client with the following requirements.



They want to make available detailed data usage CDRs for customers to verify their data usage
against the websites that they visited. In short this can be seen as an itemised bill for
data usage.  The data is currently not loaded into a RDBMS due to the volumes of data involved.
The proposed solution is to load the data into HBase, running on a HDP cluster, and make it
available for querying by the subscribers.  It is critical to ensure low latency read access
to the subscriber data, which possibly will be exposed to 25 million subscribers. We will
be running a scaled down version first for a proof of concept with the intention of it becoming
an operational data store.  Once the solution is functioning properly for the data usage CDRs
other CDR types will be added, as such we need  to build a cost effective, scalable solution
.



I am thinking of using Apache Phoenix for the following reasons:



1.      1. Current data loading into RDBMS is file based (CSV) via a staging server using
the RDBMS file load drivers

2.      2.  Use Apache Phoenix   bin/psql.py script to mimic above process to load to HBase

3.       3. Expected data volume :  60 000 files per day
                                                  1 -to 10 MB per file
                                                  500 million records per day
                                                   500 GB total volume per day


4.        4. Use Apache Phoenix client for low latency data retrieval



Is Apache Phoenix a suitable candidate for this specific use case?



Regards,

Willem


Mime
View raw message