phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Hu <ruw...@gmail.com>
Subject Re: Create View of Existing HBase table
Date Sun, 18 Jun 2017 05:12:36 GMT
That's likely the problem. The value in HBase/Java is in millisecond (
System.currentTimeMillis()) as explained in the link:

https://currentmillis.com/tutorials/system-currentTimeMillis.html#unix-timestamp

If the packet time stamp is a long value in microsecond, then it's 1000
times of the value expected in HBase, which would be a future time stamp
for HBase.

Randy

On Sat, Jun 17, 2017 at 8:40 PM, M. Aaron Bossert [via Apache Phoenix User
List] <ml+s1124778n3687h46@n5.nabble.com> wrote:

> One potential difference might be resolution.  The network packets have
> precision down to microsecond...could that cause an issue?
>
> Sent from my iPhone
>
> > On Jun 17, 2017, at 20:32, Randy Hu <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=3687&i=0>> wrote:
> >
> > Yes, the problem I had experienced was the future time stamp in cells.
> > Setting "CurrentSCN" to maximal value would enforce the scan/get to use
> > this as the timestamp, so any cells would be retuned.
> >
> > Not sure if the network packet time stamp you copied is the same as Unix
> > timestamp:
> >
> > https://currentmillis.com/tutorials/system-currentTimeMillis.html#unix-
> timestamp
> >
> > For testing purpose, maybe you can insert some cells without explicit
> > timestamp to confirm whether timestamp is the issue.
> >
> > Randy
> >
> > On Sat, Jun 17, 2017 at 6:21 PM, M. Aaron Bossert [via Apache Phoenix
> User
> > List] <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=3687&i=1>> wrote:
> >
> >> I looked through the discussion and it seems like their issue was
> >> timestamps in the future?  I do use explicit timestamps for the data.
> I am
> >> storing network traffic and am using the packet's timestamp as the
> Hbase
> >> timestamp, which is often some time in the past when ingesting bulk
> >> collection files for forensic-type analysis.
> >>
> >> How would setting the SCN value help?  Would I set it in the past to
> match
> >> the oldest or newest packets?
> >>
> >> Sent from my iPhone
> >>
> >>> On Jun 17, 2017, at 18:01, Randy Hu <[hidden email]
> >> <http:///user/SendEmail.jtp?type=node&node=3684&i=0>> wrote:
> >>>
> >>> Not sure if you have custom time stamp set when the values were
> >> persisted.
> >>> Here is the time stamp issue I discovered with view. The solution
> (work
> >>> around) is in the last post:
> >>>
> >>> http://apache-phoenix-user-list.1124778.n5.nabble.com/
> >> View-timestamp-on-existing-table-potential-defect-td3475.html
> >>>
> >>> Randy
> >>>
> >>> On Fri, Jun 16, 2017 at 2:07 PM, M. Aaron Bossert [via Apache Phoenix
> >> User
> >>> List] <[hidden email]
> >> <http:///user/SendEmail.jtp?type=node&node=3684&i=1>> wrote:
> >>>
> >>>> *I have an existing HBase table with the following characteristics:*
> >>>>
> >>>> hbase(main):032:0> describe 'bro'
> >>>>
> >>>> Table bro is ENABLED
> >>>>
> >>>>
> >>>>
> >>>> bro, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.
> >>>> coprocessor.ScanRegionObserver|805306366|', coprocessor$2 =>
> >>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
>
> >>
> >>>> coprocessor
> >>>>
> >>>> $3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
>
> >>
> >>>> coprocessor$4 => '|org.apache.phoenix.coprocessor.
> >>>> ServerCachingEndpointImpl|805306366|', METADATA =>
> >> {'COMPACTION_ENABLED'
> >>>> => 'true'}
> >>>>
> >>>> }
> >>>>
> >>>>
> >>>>
> >>>> COLUMN FAMILIES DESCRIPTION
> >>>>
> >>>>
> >>>>
> >>>> {NAME => 'bm', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY
=>
> >>>> 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE',
> >> TTL
> >>>> => 'FOREVER', COMPRESSION => 'LZ4', MIN_VERSIONS => '0', BLOCKCACHE
> =>
> >> 'tr
> >>>>
> >>>> ue', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
> >>>>
> >>>>
> >>>>
> >>>> {NAME => 'uid', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY
=>
> >>>> 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE',
> >> TTL
> >>>> => 'FOREVER', COMPRESSION => 'LZ4', MIN_VERSIONS => '0', BLOCKCACHE
> =>
> >> 't
> >>>>
> >>>> rue', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
> >>>>
> >>>>
> >>>>
> >>>> 2 row(s) in 0.0300 seconds
> >>>>
> >>>>
> >>>> *When I try to create a Phoenix VIEW (Phoenix 4.7) as such:*
> >>>>
> >>>>
> >>>> 0: jdbc:phoenix:pocc2.net:2181> CREATE VIEW "bro" (pk VARCHAR
> PRIMARY
> >>>> KEY, "bm"."id.orig_h" VARCHAR, "bm"."id.orig_p" VARCHAR,
> >> "bm"."id_resp_h"
> >>>> VARCHAR, "bm"."id_resp_p" VARCHAR, "bm"."proto" VARCHAR) AS SELECT *
> >> FROM
> >>>> "bro";
> >>>>
> >>>> No rows affected (0.019 seconds)
> >>>>
> >>>>
> >>>> *I see no errors at all, but when I try to query, there are no
> >> results.  I
> >>>> verified that the columns in question are entered using the
> >>>> Bytes.toBytes(String) method in my code, thus my choice of VARCHAR.*
> >>>>
> >>>>
> >>>> 0: jdbc:phoenix:pocc2.net:2181> select * from "bro";
> >>>>
> >>>> *+-----+------------+------------+------------+------------+--------+*
>
> >>>>
> >>>> *| **PK ** | **id.orig_h ** | **id.orig_p ** | **id_resp_h ** | *
> >>>> *id_resp_p ** | **proto ** |*
> >>>>
> >>>> *+-----+------------+------------+------------+------------+--------+*
>
> >>>>
> >>>> *+-----+------------+------------+------------+------------+--------+*
>
> >>>>
> >>>> No rows selected (0.078 seconds)
> >>>>
> >>>>
> >>>> *Any assistance would be greatly appreciated...*
> >>>>
> >>>>
> >>>> ------------------------------
> >>>> If you reply to this email, your message will be added to the
> >> discussion
> >>>> below:
> >>>> http://apache-phoenix-user-list.1124778.n5.nabble.com/
> >>>> Create-View-of-Existing-HBase-table-tp3678.html
> >>>> To start a new topic under Apache Phoenix User List, email
> >>>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=3684&i=2>
> >>>> To unsubscribe from Apache Phoenix User List, click here
> >>>> <
> >>>> .
> >>>> NAML
> >>>> <http://apache-phoenix-user-list.1124778.n5.nabble.com/
> >> template/NamlServlet.jtp?macro=macro_viewer&id=instant_
> >> html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> >> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> >> nabble.view.web.template.NodeNamespace&breadcrumbs=
> >> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> >> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context: http://apache-phoenix-user-
> >> list.1124778.n5.nabble.com/Create-View-of-Existing-HBase-
> >> table-tp3678p3682.html
> >>> Sent from the Apache Phoenix User List mailing list archive at
> >> Nabble.com.
> >>
> >>
> >> ------------------------------
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >> http://apache-phoenix-user-list.1124778.n5.nabble.com/
> >> Create-View-of-Existing-HBase-table-tp3678p3684.html
> >> To start a new topic under Apache Phoenix User List, email
> >> [hidden email] <http:///user/SendEmail.jtp?type=node&node=3687&i=2>
> >> To unsubscribe from Apache Phoenix User List, click here
> >> <
> >> .
> >> NAML
> >> <http://apache-phoenix-user-list.1124778.n5.nabble.com/
> template/NamlServlet.jtp?macro=macro_viewer&id=instant_
> html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace&breadcrumbs=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >>
> >
> >
> >
> >
> > --
> > View this message in context: http://apache-phoenix-user-
> list.1124778.n5.nabble.com/Create-View-of-Existing-HBase-
> table-tp3678p3685.html
> > Sent from the Apache Phoenix User List mailing list archive at
> Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-phoenix-user-list.1124778.n5.nabble.com/
> Create-View-of-Existing-HBase-table-tp3678p3687.html
> To start a new topic under Apache Phoenix User List, email
> ml+s1124778n1h80@n5.nabble.com
> To unsubscribe from Apache Phoenix User List, click here
> <http://apache-phoenix-user-list.1124778.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=cnV3ZWloQGdtYWlsLmNvbXwxfC04OTI3ODY3NTc=>
> .
> NAML
> <http://apache-phoenix-user-list.1124778.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://apache-phoenix-user-list.1124778.n5.nabble.com/Create-View-of-Existing-HBase-table-tp3678p3688.html
Sent from the Apache Phoenix User List mailing list archive at Nabble.com.

Mime
View raw message