incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Don Bosco Durai <>
Subject Re: [PROPOSAL] Apache Argus Proposal
Date Thu, 17 Jul 2014 20:03:26 GMT
Andrews, thanks for your feedback. My responses are inline.



On Jul 17, 2014, at 11:41 AM, Andrew Purtell <> wrote:

> Thank you for writing back with a detailed clarification.
> Regarding encryption at rest, HDFS is adding it as HDFS-6134, so likely
> there will be a new core feature option for the ecosystem to consider
> shortly.
> ​> I don’t feel one technology or one company or one small group or one
> approach can solve this problem. This has to be addressed by the community
> working together. This would also require a lot of support from each
> dependent projects and lot of co-ordination. And there would be multiple
> security solutions available for the end users to pick from.​
> ​
> Completely agreed. However, the desired community cooperation has both
> technical and political components. I think there are some concerns about
> how successful an outcome Argus may produce, informed by experience.
> Perhaps it would be worthwhile to address those concerns. Argus proposes to

The current Argus solution already has integration with the core Hadoop components like HDFS,
Hive and HBase. There are work in progress to support additional Hadoop components, which
includes Knox. Anytime, we cross project boundaries, there were would be always challenges
wrt technical and political. Working this out within the community makes more sense, rather
than doing this outside. Not attempting would be counterproductive.

> develop a common security infrastructure for the Hadoop ecosystem. In my
> opinion (and informed by personal experience) we have new incubating Hadoop
> ecosystem security projects like Sentry and Knox and proposals such as
> Argus because Hadoop core is locked down. Argus et. al. are like the
> proverbial blocked river (user demand for features) seeking a new route
> around a landslide (obvious poisonous contention and litigation-via-JIRA on
> every significant topic). I would be curious your thoughts on how to avoid
> the same end state in the Argus project. In my opinion, it would be a
> tragedy if a potential solution ends up perpetuating the dysfunction it
> seeks to bypass to a greater proportion of Foundation projects instead. A
> Hadoop ecosystem project attempting to remain independent from the
> dysfunction of Hadoop core would be well advised to stay away from adoption
> of Argus components (security is so critical) if the governance of Argus
I don’t believe Argus existence is because HDFS or any other component is locked down or
dysfunctional. Each component will continue to evolve  (core features and security) overtime
based on their priority, severity and timeline. The option to externalize security is always
a good thing. Option to externalize is a well accepted notion in the community. Commercial
databases allow externalizing security, web applications externalize authentication and authorization,
there are vulnerability management systems for file system/software version, etc. I don’t
see a security provider like Argus or Sentry extending the native security as a risk or bad
thing, instead, a good motivation for projects (particularly new) to focus on their core features.

I also don’t feel this is a short term user demand. Security requirement changes on regular
basis and as different type of industries adopt hadoop, their security requirements might
be also different. They will look for different options for security and select what addresses
their needs.

> perpetuates that dysfunction. By the way, it is also not too late for Knox
> and Sentry.
Security is most effective when it is deployed as layered solution. Knox addresses the perimeter
security. Currently it is REST and they will extend it to support security for more external
API technologies. Argus will not replace it, but complement Knox by centralizing the administration
and common auditing. Sentry and Argus have some overlap, but at the core, they have different
philosophies and approach. Similar discussion can be made between no-sql projects like Apache
HBase, Apache Accumulo and Apache Cassandra. Varying options is always healthy.

NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message