community-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <>
Subject [jira] [Updated] (COMDEV-183) Add HTrace distributed tracing integration to YARN
Date Mon, 14 Mar 2016 01:04:33 GMT


Colin Patrick McCabe updated COMDEV-183:
    Summary: Add HTrace distributed tracing integration to YARN  (was: Add distributed tracing
integration to YARN)

> Add HTrace distributed tracing integration to YARN
> --------------------------------------------------
>                 Key: COMDEV-183
>                 URL:
>             Project: Community Development
>          Issue Type: New Feature
>            Reporter: Colin Patrick McCabe
>              Labels: Gsoc2016, gsoc, mentor
> Distributed tracing allows users to follow a request through the entire distributed system,
crossing network and project boundaries.  The Apache HTrace project has added distributed
tracing to HDFS and HBase (among other projects).  We should add tracing to YARN so that MapReduce,
spark, and other frameworks can make use of it.
> Tracing should identify which yarn rpcs were made, and also tag top level trace spans
with the yarn job that they are associated with.  That would let us analyze information relevant
to a particular YARN job or set of jobs.  There are a lot of interesting projects here-- detecting
hardware failures in production clusters, analyzing patterns in MR or Spark jobs, etc.

This message was sent by Atlassian JIRA

View raw message