community-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (COMDEV-183) Add distributed tracing integration to YARN
Date Mon, 14 Mar 2016 01:03:33 GMT

     [ https://issues.apache.org/jira/browse/COMDEV-183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Colin Patrick McCabe updated COMDEV-183:
----------------------------------------
    Description: 
Distributed tracing allows users to follow a request through the entire distributed system,
crossing network and project boundaries.  The Apache HTrace project has added distributed
tracing to HDFS and HBase (among other projects).  We should add tracing to YARN so that MapReduce,
spark, and other frameworks can make use of it.

Tracing should identify which yarn rpcs were made, and also tag top level trace spans with
the yarn job that they are associated with.  That would let us analyze information relevant
to a particular YARN job or set of jobs, and get some interesting statistics.  It would also
enable optimizing YARN itself.

  was:
Distributed tracing allows users to follow a request through the entire distributed system,
crossing network and project boundaries.  The Apache HTrace project has added distributed
tracing to HDFS and HBase (among other projects).  We should add tracing to YARN so that MapReduce,
spark, and other frameworks can make use of it.

Tracing should identify which yarn rpcs were made, and also tag top level trace spans with
the yarn job that they are associated with.


> Add distributed tracing integration to YARN
> -------------------------------------------
>
>                 Key: COMDEV-183
>                 URL: https://issues.apache.org/jira/browse/COMDEV-183
>             Project: Community Development
>          Issue Type: New Feature
>            Reporter: Colin Patrick McCabe
>              Labels: Gsoc2016, gsoc, mentor
>
> Distributed tracing allows users to follow a request through the entire distributed system,
crossing network and project boundaries.  The Apache HTrace project has added distributed
tracing to HDFS and HBase (among other projects).  We should add tracing to YARN so that MapReduce,
spark, and other frameworks can make use of it.
> Tracing should identify which yarn rpcs were made, and also tag top level trace spans
with the yarn job that they are associated with.  That would let us analyze information relevant
to a particular YARN job or set of jobs, and get some interesting statistics.  It would also
enable optimizing YARN itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message