kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka git commit: KAFKA-3421; Follow up to fix name of SourceTask method and add documentation of connector status REST API
Date Mon, 09 May 2016 22:42:18 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk 22c098b71 -> 36ed00d9b


KAFKA-3421; Follow up to fix name of SourceTask method and add documentation of connector
status REST API

Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1355 from ewencp/kafka-3421-follow-up


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/36ed00d9
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/36ed00d9
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/36ed00d9

Branch: refs/heads/trunk
Commit: 36ed00d9bf82d48b0a02b6414734b7698c9ac4ea
Parents: 22c098b
Author: Ewen Cheslack-Postava <me@ewencp.org>
Authored: Mon May 9 23:42:09 2016 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Mon May 9 23:42:09 2016 +0100

----------------------------------------------------------------------
 docs/connect.html | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/36ed00d9/docs/connect.html
----------------------------------------------------------------------
diff --git a/docs/connect.html b/docs/connect.html
index ca45165..c3cf583 100644
--- a/docs/connect.html
+++ b/docs/connect.html
@@ -95,7 +95,9 @@ Since Kafka Connect is intended to be run as a service, it also provides
a REST
     <li><code>GET /connectors/{name}</code> - get information about a specific
connector</li>
     <li><code>GET /connectors/{name}/config</code> - get the configuration
parameters for a specific connector</li>
     <li><code>PUT /connectors/{name}/config</code> - update the configuration
parameters for a specific connector</li>
+    <li><code>GET /connectors/{name}/status</code> - get current status
of the connector, including if it is running, failed, paused, etc., which worker it is assigned
to, error information if it has failed, and the state of all its tasks</li>
     <li><code>GET /connectors/{name}/tasks</code> - get a list of tasks
currently running for a connector</li>
+    <li><code>GET /connectors/{name}/tasks/{taskid}/status</code> - get
current status of the task, including if it is running, failed, paused, etc., which worker
it is assigned to, and error information if it has failed</li>
     <li><code>DELETE /connectors/{name}</code> - delete a connector, halting
all tasks and deleting its configuration</li>
 </ul>
 
@@ -191,8 +193,8 @@ public List&lt;Map&lt;String, String&gt;&gt; getTaskConfigs(int
maxTasks) {
 }
 </pre>
 
-Although not used in the example, <code>SourceTask</code> also provides two APIs
to commit offsets in the source system: <code>commit</code> and <code>commitSourceRecord</code>.
The APIs are provided for source systems which have an acknowledgement mechanism for messages.
Overriding these methods allows the source connector to acknowledge messages in the source
system, either in bulk or individually, once they have been written to Kafka.
-The <code>commit</code> API stores the offsets in the source system, up to the
offsets that have been returned by <code>poll</code>. The implementation of this
API should block until the commit is complete. The <code>commitSourceRecord</code>
API saves the offset in the source system for each <code>SourceRecord</code> after
it is written to Kafka. As Kafka Connect will record offsets automatically, <code>SourceTask</code>s
are not required to implement them. In cases where a connector does need to acknowledge messages
in the source system, only one of the APIs is typically required.
+Although not used in the example, <code>SourceTask</code> also provides two APIs
to commit offsets in the source system: <code>commit</code> and <code>commitRecord</code>.
The APIs are provided for source systems which have an acknowledgement mechanism for messages.
Overriding these methods allows the source connector to acknowledge messages in the source
system, either in bulk or individually, once they have been written to Kafka.
+The <code>commit</code> API stores the offsets in the source system, up to the
offsets that have been returned by <code>poll</code>. The implementation of this
API should block until the commit is complete. The <code>commitRecord</code> API
saves the offset in the source system for each <code>SourceRecord</code> after
it is written to Kafka. As Kafka Connect will record offsets automatically, <code>SourceTask</code>s
are not required to implement them. In cases where a connector does need to acknowledge messages
in the source system, only one of the APIs is typically required.
 
 Even with multiple tasks, this method implementation is usually pretty simple. It just has
to determine the number of input tasks, which may require contacting the remote service it
is pulling data from, and then divvy them up. Because some patterns for splitting work among
tasks are so common, some utilities are provided in <code>ConnectorUtils</code>
to simplify these cases.
 


Mime
View raw message