kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka git commit: KAFKA-3681; Connect doc formatting
Date Mon, 09 May 2016 16:01:25 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk bce3415b1 -> b86378840


KAFKA-3681; Connect doc formatting

Author: Kaufman Ng <kaufman@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1351 from coughman/KAFKA-3681-connect-doc-formatting


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/b8637884
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/b8637884
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/b8637884

Branch: refs/heads/trunk
Commit: b86378840a2a6defcbdc59c1c1462ecbfd2c47d4
Parents: bce3415
Author: Kaufman Ng <kaufman@confluent.io>
Authored: Mon May 9 17:01:16 2016 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Mon May 9 17:01:16 2016 +0100

----------------------------------------------------------------------
 docs/connect.html | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/b8637884/docs/connect.html
----------------------------------------------------------------------
diff --git a/docs/connect.html b/docs/connect.html
index 5cd4130..ca45165 100644
--- a/docs/connect.html
+++ b/docs/connect.html
@@ -192,7 +192,7 @@ public List&lt;Map&lt;String, String&gt;&gt; getTaskConfigs(int
maxTasks) {
 </pre>
 
 Although not used in the example, <code>SourceTask</code> also provides two APIs
to commit offsets in the source system: <code>commit</code> and <code>commitSourceRecord</code>.
The APIs are provided for source systems which have an acknowledgement mechanism for messages.
Overriding these methods allows the source connector to acknowledge messages in the source
system, either in bulk or individually, once they have been written to Kafka.
-The <code>commit<code> API stores the offsets in the source system, up to the
offsets that have been returned by <code>poll</code>. The implementation of this
API should block until the commit is complete. The <code>commitSourceRecord</code>
API saves the offset in the source system for each <code>SourceRecord</code> after
it is written to Kafka. As Kafka Connect will record offsets automatically, <code>SourceTask<code>s
are not required to implement them. In cases where a connector does need to acknowledge messages
in the source system, only one of the APIs is typically required.
+The <code>commit</code> API stores the offsets in the source system, up to the
offsets that have been returned by <code>poll</code>. The implementation of this
API should block until the commit is complete. The <code>commitSourceRecord</code>
API saves the offset in the source system for each <code>SourceRecord</code> after
it is written to Kafka. As Kafka Connect will record offsets automatically, <code>SourceTask</code>s
are not required to implement them. In cases where a connector does need to acknowledge messages
in the source system, only one of the APIs is typically required.
 
 Even with multiple tasks, this method implementation is usually pretty simple. It just has
to determine the number of input tasks, which may require contacting the remote service it
is pulling data from, and then divvy them up. Because some patterns for splitting work among
tasks are so common, some utilities are provided in <code>ConnectorUtils</code>
to simplify these cases.
 
@@ -232,7 +232,7 @@ Next, we implement the main functionality of the task, the <code>poll()</code>
m
 public List&lt;SourceRecord&gt; poll() throws InterruptedException {
     try {
         ArrayList&lt;SourceRecord&gt; records = new ArrayList&lt;&gt;();
-        while (streamValid(stream) && records.isEmpty()) {
+        while (streamValid(stream) &amp;&amp; records.isEmpty()) {
             LineAndOffset line = readToNextLine(stream);
             if (line != null) {
                 Map<String, Object> sourcePartition = Collections.singletonMap("filename",
filename);


Mime
View raw message