kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ij...@apache.org
Subject kafka git commit: KAFKA-3681; Connect doc formatting
Date Mon, 09 May 2016 16:01:37 GMT
Repository: kafka
Updated Branches:
  refs/heads/0.10.0 9f583d96f -> 1142f51bf


KAFKA-3681; Connect doc formatting

Author: Kaufman Ng <kaufman@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1351 from coughman/KAFKA-3681-connect-doc-formatting

(cherry picked from commit b86378840a2a6defcbdc59c1c1462ecbfd2c47d4)
Signed-off-by: Ismael Juma <ismael@juma.me.uk>


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/1142f51b
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/1142f51b
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/1142f51b

Branch: refs/heads/0.10.0
Commit: 1142f51bf72095c8817db52c23d3a20d2099a5dd
Parents: 9f583d9
Author: Kaufman Ng <kaufman@confluent.io>
Authored: Mon May 9 17:01:16 2016 +0100
Committer: Ismael Juma <ismael@juma.me.uk>
Committed: Mon May 9 17:01:32 2016 +0100

----------------------------------------------------------------------
 docs/connect.html | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/1142f51b/docs/connect.html
----------------------------------------------------------------------
diff --git a/docs/connect.html b/docs/connect.html
index 5cd4130..ca45165 100644
--- a/docs/connect.html
+++ b/docs/connect.html
@@ -192,7 +192,7 @@ public List&lt;Map&lt;String, String&gt;&gt; getTaskConfigs(int
maxTasks) {
 </pre>
 
 Although not used in the example, <code>SourceTask</code> also provides two APIs
to commit offsets in the source system: <code>commit</code> and <code>commitSourceRecord</code>.
The APIs are provided for source systems which have an acknowledgement mechanism for messages.
Overriding these methods allows the source connector to acknowledge messages in the source
system, either in bulk or individually, once they have been written to Kafka.
-The <code>commit<code> API stores the offsets in the source system, up to the
offsets that have been returned by <code>poll</code>. The implementation of this
API should block until the commit is complete. The <code>commitSourceRecord</code>
API saves the offset in the source system for each <code>SourceRecord</code> after
it is written to Kafka. As Kafka Connect will record offsets automatically, <code>SourceTask<code>s
are not required to implement them. In cases where a connector does need to acknowledge messages
in the source system, only one of the APIs is typically required.
+The <code>commit</code> API stores the offsets in the source system, up to the
offsets that have been returned by <code>poll</code>. The implementation of this
API should block until the commit is complete. The <code>commitSourceRecord</code>
API saves the offset in the source system for each <code>SourceRecord</code> after
it is written to Kafka. As Kafka Connect will record offsets automatically, <code>SourceTask</code>s
are not required to implement them. In cases where a connector does need to acknowledge messages
in the source system, only one of the APIs is typically required.
 
 Even with multiple tasks, this method implementation is usually pretty simple. It just has
to determine the number of input tasks, which may require contacting the remote service it
is pulling data from, and then divvy them up. Because some patterns for splitting work among
tasks are so common, some utilities are provided in <code>ConnectorUtils</code>
to simplify these cases.
 
@@ -232,7 +232,7 @@ Next, we implement the main functionality of the task, the <code>poll()</code>
m
 public List&lt;SourceRecord&gt; poll() throws InterruptedException {
     try {
         ArrayList&lt;SourceRecord&gt; records = new ArrayList&lt;&gt;();
-        while (streamValid(stream) && records.isEmpty()) {
+        while (streamValid(stream) &amp;&amp; records.isEmpty()) {
             LineAndOffset line = readToNextLine(stream);
             if (line != null) {
                 Map<String, Object> sourcePartition = Collections.singletonMap("filename",
filename);


Mime
View raw message