kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ewe...@apache.org
Subject kafka git commit: KAFKA-4467: Run tests on travis-ci using docker
Date Fri, 10 Mar 2017 00:24:46 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk 65650ba4d -> dbcbd7920


KAFKA-4467: Run tests on travis-ci using docker

ijuma ewencp cmccabe harshach Please review.
Here is a sample run:
https://travis-ci.org/raghavgautam/kafka/builds/191714520

In this run 214 tests were run and 144 tests passed.

I will open separate jiras for fixing failures.

Author: Raghav Kumar Gautam <raghav@apache.org>

Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2376 from raghavgautam/trunk


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/dbcbd792
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/dbcbd792
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/dbcbd792

Branch: refs/heads/trunk
Commit: dbcbd7920f7056c80d55453ad543ea9ba87da9be
Parents: 65650ba
Author: Raghav Kumar Gautam <raghav@apache.org>
Authored: Thu Mar 9 16:24:38 2017 -0800
Committer: Ewen Cheslack-Postava <me@ewencp.org>
Committed: Thu Mar 9 16:24:38 2017 -0800

----------------------------------------------------------------------
 .travis.yml             |  54 ++++++++
 tests/README.md         | 303 +++++++++++++++++++++++++++++++++++++++++++
 tests/docker/Dockerfile |   2 +-
 3 files changed, 358 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/dbcbd792/.travis.yml
----------------------------------------------------------------------
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 0000000..9be5c58
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,54 @@
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+sudo: required
+dist: trusty
+language: java
+
+env:
+  - _DUCKTAPE_OPTIONS="--subset 0  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 1  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 2  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 3  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 4  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 5  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 6  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 7  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 8  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 9  --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 10 --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 11 --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 12 --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 13 --subsets 15"
+  - _DUCKTAPE_OPTIONS="--subset 14 --subsets 15"
+
+jdk:
+  - oraclejdk8
+
+before_install:
+  - gradle wrapper
+
+script:
+  - ./gradlew rat
+  - ./gradlew releaseTarGz && /bin/bash ./tests/docker/run_tests.sh
+
+services:
+  - docker
+
+before_cache:
+  - rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock
+  - rm -fr $HOME/.gradle/caches/*/plugin-resolution/
+cache:
+  directories:
+    - "$HOME/.m2/repository"
+    - "$HOME/.gradle/caches/"
+    - "$HOME/.gradle/wrapper/"

http://git-wip-us.apache.org/repos/asf/kafka/blob/dbcbd792/tests/README.md
----------------------------------------------------------------------
diff --git a/tests/README.md b/tests/README.md
index 87a6fed..ec6ab31 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -31,6 +31,309 @@ TC_PATHS="tests/kafkatest/tests/streams tests/kafkatest/tests/tools" bash
tests/
   - The docker containers are named knode01, knode02 etc.
    These nodes can't be used for any other purpose.
 
+Examining CI run
+----------------
+* Set BUILD_ID is travis ci's build id. E.g. build id is 169519874 for the following build
+```bash
+https://travis-ci.org/apache/kafka/builds/169519874
+```
+
+* Getting number of tests that were actually run
+```bash
+for id in $(curl -sSL https://api.travis-ci.org/builds/$BUILD_ID | jq '.matrix|map(.id)|.[]');
do curl -sSL "https://api.travis-ci.org/jobs/$id/log.txt?deansi=true" ; done | grep -cE 'RunnerClient:
Loading test'
+```
+
+* Getting number of tests that passed
+```bash
+for id in $(curl -sSL https://api.travis-ci.org/builds/$BUILD_ID | jq '.matrix|map(.id)|.[]');
do curl -sSL "https://api.travis-ci.org/jobs/$id/log.txt?deansi=true" ; done | grep -cE 'RunnerClient.*PASS'
+```
+* Getting all the logs produced from a run
+```bash
+for id in $(curl -sSL https://api.travis-ci.org/builds/$BUILD_ID | jq '.matrix|map(.id)|.[]');
do curl -sSL "https://api.travis-ci.org/jobs/$id/log.txt?deansi=true" ; done
+```
+* Explanation of curl calls to travis-ci & jq commands
+  - We get json information of the build using the following command
+```bash
+curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874
+```
+This produces a json about the build which looks like:
+```json
+{
+  "id": 169519874,
+  "repository_id": 6097916,
+  "number": "19",
+  "config": {
+    "sudo": "required",
+    "dist": "trusty",
+    "language": "java",
+    "env": [
+      "TC_PATHS=\"tests/kafkatest/tests/client\"",
+      "TC_PATHS=\"tests/kafkatest/tests/connect tests/kafkatest/tests/streams tests/kafkatest/tests/tools\"",
+      "TC_PATHS=\"tests/kafkatest/tests/mirror_maker\"",
+      "TC_PATHS=\"tests/kafkatest/tests/replication\"",
+      "TC_PATHS=\"tests/kafkatest/tests/upgrade\"",
+      "TC_PATHS=\"tests/kafkatest/tests/security\"",
+      "TC_PATHS=\"tests/kafkatest/tests/core\""
+    ],
+    "jdk": [
+      "oraclejdk8"
+    ],
+    "before_install": null,
+    "script": [
+      "./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
+    ],
+    "services": [
+      "docker"
+    ],
+    "before_cache": [
+      "rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock",
+      "rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
+    ],
+    "cache": {
+      "directories": [
+        "$HOME/.m2/repository",
+        "$HOME/.gradle/caches/",
+        "$HOME/.gradle/wrapper/"
+      ]
+    },
+    ".result": "configured",
+    "group": "stable"
+  },
+  "state": "finished",
+  "result": null,
+  "status": null,
+  "started_at": "2016-10-21T13:35:43Z",
+  "finished_at": "2016-10-21T14:46:03Z",
+  "duration": 16514,
+  "commit": "7e583d9ea08c70dbbe35a3adde72ed203a797f64",
+  "branch": "trunk",
+  "message": "respect _DUCK_OPTIONS",
+  "committed_at": "2016-10-21T00:12:36Z",
+  "author_name": "Raghav Kumar Gautam",
+  "author_email": "raghav@apache.org",
+  "committer_name": "Raghav Kumar Gautam",
+  "committer_email": "raghav@apache.org",
+  "compare_url": "https://github.com/raghavgautam/kafka/compare/cc788ac99ca7...7e583d9ea08c",
+  "event_type": "push",
+  "matrix": [
+    {
+      "id": 169519875,
+      "repository_id": 6097916,
+      "number": "19.1",
+      "config": {
+        "sudo": "required",
+        "dist": "trusty",
+        "language": "java",
+        "env": "TC_PATHS=\"tests/kafkatest/tests/client\"",
+        "jdk": "oraclejdk8",
+        "before_install": null,
+        "script": [
+          "./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
+        ],
+        "services": [
+          "docker"
+        ],
+        "before_cache": [
+          "rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock",
+          "rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
+        ],
+        "cache": {
+          "directories": [
+            "$HOME/.m2/repository",
+            "$HOME/.gradle/caches/",
+            "$HOME/.gradle/wrapper/"
+          ]
+        },
+        ".result": "configured",
+        "group": "stable",
+        "os": "linux"
+      },
+      "result": null,
+      "started_at": "2016-10-21T13:35:43Z",
+      "finished_at": "2016-10-21T14:24:50Z",
+      "allow_failure": false
+    },
+    {
+      "id": 169519876,
+      "repository_id": 6097916,
+      "number": "19.2",
+      "config": {
+        "sudo": "required",
+        "dist": "trusty",
+        "language": "java",
+        "env": "TC_PATHS=\"tests/kafkatest/tests/connect tests/kafkatest/tests/streams tests/kafkatest/tests/tools\"",
+        "jdk": "oraclejdk8",
+        "before_install": null,
+        "script": [
+          "./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
+        ],
+        "services": [
+          "docker"
+        ],
+        "before_cache": [
+          "rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock",
+          "rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
+        ],
+        "cache": {
+          "directories": [
+            "$HOME/.m2/repository",
+            "$HOME/.gradle/caches/",
+            "$HOME/.gradle/wrapper/"
+          ]
+        },
+        ".result": "configured",
+        "group": "stable",
+        "os": "linux"
+      },
+      "result": 1,
+      "started_at": "2016-10-21T13:35:46Z",
+      "finished_at": "2016-10-21T14:22:05Z",
+      "allow_failure": false
+    },
+
+    ...
+  ]
+}
+
+```
+  - By passing this through jq filter `.matrix` we extract the matrix part of the json
+```bash
+curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874 | jq '.matrix'
+```
+The resulting json looks like:
+```json
+[
+  {
+    "id": 169519875,
+    "repository_id": 6097916,
+    "number": "19.1",
+    "config": {
+      "sudo": "required",
+      "dist": "trusty",
+      "language": "java",
+      "env": "TC_PATHS=\"tests/kafkatest/tests/client\"",
+      "jdk": "oraclejdk8",
+      "before_install": null,
+      "script": [
+        "./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
+      ],
+      "services": [
+        "docker"
+      ],
+      "before_cache": [
+        "rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock",
+        "rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
+      ],
+      "cache": {
+        "directories": [
+          "$HOME/.m2/repository",
+          "$HOME/.gradle/caches/",
+          "$HOME/.gradle/wrapper/"
+        ]
+      },
+      ".result": "configured",
+      "group": "stable",
+      "os": "linux"
+    },
+    "result": null,
+    "started_at": "2016-10-21T13:35:43Z",
+    "finished_at": "2016-10-21T14:24:50Z",
+    "allow_failure": false
+  },
+  {
+    "id": 169519876,
+    "repository_id": 6097916,
+    "number": "19.2",
+    "config": {
+      "sudo": "required",
+      "dist": "trusty",
+      "language": "java",
+      "env": "TC_PATHS=\"tests/kafkatest/tests/connect tests/kafkatest/tests/streams tests/kafkatest/tests/tools\"",
+      "jdk": "oraclejdk8",
+      "before_install": null,
+      "script": [
+        "./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
+      ],
+      "services": [
+        "docker"
+      ],
+      "before_cache": [
+        "rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock",
+        "rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
+      ],
+      "cache": {
+        "directories": [
+          "$HOME/.m2/repository",
+          "$HOME/.gradle/caches/",
+          "$HOME/.gradle/wrapper/"
+        ]
+      },
+      ".result": "configured",
+      "group": "stable",
+      "os": "linux"
+    },
+    "result": 1,
+    "started_at": "2016-10-21T13:35:46Z",
+    "finished_at": "2016-10-21T14:22:05Z",
+    "allow_failure": false
+  },
+
+  ...
+]
+
+```
+  - By further passing this through jq filter `map(.id)` we extract the id of
+  the builds for each of the splits
+```bash
+curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874 | jq '.matrix|map(.id)'
+```
+The resulting json looks like:
+```json
+[
+  169519875,
+  169519876,
+  169519877,
+  169519878,
+  169519879,
+  169519880,
+  169519881
+]
+```
+  - To use these ids in for loop we want to get rid of `[]` which is done by
+  passing it through `.[]` filter
+```bash
+curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874 | jq '.matrix|map(.id)|.[]'
+```
+And we get
+```text
+169519875
+169519876
+169519877
+169519878
+169519879
+169519880
+169519881
+```
+  - In the for loop we have made calls to fetch logs
+```bash
+curl -sSL "https://api.travis-ci.org/jobs/169519875/log.txt?deansi=true" | tail
+```
+which gives us
+```text
+[INFO:2016-10-21 14:21:12,538]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling:
test 16 of 28
+[INFO:2016-10-21 14:21:12,538]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling:
setting up
+[INFO:2016-10-21 14:21:30,810]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling:
running
+[INFO:2016-10-21 14:24:35,519]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling:
PASS
+[INFO:2016-10-21 14:24:35,519]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling:
tearing down
+
+
+The job exceeded the maximum time limit for jobs, and has been terminated.
+
+```
+* Links
+  - [Travis-CI REST api documentation](https://docs.travis-ci.com/api)
+  - [jq Manual](https://stedolan.github.io/jq/manual/)
+
 Local Quickstart
 ----------------
 This quickstart will help you run the Kafka system tests on your local machine. Note this
requires bringing up a cluster of virtual machines on your local computer, which is memory
intensive; it currently requires around 10G RAM.

http://git-wip-us.apache.org/repos/asf/kafka/blob/dbcbd792/tests/docker/Dockerfile
----------------------------------------------------------------------
diff --git a/tests/docker/Dockerfile b/tests/docker/Dockerfile
index 78b2e32..b349f7e 100644
--- a/tests/docker/Dockerfile
+++ b/tests/docker/Dockerfile
@@ -21,7 +21,7 @@ ENV MIRROR="http://apache.cs.utah.edu/"
 
 ADD ssh /root/.ssh
 RUN chmod 600 /root/.ssh/id_rsa
-RUN apt update && apt install -y unzip wget curl jq coreutils openssh-server net-tools
vim openjdk-8-jdk python-pip python-dev libffi-dev libssl-dev
+RUN apt update && apt install -y unzip wget curl jq coreutils openssh-server net-tools
vim python-pip python-dev libffi-dev libssl-dev
 RUN pip install -U pip && pip install --upgrade cffi ducktape==0.6.0
 RUN mkdir -p "/opt/kafka-0.8.2.2" && curl -s "${MIRROR}kafka/0.8.2.2/kafka_2.10-0.8.2.2.tgz"
| tar xz --strip-components=1 -C "/opt/kafka-0.8.2.2"
 RUN mkdir -p "/opt/kafka-0.9.0.1" && curl -s "${MIRROR}kafka/0.9.0.1/kafka_2.10-0.9.0.1.tgz"
| tar xz --strip-components=1 -C "/opt/kafka-0.9.0.1"


Mime
View raw message