kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ewe...@apache.org
Subject kafka git commit: MINOR: Fixes to AWS init script for testing
Date Thu, 05 May 2016 02:25:09 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk ad3165097 -> 694303a3d


MINOR: Fixes to AWS init script for testing

A path was wrong in the script and in the documentation.

Author: Roger Hoover <roger.hoover@gmail.com>

Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #1309 from theduderog/fix_aws_init


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/694303a3
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/694303a3
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/694303a3

Branch: refs/heads/trunk
Commit: 694303a3da18e73cb08dfe229a07c7b31e0fa10c
Parents: ad31650
Author: Roger Hoover <roger.hoover@gmail.com>
Authored: Wed May 4 19:24:58 2016 -0700
Committer: Ewen Cheslack-Postava <me@ewencp.org>
Committed: Wed May 4 19:24:58 2016 -0700

----------------------------------------------------------------------
 tests/README.md         | 35 +++++++++++++++++------------------
 vagrant/aws/aws-init.sh |  2 +-
 2 files changed, 18 insertions(+), 19 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/694303a3/tests/README.md
----------------------------------------------------------------------
diff --git a/tests/README.md b/tests/README.md
index 143711d..1176b5f 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -1,15 +1,15 @@
 System Integration & Performance Testing
 ========================================
 
-This directory contains Kafka system integration and performance tests. 
+This directory contains Kafka system integration and performance tests.
 [ducktape](https://github.com/confluentinc/ducktape) is used to run the tests.  
-(ducktape is a distributed testing framework which provides test runner, 
-result reporter and utilities to pull up and tear down services.) 
+(ducktape is a distributed testing framework which provides test runner,
+result reporter and utilities to pull up and tear down services.)
 
 Local Quickstart
 ----------------
 This quickstart will help you run the Kafka system tests on your local machine. Note this
requires bringing up a cluster of virtual machines on your local computer, which is memory
intensive; it currently requires around 10G RAM.
-For a tutorial on how to setup and run the Kafka system tests, see 
+For a tutorial on how to setup and run the Kafka system tests, see
 https://cwiki.apache.org/confluence/display/KAFKA/tutorial+-+set+up+and+run+Kafka+system+tests+with+ducktape
 
 * Install Virtual Box from [https://www.virtualbox.org/](https://www.virtualbox.org/) (run
`$ vboxmanage --version` to check if it's installed).
@@ -30,11 +30,11 @@ https://cwiki.apache.org/confluence/display/KAFKA/tutorial+-+set+up+and+run+Kafk
         $ # When using Virtualbox, it also works to run: vagrant up
 
 * Build the desired branch of Kafka
-       
+
         $ git checkout $BRANCH
         $ gradle  # (only if necessary)
         $ ./gradlew systemTestLibs
-     
+
 * Run the system tests using ducktape:
 
         $ ducktape tests/kafkatest/tests
@@ -42,40 +42,40 @@ https://cwiki.apache.org/confluence/display/KAFKA/tutorial+-+set+up+and+run+Kafk
 EC2 Quickstart
 --------------
 This quickstart will help you run the Kafka system tests on EC2. In this setup, all logic
is run
-on EC2 and none on your local machine. 
+on EC2 and none on your local machine.
 
 There are a lot of steps here, but the basic goals are to create one distinguished EC2 instance
that
 will be our "test driver", and to set up the security groups and iam role so that the test
driver
 can create, destroy, and run ssh commands on any number of "workers".
 
-As a convention, we'll use "kafkatest" in most names, but you can use whatever name you want.

+As a convention, we'll use "kafkatest" in most names, but you can use whatever name you want.
 
 Preparation
 -----------
-In these steps, we will create an IAM role which has permission to create and destroy EC2
instances, 
+In these steps, we will create an IAM role which has permission to create and destroy EC2
instances,
 set up a keypair used for ssh access to the test driver and worker machines, and create a
security group to allow the test driver and workers to all communicate via TCP.
 
 * [Create an IAM role](http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_SettingUpUser.html#Using_CreateUser_console).
We'll give this role the ability to launch or kill additional EC2 machines.
  - Create role "kafkatest-master"
  - Role type: Amazon EC2
  - Attach policy: AmazonEC2FullAccess (this will allow our test-driver to create and destroy
EC2 instances)
- 
+
 * If you haven't already, [set up a keypair to use for SSH access](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html).
For the purpose
 of this quickstart, let's say the keypair name is kafkatest, and you've saved the private
key in kafktest.pem
 
-* Next, create a security group called "kafkatest". 
+* Next, create a security group called "kafkatest".
  - After creating the group, inbound rules: allow SSH on port 22 from anywhere; also, allow
access on all ports (0-65535) from other machines in the kafkatest group.
 
 Create the Test Driver
 ----------------------
-* Launch a new test driver machine 
+* Launch a new test driver machine
  - OS: Ubuntu server is recommended
  - Instance type: t2.medium is easily enough since this machine is just a driver
  - Instance details: Most defaults are fine.
  - IAM role -> kafkatest-master
- - Tagging the instance with a useful name is recommended. 
+ - Tagging the instance with a useful name is recommended.
  - Security group -> 'kafkatest'
-  
+
 
 * Once the machine is started, upload the SSH key to your test driver:
 
@@ -85,7 +85,7 @@ Create the Test Driver
 * Grab the public hostname/IP (available for example by navigating to your EC2 dashboard
and viewing running instances) of your test driver and SSH into it:
 
         $ ssh -i /path/to/kafkatest.pem ubuntu@public.hostname.amazonaws.com
-        
+
 Set Up the Test Driver
 ----------------------
 The following steps assume you have ssh'd into
@@ -99,11 +99,11 @@ the test driver machine.
 * Get Kafka:
 
         $ git clone https://git-wip-us.apache.org/repos/asf/kafka.git kafka
-        
+
 * Install some dependencies:
 
         $ cd kafka
-        $ kafka/vagrant/aws/aws-init.sh
+        $ ./vagrant/aws/aws-init.sh
         $ . ~/.bashrc
 
 * An example Vagrantfile.local has been created by aws-init.sh which looks something like:
@@ -133,4 +133,3 @@ the test driver machine.
         $ ducktape kafkatest/tests
 
 * To halt your workers without destroying persistent state, run `vagrant halt`. Run `vagrant
destroy -f` to destroy all traces of your workers.
-

http://git-wip-us.apache.org/repos/asf/kafka/blob/694303a3/vagrant/aws/aws-init.sh
----------------------------------------------------------------------
diff --git a/vagrant/aws/aws-init.sh b/vagrant/aws/aws-init.sh
index dac363f..822926f 100755
--- a/vagrant/aws/aws-init.sh
+++ b/vagrant/aws/aws-init.sh
@@ -46,7 +46,7 @@ done
 
 # Create Vagrantfile.local as a convenience
 if [ ! -e "$base_dir/Vagrantfile.local" ]; then
-    cp $base_dir/aws/aws-example-Vagrantfile.local $base_dir/Vagrantfile.local
+    cp $base_dir/vagrant/aws/aws-example-Vagrantfile.local $base_dir/Vagrantfile.local
 fi
 
 gradle="gradle-2.2.1"


Mime
View raw message