sqoop-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hshreedha...@apache.org
Subject git commit: SQOOP-1262. Sqoop2: Tool: Create user guide entry for the Tooling infrastructure
Date Thu, 02 Jan 2014 17:26:30 GMT
Updated Branches:
  refs/heads/sqoop2 ad93e2a5e -> 4b93a80cf


SQOOP-1262. Sqoop2: Tool: Create user guide entry for the Tooling infrastructure

(Jarek Jarcec Cecho via Hari Shreedharan)


Project: http://git-wip-us.apache.org/repos/asf/sqoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/sqoop/commit/4b93a80c
Tree: http://git-wip-us.apache.org/repos/asf/sqoop/tree/4b93a80c
Diff: http://git-wip-us.apache.org/repos/asf/sqoop/diff/4b93a80c

Branch: refs/heads/sqoop2
Commit: 4b93a80cf009eae022699e57f8342a3e3bf23214
Parents: ad93e2a
Author: Hari Shreedharan <hshreedharan@apache.org>
Authored: Thu Jan 2 09:25:52 2014 -0800
Committer: Hari Shreedharan <hshreedharan@apache.org>
Committed: Thu Jan 2 09:25:52 2014 -0800

----------------------------------------------------------------------
 docs/src/site/sphinx/Installation.rst | 15 +++++-
 docs/src/site/sphinx/Tools.rst        | 73 ++++++++++++++++++++++++++++++
 docs/src/site/sphinx/Upgrade.rst      | 26 ++++++++++-
 docs/src/site/sphinx/index.rst        | 23 ++++++----
 4 files changed, 125 insertions(+), 12 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/sqoop/blob/4b93a80c/docs/src/site/sphinx/Installation.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/Installation.rst b/docs/src/site/sphinx/Installation.rst
index c25d394..97af412 100644
--- a/docs/src/site/sphinx/Installation.rst
+++ b/docs/src/site/sphinx/Installation.rst
@@ -14,9 +14,9 @@
    limitations under the License.
 
 
-=======================================
+============
 Installation
-=======================================
+============
 
 Sqoop ships as one binary package however it's compound from two separate parts - client
and server. You need to install server on single node in your cluster. This node will then
serve as an entry point for all connecting Sqoop clients. Server acts as a mapreduce client
and therefore Hadoop must be installed and configured on machine hosting Sqoop server. Clients
can be installed on any arbitrary number of machines. Client is not acting as a mapreduce
client and thus you do not need to install Hadoop on nodes that will act only as a Sqoop client.
 
@@ -58,6 +58,17 @@ File ``sqoop_bootstrap.properties`` specifies which configuration provider
shoul
 
 Second configuration file ``sqoop.properties`` contains remaining configuration properties
that can affect Sqoop server. File is very well documented, so check if all configuration
properties fits your environment. Default or very little tweaking should be sufficient most
common cases.
 
+You can verify the Sqoop server configuration using `Verify Tool <Tools.html#verify>`__,
for example::
+
+  ./bin/sqoop.sh tool verify
+
+Upon running the ``verify`` tool, you should see messages similar to the following::
+
+  Verification was successful.
+  Tool class org.apache.sqoop.tools.tool.VerifyTool has finished correctly
+
+Consult `Verify Tool <Tools.html#upgrade>`__ documentation page in case of any failure.
+
 Server Life Cycle
 -----------------
 

http://git-wip-us.apache.org/repos/asf/sqoop/blob/4b93a80c/docs/src/site/sphinx/Tools.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/Tools.rst b/docs/src/site/sphinx/Tools.rst
new file mode 100644
index 0000000..1c7481f
--- /dev/null
+++ b/docs/src/site/sphinx/Tools.rst
@@ -0,0 +1,73 @@
+.. Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF lANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+
+=====
+Tools
+=====
+
+Tools are server commands that administrators can execute on the Sqoop server machine in
order to perform various maintenance tasks. The tool execution will always perform a given
task and finish. There are no long running services implemented as tools.
+
+In order to perform the maintenance task each tool is suppose to do, they need to be executed
in exactly the same environment as the main Sqoop server. The tool binary will take care of
setting up the ``CLASSPATH`` and other environmental variables that might be required. However
it's up to the administrator himself to run the tool under the same user as is used for the
server. This is usually configured automatically for various Hadoop distributions (such as
Apache Bigtop).
+
+
+.. note:: Running tools under a different user such as ``root`` might prevent Sqoop Server
from running correctly.
+
+List of available tools:
+
+* verify
+* upgrade
+
+To run the desired tool, execute binary ``sqoop.sh`` with the parameter ``tool`` followed
by the desired tool name. For example to run ``verify`` tool::
+
+  bin/sqoop.sh tool verify
+
+.. note:: Running tools while the Sqoop Server is also running is not recommended as it might
lead to a data corruption and service disruption.
+
+Verify
+======
+
+The verify tool will verify Sqoop server configuration by starting all subsystems with the
exception of servlets and tearing them down.
+
+To run the ``verify`` tool::
+
+  bin/sqoop.sh tool verify
+
+If the verification process succeeds, you should see messages like::
+
+  Verification was successful.
+  Tool class org.apache.sqoop.tools.tool.VerifyTool has finished correctly
+
+If the verification process will find any inconsistencies, it will print out the following
message instead::
+
+  Verification has failed, please check Server logs for further details.
+  Tool class org.apache.sqoop.tools.tool.VerifyTool has failed.
+
+Further details why the verification has failed will be available in the Sqoop server log
- same file as the Sqoop Server logs into.
+
+Upgrade
+=======
+
+Upgrades all versionable components inside Sqoop2. This includes structural changes inside
the repository and stored metadata. Running this tool is idempotent.
+
+Upon successful upgrade you should see following message::
+
+  Tool class org.apache.sqoop.tools.tool.UpgradeTool has finished correctly.
+
+Execution failure will show the following message instead::
+
+  Tool class org.apache.sqoop.tools.tool.UpgradeTool has failed.
+
+Further details why the upgrade process has failed will be available in the Sqoop server
log - same file as the Sqoop Server logs into.

http://git-wip-us.apache.org/repos/asf/sqoop/blob/4b93a80c/docs/src/site/sphinx/Upgrade.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/Upgrade.rst b/docs/src/site/sphinx/Upgrade.rst
index 49a76c2..45d27bd 100644
--- a/docs/src/site/sphinx/Upgrade.rst
+++ b/docs/src/site/sphinx/Upgrade.rst
@@ -25,11 +25,33 @@ This page describes procedure that you need to take in order to upgrade
Sqoop fr
 Upgrading Server
 ================
 
-As Sqoop server is using external repository for persisting connection and job objects, this
repository needs to be updated in addition to updating the server bits. The capability of
performing the upgrade has been built-in to the server, however is disabled by default to
avoid any unintentional changes to the repository. You can start the upgrade procedure by
stopping the server: ::
+As Sqoop server is using external repository for persisting connection and job objects, this
repository needs to be updated in addition to updating the server bits. There are two ways
how to upgrade Sqoop server metadata and it's structures, you can either execute upgrade tool
or configure Server to perform all necessary changes on start up.
+
+It's strongly advised to back up the repository before moving on to next steps. Backup instructions
will vary depending on the repository implementation. For example, using MySQL as a repository
will require a different back procedure than Apache Derby. Please follow the repositories'
backup procedure
+
+Upgrading Server using upgrade tool
+-----------------------------------
+
+Preferred upgrade path is to explicitly run the `Upgrade Tool <Tools.html#upgrade>`_.
First step is to however shutdown the server as having both the server and upgrade utility
accessing the same repository might corrupt it::
 
   ./bin/sqoop.sh server stop
 
-It's strongly advised to back up the repository before moving on to next steps. Precise back-up
instructions depends on the repository implementation that is in use, please follow the repository
owns instructions to back it up.
+When the server has been successfully stopped, you can update the server bits and simply
run the upgrade tool::
+
+  ./bin/sqoop.sh tool upgrade
+
+You should see that the upgrade process has been successful::
+
+  Tool class org.apache.sqoop.tools.tool.UpgradeTool has finished correctly.
+
+In case of any failure, please take a look into `Upgrade Tool <Tools.html#upgrade>`_
documentation page.
+
+Upgrading Server on start-up
+----------------------------
+
+The capability of performing the upgrade has been built-in to the server, however is disabled
by default to avoid any unintentional changes to the repository. You can start the upgrade
procedure by stopping the server: ::
+
+  ./bin/sqoop.sh server stop
 
 Now you can update server bits. Before starting the server again you will need to enable
the auto-upgrade feature that will perform all necessary during Sqoop Server boot up procedure.
You need to set following properties in configuration file ``sqoop.properties``::
 

http://git-wip-us.apache.org/repos/asf/sqoop/blob/4b93a80c/docs/src/site/sphinx/index.rst
----------------------------------------------------------------------
diff --git a/docs/src/site/sphinx/index.rst b/docs/src/site/sphinx/index.rst
index f5d7d92..e9bfd51 100644
--- a/docs/src/site/sphinx/index.rst
+++ b/docs/src/site/sphinx/index.rst
@@ -40,19 +40,16 @@ License
     See the License for the specific language governing permissions and
     limitations under the License.
 
-Overview
---------
+Administrator Documentation
+---------------------------
 
-- `Download Apache Sqoop <http://www.apache.org/dyn/closer.cgi/sqoop>`_
-- `Sqoop Wiki <https://cwiki.apache.org/confluence/display/SQOOP/Home>`_
-- `Sqoop Issue Tracking (JIRA) <https://issues.apache.org/jira/browse/SQOOP>`_
-- `Sqoop Source Code <https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=summary>`_
+- `Installation <Installation.html>`_
+- `Upgrade <Upgrade.html>`_
+- `Tools <Tools.html>`_
 
 User Documentation
 ------------------
 
-- `Installation <Installation.html>`_
-- `Upgrade <Upgrade.html>`_
 - `5 Minutes Demo <Sqoop5MinutesDemo.html>`_
 - `Command Line Client <CommandLineClient.html>`_
 
@@ -64,3 +61,13 @@ Developer Guide
 - `Java Client API Guide <ClientAPI.html>`_
 - `Developping Connector <ConnectorDevelopment.html>`_
 - `REST API Guide <RESTAPI.html>`_
+
+Overview
+--------
+
+- `Download Apache Sqoop <http://www.apache.org/dyn/closer.cgi/sqoop>`_
+- `Sqoop Wiki <https://cwiki.apache.org/confluence/display/SQOOP/Home>`_
+- `Sqoop Issue Tracking (JIRA) <https://issues.apache.org/jira/browse/SQOOP>`_
+- `Sqoop Source Code <https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=summary>`_
+
+


Mime
View raw message