sqoop是SQL to Hadoop的简称，按照字面理解就是一个传统数据库与hadoop之间迁移数据的工具，配置起来跟掐面的大同小异，希望以后处理数据的时候树莓派的小身板能撑住w如果不行就再买一台协同作业好了（啥

 

0、运行环境

操作系统：Linux raspberrypi 4.1.7+

安装位置:/mnt/study

Hadoop版本：1.0.4

mySQL版本：5.5.46

JDK版本：1.7

JDK安装位置：/usr/lib/jvm/jdk-7-oracle-armhf

1、下载安装配置sqoop-1.4.6

sudo wget http://mirrors.cnnic.cn/apache/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-1.0.0.tar.gz

sudo tar -zxvf sqoop-1.4.6.bin__hadoop-1.0.0.tar.gz

sudo mv sqoop-1.4.6.bin__hadoop-1.0.0 sqoop

配置环境变量及sqoop配置文件

配置环境变量

sudo nano /etc/profile

增加一行

SQOOP_HOME=/mnt/study/sqoop

PATH后增加

:$SQOOP_HOME/bin

 

配置sqoop

cd sqoop/conf/

sudo cp sqoop-env-template.sh sqoop-env.sh

sudo nano sqoop-env.sh

去掉export注释，添加软件对应地址

#Set path to where bin/hadoop is available

export HADOOP_COMMON_HOME=/mnt/study/hadoop

 

#Set path to where hadoop-*-core.jar is available

export HADOOP_MAPRED_HOME=/mnt/study/hadoop

 

#set the path to where bin/hbase is available

export HBASE_HOME=/mnt/study/hbase

 

#Set the path to where bin/hive is available

export HIVE_HOME=/mnt/study/hive

即时生效

source /etc/profile

验证sqoop安装

sqoop help

2、连接MySQL与Listing Databases

下载MySQL Connector/J并放入sqoop的lib文件夹

sudo wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.38.tar.gz

tar -zxvf mysql-connector-java-5.1.38.tar.gz

sudo cp mysql-connector-java-5.1.38/mysql-connector-java-5.1.38-bin.jar /mnt/study/sqoop/lib/

验证与mySQL连接

sqoop list-databases --connect jdbc:mysql://localhost/ --username 我账号 --password 我密码

hadoop核心

sudo cp /mnt/study/hadoop/hadoop-core-1.0.4.jar /mnt/study/sqoop/lib

 

参考文献：

Sqoop Documentation (v1.4.6)

Sqoop安装与使用(sqoop-1.4.5 on hadoop 1.0.4)