sqoop2教程 单节点安装

🧙 Questions

• sqoop2 安装说明 (linux)

☄️Ideas

1. 下载sqoop2文件

curl https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz -o sqoop-1.99.7.tar.gz

2. 解压sqoop2到执行文件

tar -vzxf sqoop-1.99.7.tar.gz -C /data/sqoop

3. 配置sqoop2环境变量

mkdir -p /data/sqoop/lib/sqoop2/
cd
vim .bashrc

• 修改内容

export HADOOP_HOME=/data/hadoop-3.3.0
export SQOOP_HOME=/data/sqoop/sqoop-1.99.7
export PATH=$PATH:$HADOOP_HOME/bin:$SQOOP_HOME/bin
export SQOOP_SERVER_EXTRA_LIB=/data/sqoop/lib/sqoop2/

source .bashrc

4. 下载mysql驱动到sqoop下

cp /data/apache-hive-3.1.2-bin/lib/mysql-connector-java-8.0.22.jar /data/sqoop/lib/sqoop2/
cp /data/apache-hive-3.1.2-bin/lib/hive-jdbc-3.1.2.jar /data/sqoop/lib/sqoop2/

5. 修改hadoop配置文件(当前用户ispong)

vim /data/hadoop-3.3.0/etc/hadoop/core-site.xml

• 修改内容

<configuration>

    <property>
        <name>hadoop.proxyuser.ispong.hosts</name>
        <value>*</value>
    </property>
    
    <property>
        <name>hadoop.proxyuser.ispong.groups</name>
        <value>*</value>
    </property>
</configuration>

6. 修改sqoop2配置文件

vim conf/sqoop.properties

• 修改内容

最好把日志的位置固定一下

# 修改sqoop2端口号
org.apache.sqoop.jetty.port=10125
# 修改hdfs地址
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/data/hadoop-3.3.0/etc/hadoop

7. 启动/关闭 sqoop2服务

• 启动sqoop2服务

sqoop2-server start

• 关闭sqoop2服务

sqoop2-server stop

🔗 Links

• sqoop website
• sqoop docs
• sqoop server install
• sqoop wiki
• sqoop csdn