标签：IMPALA 编译 3.4 BUILD impala3.4 apache HOME impala

安装

升级cmake

1 版本要求高于

下载地址

	https://cmake.org/files/v3.19/cmake-3.19.0-Linux-x86_64.tar.gz
	yum remove cmake
	
	export CMAKE_HOME=/app/cmake
	export PATH=$PATH:$CMAKE_HOME/bin

2 安装boost

	scl enable devtoolset-9 bash
	yum -y install boost

配置编译环境

1 部署配置maven

	tar -xvf apache-maven-3.5.4-bin.tar.gz
	mv apache-maven-3.5.4 maven
	
	vi maven/conf/settings.xml
	<localRepository>/app/compileenv/maven/repo</localRepository>

2 修改环境变量

	cd ~
	vi .bash_profile
	export M2_HOME=/app/compileenv/maven
	export M2=$M2_HOME/bin
	export PATH=$M2:$PATH
	export IMPALA_HOME=/app/impala

3 编译

	cd /usr/MyWorkSpace/CDH/apache-impala-3.4.0
	./buildall.sh -notests

4 拷贝编译后文件

#!/bin/bash
IMPALA_HOME=/app/impala
IMPALA_BUILD=/app/impala-build

# if exist the dest directory then clear it.
if [ -d "${IMPALA_BUILD}" ]; then
        rm -rf ${IMPALA_BUILD}/*
else
        mkdir -p ${IMPALA_BUILD}
fi

mkdir -p ${IMPALA_BUILD}/etc/default/
mkdir -p ${IMPALA_BUILD}/etc/hadoop/
mkdir -p ${IMPALA_BUILD}/be
mkdir -p ${IMPALA_BUILD}/lib64/
mkdir ${IMPALA_BUILD}/hadoop/
mkdir ${IMPALA_BUILD}/dependency/
mkdir ${IMPALA_BUILD}/shell

cp -rf ${IMPALA_HOME}/be/build/debug/* ${IMPALA_BUILD}/be/
cp -rf ${IMPALA_HOME}/toolchain/gcc-4.9.2/lib64/*   ${IMPALA_BUILD}/lib64/
cp -rf ${IMPALA_HOME}/fe/target/impala-frontend-0.1-SNAPSHOT.jar   ${IMPALA_BUILD}/dependency/
cp -rf ${IMPALA_HOME}/fe/target/dependency/*      ${IMPALA_BUILD}/dependency/
cp -rf ${IMPALA_HOME}/shell/build/impala-shell-3.4.0-RELEASE/*   ${IMPALA_BUILD}/shell
cp -r ${IMPALA_HOME}/www ${IMPALA_BUILD}/

echo "Finished"

bug处理

1 修改bootstrap_toolchain.py报错，由于这个文件里的包都比较大，下载容易失败建议将方法改为如下，自己手动下载相关包

	def needs_download(self):
		return False
	
	vi /usr/MyWorkSpace/CDH/apache-impala-3.4.0/bin/bootstrap_toolchain.py 
	加入
	import sys
	reload(sys)
	sys.setdefaultencoding('utf-8')

2 解决DistributionNotFound: No matching distribution found for thrift>=0.9.3 (from impyla==0.16.2->-r /usr/MyWorkSpace/CDH/apache-impala-3.4.0/bin/…/infra/python/deps/compiled-requirements.txt (line 2))

	wget https://bootstrap.pypa.io/2.7/get-pip.py
	python get-pip.py
	/usr/MyWorkSpace/CDH/apache-impala-3.4.0/infra/python/env/bin/pip install thrift==0.9.3

3 解决boost报错问题

下载boost版本boost-1.61.0

	cp boost_1_61_0.tar.gz /usr/MyWorkSpace/CDH/apache-impala-3.4.0/toolchain
	cd /usr/MyWorkSpace/CDH/apache-impala-3.4.0/toolchain
	tar -xvf boost_1_61_0.tar.gz
	mv boost_1_61_0.tar.gz boost-1.61.0-p2
	cd boost-1.61.0-p2
	./bootstrap.sh
	./b2

4 解决logredactor-2.0.7报错

github上下载https://github.com/cloudera/logredactor/tree/2.0.7
	mvn clean install

修改cloudera仓库地址

	Plugin net.sourceforge.czt.dev:cup-maven-plugin:1.6-cdh or one of its dependencies could not be resolved: Could not find artifact net.sourceforge.czt.dev:cup-maven-plugin:jar:1.6-cdh in cloudera.thirdparty.repo (https://repository.cloudera.com/content/repositories/third-party)

修改/usr/MyWorkSpace/CDH/apache-impala-3.4.0/impala-parent/pom.xml
	
	<repository>
  		<id>cloudera.thirdparty.repo</id>
  		<url>https://repository.cloudera.com/artifactory/public</url>
  		<name>Cloudera Third Party Repository</name>
  		<snapshots>
    		<enabled>false</enabled>
  		</snapshots>
	</repository>

	<pluginRepositories>
		<pluginRepository>
  			<id>cloudera.thirdparty.repo</id>
  			<url>https://repository.cloudera.com/artifactory/public</url>
  			<name>Cloudera Third Party Repository</name>
  			<snapshots>
    			<enabled>false</enabled>
  			</snapshots>
		</pluginRepository>
	</pluginRepositories>

5 解决ranger-plugins-common-2.0.0.7.0.2.0-212.pom

修改/usr/MyWorkSpace/CDH/apache-impala-3.4.0/fe/pom.xml

    <dependency>
      <groupId>org.apache.ranger</groupId>
      <artifactId>ranger-plugins-common</artifactId>
      <version>2.0.0</version>
      <exclusions>
        <exclusion>
          <groupId>com.sun.jersey</groupId>
          <artifactId>jersey-bundle</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

    <dependency>
      <groupId>org.apache.ranger</groupId>
      <artifactId>ranger-plugins-audit</artifactId>
      <version>2.0.0</version>
      <exclusions>
        <exclusion>
          <groupId>org.apache.kafka</groupId>
          <artifactId>kafka_2.11</artifactId>
        </exclusion>
      </exclusions>
	</dependency>

6 修改https://native-toolchain.s3.amazonaws.com/build/cdh_components/1814051/maven/org/apache/kudu/kudu-client/1.12.0-SNAPSHOT/kudu-client-1.12.0-SNAPSHOT.pom

修改/usr/MyWorkSpace/CDH/apache-impala-3.4.0/fe/pom.xml

    <dependency>
      <groupId>org.apache.kudu</groupId>
      <artifactId>kudu-client</artifactId>
      <version>1.12.0</version>
    </dependency>

标签：IMPALA,编译,3.4,BUILD,impala3.4,apache,HOME,impala
来源： https://blog.csdn.net/ciqingloveless/article/details/114531549

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。