在CentOS6.9安装Azkaban3.50.0_centos6.9 azkaban-程序员宅基地
Azkaban简介
上图是Azkaban官网提供的架构图，包括三个组件：
Relational Database (MySQL)
AzkabanWebServer
AzkabanExecutorServer
组件分工：
元数据库：用于维护任务流信息以及执行情况信息。
Web服务器：用户交互的门户，同时也是管理任务流、工程，控制调度，任务分配的核心组件。
执行服务器：任务被真正调起的服务器，可扩展。
AzkabanWebServer对数据库的使用：
项目管理
追踪任务流的状态
查看任务执行结果以及历史日志
调度任务
sla
AzkabanExecutorServer 对数据库的使用：
从数据库获取工程的信息
从数据库获取要执行的任务流
储存任务流和任务的日志信息
从数据库获取内置任务流的任务依赖关系
一、环境
操作系统：CentOS6.9
软件版本：Azkaban3.50.0
二、安装
我这里是将Azkaban安装在hadoop用户，所以所有的linux操作都在hadoop用户。
1、编译Azkaban可执行文件
1.1 下载安装包
cd /data/install_pkg
wget https://github.com/azkaban/azkaban/archive/3.50.0.tar.gz
解压安装包：
tar -zxvf 3.50.0.tar.gz
1.2 修改邮件代码
vi /data/install_pkg/azkaban-3.50.0/azkaban-common/src/main/java/azkaban/utils/EmailMessage.java
找到邮件参数配置的地方
:/props
增加以下语句：
props.put("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
1.3 安装git
因为使用gradle编译会用到git，所以要安装：
yum install git
1.4 安装JCE
如果不安装JCE，后面编译的时候会报错。安装方法见：
https://blog.csdn.net/andyguan01_2/article/details/88343641
1.5 编译文件
使用Gradle编译文件。Gradle是一个基于Apache Ant和Apache Maven的项目自动化构建工具。
cd /data/install_pkg/azkaban-3.50.0
./gradlew build installDist
编译报错，看着像是测试过程中出错，没找具体到是什么原因。
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution with configuration on demand is an incubating feature.
> Task :az-core:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-common:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-web-server:npmSetup
/data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/bin/npm -> /data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/bin/npm-cli.js
/data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/bin/npx -> /data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/bin/npx-cli.js
+ [email protected]
added 476 packages in 14.414s
> Task :azkaban-exec-server:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-common:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-hadoop-security-plugin:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /data/install_pkg/azkaban-3.50.0/azkaban-hadoop-security-plugin/src/main/java/azkaban/security/HadoopSecurityManager_H_2_0.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-common:test
azkaban.utils.UtilsTest > testValidCronExpressionV FAILED
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)EmailMessageCreatorTest
at org.junit.Assert.assertTrue(Assert.java:52)
at azkaban.utils.UtilsTest.testValidCronExpressionV(UtilsTest.java:63)
> Task :azkaban-web-server:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-exec-server:compileTestJava
Note: /data/install_pkg/azkaban-3.50.0/azkaban-exec-server/src/test/java/azkaban/execapp/AzkabanExecutorServerTest.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :azkaban-web-server:npm_install
added 39 packages in 3.23s
> Task :azkaban-common:test
328 tests completed, 1 failed, 18 skipped
> Task :azkaban-solo-server:compileJava
Note: /data/install_pkg/azkaban-3.50.0/azkaban-solo-server/src/main/java/azkaban/soloserver/AzkabanSingleServer.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :az-hdfs-viewer:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :az-reportal:compileJava
Note: /data/install_pkg/azkaban-3.50.0/az-reportal/src/main/java/azkaban/reportal/util/StreamProviderHDFS.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :az-hadoop-jobtype-plugin:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /data/install_pkg/azkaban-3.50.0/az-hadoop-jobtype-plugin/src/main/java/azkaban/jobtype/HadoopSecureSparkWrapper.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':azkaban-common:test'.
> There were failing tests. See the report at: file:///data/install_pkg/azkaban-3.50.0/azkaban-common/build/reports/tests/test/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2m 4s
71 actionable tasks: 66 executed, 5 from cache
于是改为使用命令./gradlew build installDist -x test，-x test表示跳过测试。
./gradlew clean
./gradlew build installDist -x test
1.6 复制可执行文件到安装目录
编译完成后，我们就可以获取到web，exec，db，solo的可执行文件tar包，拷贝到Azkaban安装目录：
ll /data/install_pkg/azkaban-3.50.0/azkaban-*/build/distributions/*.tar.gz
cp /data/install_pkg/azkaban-3.50.0/azkaban-*/build/distributions/*.tar.gz /u01/app/azkaban-3.50.0
ll /u01/app/azkaban-3.50.0
解压tar包到Azkaban安装目录：
cd /u01/app/azkaban-3.50.0
tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
tar -zxvf azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
2、初始化MySQL
初始化MySQL的库、用户、权限、表、数据：
mysql -u root -p
CREATE DATABASE azkaban;
CREATE USER 'azkaban'@'%' IDENTIFIED BY '填实际密码';
GRANT SELECT,INSERT,UPDATE,DELETE ON azkaban.* to 'azkaban'@'%' WITH GRANT OPTION;
flush privileges;
use azkaban;
source /u01/app/azkaban-3.50.0/azkaban-db-0.1.0-SNAPSHOT/create-all-sql-0.1.0-SNAPSHOT.sql;
3、配置SSL
keytool -keystore keystore -alias jetty -genkey -keyalg RSA
要输入密码的地方，都填相同密码。几个问题的回答，what … country code … ?填CN，Is … correct?填y，其他问题可以直接回车不填。。
完成上述工作后，将在当前目录生成 keystore 证书文件，将keystore移动到azkaban web服务器根目录中。
mv keystore /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT
4、配置web-server
因为web-server和exec-server都没有默认配置文件，我们需要复制solo的配置文件。
cd /u01/app/azkaban-3.50.0
cp -r azkaban-solo-server-0.1.0-SNAPSHOT/conf azkaban-web-server-0.1.0-SNAPSHOT
cp /data/install_pkg/azkaban-3.50.0/azkaban-web-server/src/test/resources/log4j.properties azkaban-web-server-0.1.0-SNAPSHOT/conf
4.1 配置azkaban.properties
vi /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/azkaban.properties
配置以下内容：
# Azkaban Personalization Settings
#服务器UI名称,用于服务器上方显示的名字
azkaban.name=Test
#描述
azkaban.label=My Local Azkaban
#UI颜色
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
#默认根web目录建议最好写绝对路径，以免出现找不到文件错误
web.resource.dir=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/web
#默认时区，已改为亚洲/上海，默认为美国
default.timezone.id=Asia/Shanghai
# Azkaban UserManager class
#用户权限管理默认类
user.manager.class=azkaban.user.XmlUserManager
#用户配置文件所在位置
user.manager.xml.file=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/azkaban-users.xml
# Loader for projects
# global配置文件所在位置
executor.global.properties=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/global.properties
azkaban.project.dir=projects
#数据库类型
database.type=mysql
#端口号
mysql.port=3306
#数据库连接IP
mysql.host=填实际IP
#数据库名
mysql.database=azkaban
#数据库用户名
mysql.user=azkaban
#数据库密码
mysql.password=填实际密码
#最大连接数
mysql.numconnections=100
# Azkaban Jetty server properties.
# Jetty服务器属性.
velocity.dev.mode=false
#最大线程数
jetty.maxThreads=25
#Jetty SSL端口
jetty.ssl.port=8443
#Jetty端口
jetty.port=8081
#SSL文件名
jetty.keystore=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/keystore
#SSL文件密码
jetty.password=填实际密码
#Jetty主密码 与 keystore文件相同
jetty.keypassword=填实际密码
#SSL文件名
jetty.truststore=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/keystore
# SSL文件密码
jetty.trustpassword=填实际密码
#jetty.use.ssl=false
jetty.maxThreads=25
# Azkaban Executor settings
executor.port=12321
# mail settings
mail.sender=
mail.host=
# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy
between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links.
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes
4.2 配置commonprivate.properties
先创建配置文件：
mkdir -p /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes
vi /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes/commonprivate.properties
输入以下内容：
azkaban.native.lib=false
execute.as.user=false
4.3 配置azkaban-users.xml
修改用户配置文件azkaban-users.xml：
vi /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/azkaban-users.xml
增加admin用户：（在第4行）
<azkaban-users>
<user groups="azkaban" password="填实际密码" roles="admin" username="azkaban"/>
<user password="填实际密码" roles="metrics" username="metrics"/>
<user username="admin" password="填实际密码" roles="admin,metrics"/>
<role name="admin" permissions="ADMIN"/>
<role name="metrics" permissions="METRICS"/>
</azkaban-users>
5、配置executor
将 azkaban-web-server-0.1.0-SNAPSHOT目录下的conf拷贝到 azkaban-exec-server-0.1.0-SNAPSHOT
cd /u01/app/azkaban-3.50.0
cp -r azkaban-web-server-0.1.0-SNAPSHOT/conf azkaban-exec-server-0.1.0-SNAPSHOT
cp -r azkaban-web-server-0.1.0-SNAPSHOT/plugins azkaban-exec-server-0.1.0-SNAPSHOT
6、启动服务
6.1 启动Executor Server
进入azkaban-exec-server-0.1.0-SNAPSHOT下启动（注意不要进入bin目录启动）
cd /u01/app/azkaban-3.50.0/azkaban-exec-server-0.1.0-SNAPSHOT
./bin/start-exec.sh
会在当前目录生成.out的日志文件：
ll /u01/app/azkaban-3.50.0/azkaban-exec-server-0.1.0-SNAPSHOT/*.out
more /u01/app/azkaban-3.50.0/azkaban-exec-server-0.1.0-SNAPSHOT/*.out
日志内容为：
Using Hadoop from /u01/app/hadoop
Using Hive from /u01/app/apache-hive-2.3.4-bin
./bin/internal/../..
.:/usr/java/jdk1.8.0_121/lib/dt.jar:/usr/java/jdk1.8.0_121/lib/tools.jar:/u01/app/hadoop-2.6.5/etc/hadoop:/u01/app/hadoop/share/hado
op/common/lib/*:/u01/app/hadoop/share/hadoop/common/*:/u01/app/hadoop/share/hadoop/hdfs:/u01/app/hadoop/share/hadoop/hdfs/lib/*:/u01
/app/hadoop/share/hadoop/hdfs/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/lib/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/*:/u01/app/had
oop/share/hadoop/mapreduce/lib/*:/u01/app/hadoop/share/hadoop/mapreduce/*:/u01/app/hadoop/contrib/capacity-scheduler/*.jar:./bin/int
ernal/../../lib/activation-1.1.jar:./bin/internal/../../lib/aopalliance-1.0.jar:./bin/internal/../../lib/az-core-0.1.0-SNAPSHOT.jar:
./bin/internal/../../lib/azkaban-common-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-db-0.1.0-SNAPSHOT.jar:./bin/internal/../
../lib/azkaban-exec-server-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT.jar:./bin/inter
nal/../../lib/azkaban-spi-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/c3p0-0.9.1.1.jar:./bin/internal/../../lib/commons-codec-1.9.ja
r:./bin/internal/../../lib/commons-collections-3.2.2.jar:./bin/internal/../../lib/commons-dbcp2-2.1.1.jar:./bin/internal/../../lib/c
ommons-dbutils-1.5.jar:./bin/internal/../../lib/commons-fileupload-1.2.1.jar:./bin/internal/../../lib/commons-io-2.4.jar:./bin/inter
nal/../../lib/commons-jexl-2.1.1.jar:./bin/internal/../../lib/commons-lang-2.6.jar:./bin/internal/../../lib/commons-logging-1.2.jar:
./bin/internal/../../lib/commons-math3-3.0.jar:./bin/internal/../../lib/commons-pool2-2.4.2.jar:./bin/internal/../../lib/gson-2.8.1.
jar:./bin/internal/../../lib/guava-21.0.jar:./bin/internal/../../lib/guice-4.1.0.jar:./bin/internal/../../lib/httpclient-4.5.3.jar:.
/bin/internal/../../lib/httpcore-4.4.6.jar:./bin/internal/../../lib/jackson-core-asl-1.9.5.jar:./bin/internal/../../lib/jackson-mapp
er-asl-1.9.5.jar:./bin/internal/../../lib/javax.inject-1.jar:./bin/internal/../../lib/jetty-6.1.26.jar:./bin/internal/../../lib/jett
y-util-6.1.26.jar:./bin/internal/../../lib/joda-time-2.0.jar:./bin/internal/../../lib/jopt-simple-4.3.jar:./bin/internal/../../lib/j
sr305-3.0.2.jar:./bin/internal/../../lib/kafka-clients-0.10.0.0.jar:./bin/internal/../../lib/kafka-log4j-appender-0.10.0.0.jar:./bin
/internal/../../lib/log4j-1.2.17.jar:./bin/internal/../../lib/lz4-1.3.0.jar:./bin/internal/../../lib/mail-1.4.5.jar:./bin/internal/.
./../lib/metrics-core-3.1.0.jar:./bin/internal/../../lib/metrics-jvm-3.1.0.jar:./bin/internal/../../lib/mysql-connector-java-5.1.28.
jar:./bin/internal/../../lib/quartz-2.2.1.jar:./bin/internal/../../lib/servlet-api-2.5-20081211.jar:./bin/internal/../../lib/slf4j-a
pi-1.7.21.jar:./bin/internal/../../lib/slf4j-log4j12-1.7.21.jar:./bin/internal/../../lib/snakeyaml-1.18.jar:./bin/internal/../../lib
/snappy-java-1.1.2.4.jar:./bin/internal/../../lib/velocity-1.7.jar:./bin/internal/../../extlib/*.jar:./bin/internal/../../plugins/*/
*.jar:/u01/app/hadoop/conf:/u01/app/hadoop/*:/u01/app/apache-hive-2.3.4-bin/conf:/u01/app/apache-hive-2.3.4-bin/lib/*
Starting AzkabanExecutorServer on port 12321 ...
2019/03/08 15:01:08.059 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/03/08 15:01:08.084 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./bin/internal/../../conf
2019/03/08 15:01:08.084 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/03/08 15:01:08.094 +0800 ERROR [StdOutErrRedirect] SLF4J: Class path contains multiple SLF4J bindings.
2019/03/08 15:01:08.094 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/hadoop-2.6.5/share/hadoop/common/
lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:01:08.095 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/azkaban-3.50.0/azkaban-exec-serve
r-0.1.0-SNAPSHOT/lib/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:01:08.095 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/apache-hive-2.3.4-bin/lib/log4j-s
lf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:01:08.095 +0800 ERROR [StdOutErrRedirect] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanat
ion.
2019/03/08 15:01:08.097 +0800 ERROR [StdOutErrRedirect] SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2019/03/08 15:01:08.607 +0800 WARN [StorageCleaner] Project cleanup disabled. All artifacts will be stored.
2019/03/08 15:01:08.608 +0800 INFO [AzkabanExecServerModule] Event reporter is not enabled
2019/03/08 15:01:08.608 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/03/08 15:01:08.609 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/03/08 15:01:08.611 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:01:08.611 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/03/08 15:01:08.612 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/03/08 15:01:08.613 +0800 INFO [JobTypeManager] Loading plugin default job types
2019/03/08 15:01:08.616 +0800 INFO [JobTypeManager] Job type plugin directory set. Loading extra job types from /u01/app/azkaban-3.5
0.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes
2019/03/08 15:01:08.616 +0800 INFO [JobTypeManager] Common plugin job props file /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SN
APSHOT/plugins/jobtypes/common.properties not found. Using only globals props
2019/03/08 15:01:08.616 +0800 INFO [JobTypeManager] Common plugin load props file /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-S
NAPSHOT/plugins/jobtypes/commonprivate.properties found. Attempt to load.
2019/03/08 15:01:08.626 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/03/08 15:01:08.628 +0800 INFO [ExecJettyServerModule] Setting up connector with stats on: true
2019/03/08 15:01:08.628 +0800 INFO [ExecJettyServerModule] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/03/08 15:01:08.628 +0800 INFO [ExecJettyServerModule] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/03/08 15:01:08.663 +0800 INFO [JmxJobMBeanManager] Initializing azkaban.execapp.jmx.JmxJobMBeanManager
2019/03/08 15:01:08.663 +0800 INFO [AzkabanExecutorServer] Job callback enabled? true
2019/03/08 15:01:08.665 +0800 INFO [JobCallbackManager] Initializing
2019/03/08 15:01:08.667 +0800 INFO [JobCallbackRequestMaker] responseWaitTimeoutMS: 5000
2019/03/08 15:01:08.668 +0800 INFO [JobCallbackRequestMaker] Global request configuration [expectContinueEnabled=false, proxy=null,
localAddress=null, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=
false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=3000, c
onnectTimeout=3000, socketTimeout=3000, contentCompressionEnabled=true]
2019/03/08 15:01:08.942 +0800 INFO [JobCallbackRequestMaker] Jobcall thread pool size: 10
2019/03/08 15:01:08.943 +0800 INFO [JobCallbackRequestMaker] Initialization for azkaban.execapp.event.JobCallbackRequestMaker is com
pleted
2019/03/08 15:01:08.944 +0800 INFO [JobCallbackManager] Initialization completed azkaban.execapp.event.JobCallbackManager
2019/03/08 15:01:08.944 +0800 INFO [JobCallbackManager] azkabanHostName oracle02.auditonline.prd.df.cn
2019/03/08 15:01:08.945 +0800 INFO [AzkabanExecutorServer] Registering MBeans...
2019/03/08 15:01:08.953 +0800 INFO [AzkabanExecutorServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/03/08 15:01:08.956 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2019/03/08 15:01:08.957 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2019/03/08 15:01:08.958 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2019/03/08 15:01:08.958 +0800 INFO [AzkabanExecutorServer] No value for property: jmx.attribute.processor.class was found
2019/03/08 15:01:08.958 +0800 INFO [log] jetty-6.1.26
2019/03/08 15:01:08.992 +0800 INFO [log] Started [email protected]:12321
2019/03/08 15:01:09.792 +0800 INFO [AzkabanExecutorServer] Started Executor Server on oracle02.auditonline.prd.df.cn:12321
2019/03/08 15:01:09.831 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/03/08 15:02:08.614 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:03:08.614 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:03:08.615 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/03/08 15:04:08.615 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:05:08.615 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:05:08.616 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/03/08 15:06:08.616 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:07:08.616 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:07:08.617 +0800 INFO [FlowRunnerManager] Cleaning recently finished
查看JPS，会出现AzkabanExecutorServer：
jps
6.2 启动Web Server
进入azkaban-web-server-0.1.0-SNAPSHOT下启动（注意不要进入bin目录启动）
cd /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT
./bin/start-web.sh
会在当前目录生成.out的日志文件：
ll /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/*.out
more /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/*.out
我这边启动后报错：
java.lang.NoSuchMethodError:com.google.common.collect.ImmutableMap.toImmutableMap
解决方法见：
https://blog.csdn.net/andyguan01_2/article/details/88353889
问题解决后，重新启动Web Server，日志内容为：
Using Hadoop from /u01/app/hadoop
Using Hive from /u01/app/apache-hive-2.3.4-bin
./bin/internal/../..
.:/usr/java/jdk1.8.0_121/lib/dt.jar:/usr/java/jdk1.8.0_121/lib/tools.jar:/u01/app/hadoop-2.6.5/etc/hadoop:/u01/app/hadoop/share/hado
op/common/lib/*:/u01/app/hadoop/share/hadoop/common/*:/u01/app/hadoop/share/hadoop/hdfs:/u01/app/hadoop/share/hadoop/hdfs/lib/*:/u01
/app/hadoop/share/hadoop/hdfs/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/lib/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/*:/u01/app/had
oop/share/hadoop/mapreduce/lib/*:/u01/app/hadoop/share/hadoop/mapreduce/*:/u01/app/hadoop/contrib/capacity-scheduler/*.jar:./bin/int
ernal/../../lib/activation-1.1.jar:./bin/internal/../../lib/antlr-2.7.2.jar:./bin/internal/../../lib/aopalliance-1.0.jar:./bin/inter
nal/../../lib/az-core-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/az-flow-trigger-dependency-plugin-0.1.0-SNAPSHOT.jar:./bin/interna
l/../../lib/azkaban-common-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-db-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaba
n-spi-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-web-server-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/c3p0-0.9.1.1.jar:./
bin/internal/../../lib/cglib-nodep-2.2.jar:./bin/internal/../../lib/commons-beanutils-1.7.0.jar:./bin/internal/../../lib/commons-cha
in-1.1.jar:./bin/internal/../../lib/commons-codec-1.9.jar:./bin/internal/../../lib/commons-collections-3.2.2.jar:./bin/internal/../.
./lib/commons-compress-1.2.jar:./bin/internal/../../lib/commons-dbcp2-2.1.1.jar:./bin/internal/../../lib/commons-dbutils-1.5.jar:./b
in/internal/../../lib/commons-digester-1.8.jar:./bin/internal/../../lib/commons-fileupload-1.2.1.jar:./bin/internal/../../lib/common
s-io-2.4.jar:./bin/internal/../../lib/commons-jexl-2.1.1.jar:./bin/internal/../../lib/commons-lang-2.6.jar:./bin/internal/../../lib/
commons-logging-1.2.jar:./bin/internal/../../lib/commons-math3-3.0.jar:./bin/internal/../../lib/commons-pool2-2.4.2.jar:./bin/intern
al/../../lib/commons-validator-1.3.1.jar:./bin/internal/../../lib/data-1.15.7.jar:./bin/internal/../../lib/data-transform-1.15.7.jar
:./bin/internal/../../lib/dom4j-1.1.jar:./bin/internal/../../lib/gson-2.8.1.jar:./bin/internal/../../lib/guava-21.0.jar:./bin/intern
al/../../lib/guice-4.1.0.jar:./bin/internal/../../lib/httpclient-4.5.3.jar:./bin/internal/../../lib/httpcore-4.4.6.jar:./bin/interna
l/../../lib/jackson-core-2.2.2.jar:./bin/internal/../../lib/jackson-core-asl-1.9.5.jar:./bin/internal/../../lib/jackson-mapper-asl-1
.9.5.jar:./bin/internal/../../lib/javax.inject-1.jar:./bin/internal/../../lib/javax.servlet-api-3.0.1.jar:./bin/internal/../../lib/j
etty-6.1.26.jar:./bin/internal/../../lib/jetty-util-6.1.26.jar:./bin/internal/../../lib/joda-time-2.0.jar:./bin/internal/../../lib/j
opt-simple-4.3.jar:./bin/internal/../../lib/json-20070829.jar:./bin/internal/../../lib/li-jersey-uri-1.15.7.jar:./bin/internal/../..
/lib/log4j-1.2.17.jar:./bin/internal/../../lib/mail-1.4.5.jar:./bin/internal/../../lib/metrics-core-3.1.0.jar:./bin/internal/../../l
ib/metrics-jvm-3.1.0.jar:./bin/internal/../../lib/mina-core-1.1.7.jar:./bin/internal/../../lib/mysql-connector-java-5.1.28.jar:./bin
/internal/../../lib/netty-3.2.3.Final.jar:./bin/internal/../../lib/oro-2.0.8.jar:./bin/internal/../../lib/parseq-1.3.6.jar:./bin/int
ernal/../../lib/pegasus-common-1.15.7.jar:./bin/internal/../../lib/quartz-2.2.1.jar:./bin/internal/../../lib/r2-1.15.7.jar:./bin/int
ernal/../../lib/restli-common-1.15.7.jar:./bin/internal/../../lib/restli-server-1.15.7.jar:./bin/internal/../../lib/servlet-api-2.5-
20081211.jar:./bin/internal/../../lib/slf4j-api-1.7.18.jar:./bin/internal/../../lib/slf4j-log4j12-1.7.18.jar:./bin/internal/../../li
b/snakeyaml-1.18.jar:./bin/internal/../../lib/snappy-0.3.jar:./bin/internal/../../lib/sslext-1.2-0.jar:./bin/internal/../../lib/stru
ts-core-1.3.8.jar:./bin/internal/../../lib/struts-taglib-1.3.8.jar:./bin/internal/../../lib/struts-tiles-1.3.8.jar:./bin/internal/..
/../lib/velocity-1.7.jar:./bin/internal/../../lib/velocity-tools-2.0.jar:./bin/internal/../../extlib/*.jar:./bin/internal/../../plug
ins/*/*.jar:/u01/app/hadoop/conf:/u01/app/hadoop/*:/u01/app/apache-hive-2.3.4-bin/conf:/u01/app/apache-hive-2.3.4-bin/lib/*
2019/03/08 15:02:55.688 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/03/08 15:02:55.708 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./bin/internal/../../conf
2019/03/08 15:02:55.709 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/03/08 15:02:55.718 +0800 ERROR [StdOutErrRedirect] SLF4J: Class path contains multiple SLF4J bindings.
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/hadoop-2.6.5/share/hadoop/common/
lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/azkaban-3.50.0/azkaban-web-server
-0.1.0-SNAPSHOT/lib/slf4j-log4j12-1.7.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/apache-hive-2.3.4-bin/lib/log4j-s
lf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanat
ion.
2019/03/08 15:02:55.722 +0800 ERROR [StdOutErrRedirect] SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2019/03/08 15:02:56.157 +0800 INFO [WebServerProvider] Excluded Cipher Suites: []
2019/03/08 15:02:56.161 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/03/08 15:02:56.161 +0800 INFO [WebServerProvider] Setting up connector with stats on: true
2019/03/08 15:02:56.163 +0800 INFO [WebServerProvider] Starting SSL server on port: 8443 # Max threads: 25
2019/03/08 15:02:56.679 +0800 INFO [ExecutorManager] Initializing local executor localhost:12321
2019/03/08 15:02:57.044 +0800 WARN [StorageCleaner] Project cleanup disabled. All artifacts will be stored.
2019/03/08 15:02:57.044 +0800 INFO [AzkabanProjectLoader] Using temp dir: /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/
temp
2019/03/08 15:02:57.044 +0800 INFO [AzkabanProjectLoader] Project version retention is set to 3
2019/03/08 15:02:57.046 +0800 WARN [XmlValidatorManager] Validator directory validators does not exist or is not a directory.
2019/03/08 15:02:57.046 +0800 WARN [XmlValidatorManager] Azkaban properties file does not contain the key project.validators.xml.fil
e
2019/03/08 15:02:57.103 +0800 INFO [TriggerManager] TriggerManager loaded.
2019/03/08 15:02:57.125 +0800 INFO [AzkabanWebServerModule] Loading user manager class azkaban.user.XmlUserManager
2019/03/08 15:02:57.233 +0800 INFO [XmlUserManager] Loading user azkaban
2019/03/08 15:02:57.233 +0800 INFO [XmlUserManager] Loading user metrics
2019/03/08 15:02:57.234 +0800 INFO [XmlUserManager] Loading user admin
2019/03/08 15:02:57.287 +0800 INFO [AzkabanWebServer] Loading built-in checker and action types
2019/03/08 15:02:57.290 +0800 INFO [CheckerTypeLoader] Registering checker BasicTimeChecker
2019/03/08 15:02:57.291 +0800 INFO [CheckerTypeLoader] Registering checker SlaChecker
2019/03/08 15:02:57.291 +0800 INFO [CheckerTypeLoader] Registering checker ExecutionChecker
2019/03/08 15:02:57.291 +0800 INFO [ActionTypeLoader] Registering action ExecuteFlowAction
2019/03/08 15:02:57.291 +0800 INFO [ActionTypeLoader] Registering action KillExecutionAction
2019/03/08 15:02:57.292 +0800 INFO [ActionTypeLoader] Registering action AlertAction
2019/03/08 15:02:57.292 +0800 INFO [ActionTypeLoader] Registering action CreateTriggerAction
2019/03/08 15:02:57.293 +0800 INFO [PluginCheckerAndActionsLoader] Loading plug-in checker and action types
2019/03/08 15:02:57.293 +0800 ERROR [PluginCheckerAndActionsLoader] plugin path plugins/triggers doesn't exist!
2019/03/08 15:02:57.314 +0800 INFO [AzkabanWebServer] Setting timezone to Asia/Shanghai
2019/03/08 15:02:57.314 +0800 INFO [AzkabanWebServer] Registering MBeans...
2019/03/08 15:02:57.325 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/03/08 15:02:57.328 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxTriggerManager registered.
2019/03/08 15:02:57.331 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxExecutorManager registered.
2019/03/08 15:02:57.349 +0800 INFO [AzkabanWebServer] Bean org.apache.log4j.jmx.HierarchyDynamicMBean registered.
2019/03/08 15:02:57.349 +0800 INFO [AzkabanWebServer] ************* loginLoggerObjName is null, make sure there is a logger with nam
e azkaban.webapp.servlet.LoginAbstractAzkabanServlet
2019/03/08 15:02:57.350 +0800 INFO [ExecutorManager] Cleaning old logs from execution_logs
2019/03/08 15:02:57.353 +0800 INFO [AzkabanWebServer] Setting up web resource dir /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-S
NAPSHOT/web
2019/03/08 15:02:57.383 +0800 INFO [ExecutorManager] Cleaning old log files before 2018-12-14T15:02:57.350+08:00
2019/03/08 15:02:57.385 +0800 INFO [ExecutorManager] Cleaned up 0 log entries.
2019/03/08 15:02:57.385 +0800 INFO [ExecutorManager] log clean up time: 0 seconds.
2019/03/08 15:02:57.435 +0800 INFO [JdbcTriggerImpl] Loading all triggers from db.
2019/03/08 15:02:57.437 +0800 INFO [JdbcTriggerImpl] Loaded 0 triggers.
2019/03/08 15:02:57.438 +0800 INFO [log] jetty-6.1.26
2019/03/08 15:02:58.736 +0800 INFO [ProjectManagerServlet] downloadBufferSize: 8192
2019/03/08 15:02:58.879 +0800 INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.UserManagerResource'
2019/03/08 15:02:58.881 +0800 INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.ProjectManagerResource'
2019/03/08 15:02:59.057 +0800 INFO [log] Started [email protected]:8443
2019/03/08 15:02:59.057 +0800 INFO [AzkabanWebServer] Server started
查看JPS，会出现AzkabanWebServer：
jps
6.3 登录管理界面
打开浏览器，输入https://10.200.4.117:8443，登录admin用户：
完毕。
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
智能推荐
1444-DYN04-01RA AB 控制器传感器_1444-dyn04-01ra手册-程序员宅基地
文章浏览阅读861次，点赞15次，收藏19次。总之，1444-DYN04-01RA AB 控制器传感器是一款功能强大、可靠稳定的传感器，适用于各种旋转和往复机械的监测和控制。它的应用可以提高机械的运行效率和安全性，为工业生产和能源领域的发展提供有力支持。此外，1444-DYN04-01RA AB 控制器传感器还具有友好的人机界面，可以方便地进行操作和维护。1444-DYN04-01RA AB 控制器传感器是一款用于监测旋转和往复机械的传感器，可以测量振动和压力等动态输入以及轴位移、偏心和活塞杆沉降等静态输入。_1444-dyn04-01ra手册
Verilog学习 | 基于vivado平台的DDS、FIR、FFT核的综合学习使用_vivado fir的输出当作fft的输入-程序员宅基地
文章浏览阅读5k次，点赞26次，收藏164次。一、自我介绍 第一篇文章，先做一个简短的自我介绍。目前笔者就读于西电本科三年级，攻读电子信息工程方向。之前一直苦于学过的、做过的东西总会随着时间大浪淘沙，因此想在CSDN这个平台上记录下这些东西，方便日后温故知新，同时也希望自己记录的东西能帮到同行者，与诸君共勉！二、学习任务及内容 学习FPGA一个月有余，最近在着手老师布置的最后一个学习任务：利用FPGA的DDS核产生两个不同频率的点频信号叠加后进行频谱分析（使用FFT核），然后设计FIR滤..._vivado fir的输出当作fft的输入
Handler、MessageQueue、Runnable与Looper的源码浅析_runnable run looper.praper-程序员宅基地
文章浏览阅读445次。Handler、MessageQueue、Runnable与Looper的个人总结工作原理Handler发送消息到MessageQueue中，Looper不断获取MessageQueue中的一个Message，然后分发给Handler去处理。这几个对象和线程的关系为：每个Thread 只对应一个Looper每个Looper 只对应一个MessageQueue每个MessageQueue包含了N_runnable run looper.praper
职高计算机专业烧钱吗,大学中最烧钱的3个专业，花钱就是无底洞，根本停不下来...-程序员宅基地
文章浏览阅读298次。失去梦想的人，犹如一颗寂寞的流星，不知流落何方。为了梦想，我甘愿放弃生活，像凡?高一样视艺术如生命。为了梦想，他吃尽了苦头，遭遇了无数的挫败。渐渐地，梦想的火花渐渐微弱。大学选专业也是令人头疼的一大问题，选不好专业势必会影响到未来的就业，虽然毕业后就业未必会从事相关专业的工作，但是对于很多普通人来说，选择相关专业领域就业会更有保障一些。毕竟，步入社会重新学习新的知识也是一个很漫长的过程。大学中最烧..._职高计算计要买吗
ubuntu16.04系统备份与恢复 ubuntu16.04 system backup and restore_ubuntu 16.04系统修复-程序员宅基地
文章浏览阅读1.6k次。说明：将一个工控机ubuntu16.04系统备分到其他工控机后者电脑上。新电脑磁盘容量需大于等于备份文件，一般都会满足。制作ubuntu16.04启动U盘U盘启动，启动原工控机进入BIOS，U盘启动系统，进入Try ubuntu。挂载原工控机硬盘，并打包原系统文件（linux系统全都是文件形式）(1) $sudo fdisk -l命令查看磁盘符号，一般较大容量的硬盘为系统盘（单个硬盘工控机）原工控机磁盘结构如下：主磁盘盘符是/dev/sda(2) $sudo passwd命令设置root_ubuntu 16.04系统修复
poj 1258 Agri-Net（最小生成树果题）_given a list of how much fiber it takes to connect-程序员宅基地
文章浏览阅读2.2k次。poj 1258 Agri-Net（最小生成树果题）_given a list of how much fiber it takes to connect each pair of farms, you m
随便推点
影视级调色lut预设包 Triune Color Cinematic LUTs V2-程序员宅基地
文章浏览阅读3.7k次。与大家分享一套影视级调色预设 Triune Color Cinematic LUTs V2。兼容ps,fcpx,pr,ae,lr等后期视频和图像处理软件，帮助大家轻松实现电影级调色，让作品获得更震撼、更高级的色彩表现。原文及下载地址：https://www.mac69.com/cj/1287.html影视级调色预设 Triune Color Cinematic LUTs V2 官方介绍..._triune color cinematic luts
Go 检测密码强度（密码安全性）_go密码复杂度验证-程序员宅基地
文章浏览阅读3.3k次，点赞2次，收藏5次。Go 检测密码强度。_go密码复杂度验证
埃斯顿工业机器人控制柜_埃斯顿机器人调试作业指导书.PDF-程序员宅基地
文章浏览阅读1.5k次。埃斯顿机器人调试作业指导书埃斯顿机器人调试作业指导书1、 目的为了保证成品机器人的质量，特制定本标准。2、适用范围该规范适用于所有产品机器人调试流程。3、引用标准GB/T 5226.1-2002 机械安全机械电气设各第1部分:通用技术条件4、调试步骤4.1首先核对电柜序列号、型号与与机器人本体是否相符。4.2确认机器人控制柜检验合格。参考《埃斯顿机器人成品电柜检验流程》。(注：电柜通过《埃斯顿机器..._埃斯顿机器人修改系统网址
rostcm6情感分析案例分析_ROSTCM6 情感分析软件 这款软件能分析文章中所表达的情绪内容 从而帮助使用者得出文章的情绪 - 下载 - 搜珍网...-程序员宅基地
文章浏览阅读1.2k次。文件名大小更新时间ROSTCM6.exedict/1.datdict/2.datdict/3.datdict/4.datdict/5.datdict/6.datdict/8.datdict/9.datdict/F.datdict/new.datdict/s.datdict/SSCItitle.txtdict/Z.datsample/模拟群(437343630).txtuser/Feature.tx..._rostcm6.exe
前后端参数传递总结（@RequestParam @RequestBody@PathVariable）_pathvariable前端传参-程序员宅基地
文章浏览阅读1.8w次，点赞46次，收藏371次。前后端参数传递总结（@RequestParam @RequestBody@PathVariable @RequestHeader ）@RequestParam@RequestBody合理的创建标题，有助于目录的生成如何改变文本的样式@RequestParam@RequestParam 常用来处理简单类型的绑定，通过Request.getParameter() 获取的String可直接转换为简单类型的情况（ String–> 简单类型的转换操作由ConversionService配置的转换器来完成_pathvariable前端传参
C语言 strstr函数的用法及模拟实现strstr函数-程序员宅基地
文章浏览阅读2.9w次，点赞29次，收藏113次。C语言 strstr函数的用法及模拟实现strstr函数一、strstr函数的用法二、模拟实现strstr函数的功能一、strstr函数的用法1.strstr函数原型：char* strstr(const char* str1,const char* str2)2.功能：strstr（）是一个参数为两个字符指针类型，返回值是char*类型的函数，它用于找到子串（str2）在一个字符串（str1）中第一次出现的位置。这里因为传进来的地址指向的内容不会在发生改变，所以我们在两个形参（char*）前加上c_strstr