安装airflow :https://pythonhosted.org/airflow/installation.html
yum install libxml2-devel
yum install libxslt-devel
yum install libffi-devel
# airflow needs a home, ~/airflow is the default,
# but you can lay foundation somewhere else if you prefer
# (optional)
export AIRFLOW_HOME=~/airflow
# install from pypi using pip
pip2 install numpy
pip2 install cython
pip2 install pandas
pip2 install airflow
pip2 install airflow[hive]
pip2 install airflow[mysql]
pip2 install airflow[celery]
pip2 install airflow[rabbitmq]
pip2 install airflow[crypto]
pip2 install airflow[password]
pip2 install celery
pip2 install flower
# initialize the database
airflow initdb
如果需要mysql,请修改全局配置文件airflow.cfg
,并创建airflow库:
create database airflow default charset utf8;
在使用airflow-1.8.0版本时,如果有如下报错:
>>>> sqlalchemy.exc.ProgrammingError: (_mysql_exceptions.ProgrammingError)
>>>> (1064, "You have an error in your SQL syntax; check the manual that
>>>> corresponds to your MySQL server version for the right syntax to use
>> near
>>>> '(6) NULL' at line 1") [SQL: u'ALTER TABLE dag MODIFY last_scheduler_run
>>>> DATETIME(6) NULL']
需要变更
airflow/migrations/versions/4addfa1236f1_add_fractional_seconds_to_mysql_tables.py
将 mysql.DATETIME(fsp=6)
全部改为 mysql.DATETIME()
或将mysql升级到5.7或以上的版本。
如果使用了CeleryExecutor,需要启动
airflow worker -D
airflow scheduler -D
# start the web server, default port is 8080
airflow webserver -p 8080 -D
如果使用了celery
airflow flower
$ cd ~/airflow
$ ls airflow.cfg airflow.db airflow-webserver.pid unittests.cfg
$vim airflow.cfg
# Set to true to turn on authentication:
# http://pythonhosted.org/airflow/installation.html#web-authentication
#authenticate = False
authenticate = True
auth_backend = airflow.contrib.auth.backends.password_auth
import airflow
from airflow import models, settings
from airflow.contrib.auth.backends.password_auth import PasswordUser
user = PasswordUser(models.User())
user.username = 'your usrename'
user.email = 'username@email.com'
user.password = 'your password'
session = settings.Session()
session.add(user)
session.commit()
session.close()
exit()
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。