Kudu遇到的问题
安装Kudu
kudu已经集成在了CDP runtime中，安装比较简单，选择master和tablet之后，配置数据目录即可
安装完成后，我们要在impala中显式集成kudu
为了省去每次建表都需要在TBLPROPERTIES中添加kudumasteraddresses属性，我们还要在Impala的高级配置KuduMaster的地址 --kudu_master_hosts=192.168.0.207:7051
在impala-shell中建kudu表
[root@cdh2 ~]# impala-shell
Starting Impala Shell without Kerberos authentication
Opened TCP connection to cdh2.macro.com:21000
Connected to cdh2.macro.com:21000
Server version: impalad version 3.4.0-SNAPSHOT RELEASE (build 25402784335c39cc24076d71dab7a3ccbd562094)
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v3.4.0-SNAPSHOT (2540278) built on Wed Aug 5 11:07:32 UTC 2020)
You can change the Impala daemon that you're connected to by using the CONNECT
command.To see how Impala will plan to run your query without actually executing
it, use the EXPLAIN command. You can change the level of detail in the EXPLAIN
output by setting the EXPLAIN_LEVEL query option.
***********************************************************************************
[cdh2.macro.com:21000] default> CREATE TABLE my_first_table
> (
> id BIGINT,
> name STRING,
> PRIMARY KEY(id)
> )
> PARTITION BY HASH PARTITIONS 16
> STORED AS KUDU
> TBLPROPERTIES (
> 'kudu.master_addresses' = 'cdh2.macro.com:7051'
> );
Query: CREATE TABLE my_first_table
(
id BIGINT,
name STRING,
PRIMARY KEY(id)
)
PARTITION BY HASH PARTITIONS 16
STORED AS KUDU
TBLPROPERTIES (
'kudu.master_addresses' = 'cdh2.macro.com:7051'
)
+-------------------------+
| summary |
+-------------------------+
| Table has been created. |
+-------------------------+
Fetched 1 row(s) in 2.35s
[cdh2.macro.com:21000] default> desc formatted my_first_table;
Query: describe formatted my_first_table
+------------------------------+------------------------------------------------------------------------------+------------------------------------------------+
| name | type | comment |
+------------------------------+------------------------------------------------------------------------------+------------------------------------------------+
| # col_name | data_type | comment |
| | NULL | NULL |
| id | bigint | NULL |
| name | string | NULL |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | default | NULL |
| OwnerType: | USER | NULL |
| Owner: | root | NULL |
| CreateTime: | Sat Sep 12 16:50:11 CST 2020 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://cdh2.macro.com:8020/warehouse/tablespace/external/hive/my_first_table | NULL |
| Table Type: | EXTERNAL_TABLE | NULL |
| Table Parameters: | NULL | NULL |
| | EXTERNAL | TRUE |
| | TRANSLATED_TO_EXTERNAL | TRUE |
| | external.table.purge | TRUE |
| | kudu.master_addresses | cdh2.macro.com:7051 |
| | kudu.table_name | impala::default.my_first_table |
| | storage_handler | org.apache.hadoop.hive.kudu.KuduStorageHandler |
| | transient_lastDdlTime | 1599900611 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.kudu.KuduSerDe | NULL |
| InputFormat: | org.apache.hadoop.hive.kudu.KuduInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.kudu.KuduOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | 0 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
+------------------------------+------------------------------------------------------------------------------+------------------------------------------------+
Fetched 31 row(s) in 0.83s我们可以看到kudu表创建成功。
遇到的问题
1、启动过程中报错
Check failed: _s.ok() Bad status:
Invalid argument: Unable to initialize catalog manager: Failed to initialize sys tables
async: on-disk master list解决办法
停掉master和tserver 删掉之前残余的 /kudu_master/fswal_dir/kudu_master/fsdata_dir/kudu_tablet/fswal_dir/kudu_tablet/fsdata_dir几个数据目录
2.建表一直卡住，然后报错:
CreateTablet RPC failed for tablet :not authorized: client connection negotiation failed: client connection to 192.168.0.207:7050: FATAL_UNAUTHORIZED: not authorized: unencrypted connections from publicly routable IPs are prohibited.我们需要在全局gflagfile中配置如下
--rpc_encryption=disabled
--rpc_authentication=disabled
--rusted_subnets=0.0.0.0/0
3.impala-shell中直接报错
FATAL_UNAUTHORIZED: not authorized: unencrypted connections from publicly routable IPs are prohibited解决办法：
同样在impala中配置
--rpc_encryption=disabled
--rpc_authentication=disabled
--rusted_subnets=0.0.0.0/0
4.建表报错
ERROR: ImpalaRuntimeException: Error creating Kudu table 'impala::default.my_first_table'
CAUSED BY: NonRecoverableException: not enough live tablet servers to create a table with the requested replication factor 3; 2 tablet servers are alive这是因为kudu默认的存储副本是3，由于我这是单节点，副本只有1份（这与HDFS不同，HDFS没作此强制限制，1份副本也可以），于是到CM里面修改副本设置，如下图：
本文参与 腾讯云自媒体同步曝光计划，分享自微信公众号。
原始发表：2020-09-27，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读
目录
相关产品与服务
腾讯云开发者
