广东旅游网站建设,域名服务商有哪些,深圳网站设计兴田德润i简介,qt 做网站hive—high Avaliable
hive的搭建方式有三种#xff0c;分别是
1、Local/Embedded Metastore Database (Derby)
2、Remote Metastore Database
3、Remote Metastore Server
一般情况下#xff0c;我们在学习的时候直接使用hive –service metastore的方式…hive—high Avaliable
hive的搭建方式有三种分别是
1、Local/Embedded Metastore Database (Derby)
2、Remote Metastore Database
3、Remote Metastore Server
一般情况下我们在学习的时候直接使用hive –service metastore的方式启动服务端使用hive的方式直接访问登录客户端除了这种方式之外hive提供了hiveserver2的服务端启动方式提供了beeline和jdbc的支持并且官网也提出一般在生产环境中使用hiveserver2的方式比较多如图 使用hiveserver2的优点如下
1、在应用端不需要部署hadoop和hive的客户端
2、hiveserver2不用直接将hdfs和metastore暴露给用户
3、有HA机制解决应用端的并发和负载问题
4、jdbc的连接方式可以使用任何语言方便与应用进行数据交互
本文档主要介绍如何进行hive的HA的搭建
如何进行搭建参照之前hadoop的HA使用zookeeper完成HA 1、环境如下:
Node01Node02Node03Node04Namenode11Journalnode111Datanode111Zkfc11zookeeper111resourcemanager111nodemanager111Hiveserver21beeline1
2、node02—hive-site.xml
property namehive.metastore.warehouse.dir/name value/user/hive/warehouse/value
/property
property namejavax.jdo.option.ConnectionURL/name valuejdbc:mysql://node01:3306/hive?createDatabaseIfNotExisttrue/value
/property
property namejavax.jdo.option.ConnectionDriverName/name valuecom.mysql.jdbc.Driver/value
/property
property namejavax.jdo.option.ConnectionUserName/name valueroot/value
/property
property namejavax.jdo.option.ConnectionPassword/name value123/value
/property
propertynamehive.server2.support.dynamic.service.discovery/namevaluetrue/value
/property
propertynamehive.server2.zookeeper.namespace/namevaluehiveserver2_zk/value
/property
propertynamehive.zookeeper.quorum/namevaluenode01:2181,node02:2181,node03:2181/value
/property
propertynamehive.zookeeper.client.port/namevalue2181/value
/property
propertynamehive.server2.thrift.bind.host/namevaluenode02/value
/property
propertynamehive.server2.thrift.port/namevalue10001/value
/property3、node4—hive-site.xml
property namehive.metastore.warehouse.dir/name value/user/hive/warehouse/value
/property
property namejavax.jdo.option.ConnectionURL/name valuejdbc:mysql://node01:3306/hive?createDatabaseIfNotExisttrue/value
/property
property namejavax.jdo.option.ConnectionDriverName/name valuecom.mysql.jdbc.Driver/value
/property
property namejavax.jdo.option.ConnectionUserName/name valueroot/value
/property
property namejavax.jdo.option.ConnectionPassword/name value123/value
/property
propertynamehive.server2.support.dynamic.service.discovery/namevaluetrue/value
/property
propertynamehive.server2.zookeeper.namespace/namevaluehiveserver2_zk/value
/property
propertynamehive.zookeeper.quorum/namevaluenode01:2181,node02:2181,node03:2181/value
/property
propertynamehive.zookeeper.client.port/namevalue2181/value
/property
propertynamehive.server2.thrift.bind.host/namevaluenode04/value
/property
propertynamehive.server2.thrift.port/namevalue10001/value
/property4、使用jdbc或者beeline两种方式进行访问
1 beeline
!connect jdbc:hive2://node01,node02,node03/;serviceDiscoveryModezooKeeper;zooKeeperNamespacehiveserver2_zk root 123
2jdbc
public class HiveJdbcClient2 {private static String driverName org.apache.hive.jdbc.HiveDriver;public static void main(String[] args) throws SQLException {try {Class.forName(driverName);} catch (ClassNotFoundException e) {e.printStackTrace();}Connection conn DriverManager.getConnection(jdbc:hive2://node01,node02,node03/default;serviceDiscoveryModezooKeeper;zooKeeperNamespacehiveserver2_zk, root, );Statement stmt conn.createStatement();String sql select * from tbl;ResultSet res stmt.executeQuery(sql);while (res.next()) {System.out.println(res.getString(1));}}
}
