在centos系统中,hbase的资源调度主要依赖于yarn(yet another resource negotiator)来实现。以下是hbase在centos中进行资源调度的基本步骤:
1. 安装和配置YARN
首先,确保你的CentOS系统上已经安装了Hadoop和YARN。如果没有安装,可以参考Hadoop官方文档进行安装和配置。
安装Hadoop
wget https://archive.apache.org/dist/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz tar -xzvf hadoop-3.3.1.tar.gz -C /usr/local/
配置Hadoop
编辑/usr/local/hadoop/etc/hadoop/core-site.xml:
<configuration> <property> <name>fs.defaultFSname> <value>hdfs://localhost:9000value> property> configuration>
编辑/usr/local/hadoop/etc/hadoop/hdfs-site.xml:
<configuration> <property> <name>dfs.replicationname> <value>1value> property> configuration>
编辑/usr/local/hadoop/etc/hadoop/yarn-site.xml:
自定义设置的程度更高可以满足大部分中小型企业的建站需求,同时修正了上一版中发现的BUG,优化了核心的代码占用的服务器资源更少,执行速度比上一版更快 主要的特色功能如下: 1)特色的菜单设置功能,菜单设置分为顶部菜单和底部菜单,每一项都可以进行更名、选择是否隐 藏,排序等。 2)增加企业基本信息设置功能,输入的企业信息可以在网页底部的醒目位置看到。 3)增加了在线编辑功能,输入产品信息,企业介绍等栏
<configuration> <property> <name>yarn.nodemanager.aux-servicesname> <value>mapreduce_shufflevalue> property> <property> <name>yarn.nodemanager.resource.memory-mbname> <value>4096value> property> <property> <name>yarn.nodemanager.resource.cpu-vcoresname> <value>4value> property> configuration>
2. 启动Hadoop集群
启动HDFS和YARN:
start-dfs.sh start-yarn.sh
3. 配置HBase使用YARN
编辑HBase的配置文件/usr/local/hbase/conf/hbase-site.xml,添加以下配置:
<configuration> <property> <name>hbase.rootdirname> <value>hdfs://localhost:9000/hbasevalue> property> <property> <name>hbase.cluster.distributedname> <value>truevalue> property> <property> <name>hbase.zookeeper.quorumname> <value>localhostvalue> property> <property> <name>hbase.zookeeper.property.dataDirname> <value>/usr/local/hbase/zookeepervalue> property> <property> <name>hbase.master.info.portname> <value>16000value> property> <property> <name>hbase.regionserver.info.portname> <value>16020value> property> <property> <name>hbase.regionserver.handler.countname> <value>100value> property> <property> <name>hbase.regionserver.global.memstore.sizename> <value>0.4value> property> <property> <name>hbase.regionserver.wal.codecname> <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodecvalue> property> <property> <name>hbase.regionserver.hlog.blocksizename> <value>64Mvalue> property> <property> <name>hbase.regionserver.hlog.writer.implname> <value>org.apache.hadoop.hbase.regionserver.wal.HLogWriterFactoryvalue> property> <property> <name>hbase.regionserver.hlog.syncer.countname> <value>1value> property> <property> <name>hbase.regionserver.hlog.syncer.implname> <value>org.apache.hadoop.hbase.regionserver.wal.Syncervalue> property> <property> <name>hbase.regionserver.hlog.syncer.checkpointname> <value>100value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.intervalname> <value>300000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>60000value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.sizename> <value>104857600value> property> <property> <name>hbase.regionserver.hlog.syncer.checkpoint.timeoutname> <value>6000









