1.1 安装gluterfs
1.1.1 环境准备
1.1.1.1 系统环境准备
# 系统信息 [root@gluterfs ~]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) [root@gluterfs ~]# uname -r 3.10.0-327.el7.x86_64 # 网络信息 [root@gluterfs01 ~]# hostname -I 10.0.0.211 172.16.1.211 [root@gluterfs02 ~]# hostname -I 10.0.0.212 172.16.1.212
提示:先在虚拟机中挂载一块新的硬盘作为非系统盘,实际生产中视情况是否使用独立的存储设备。
1.1.1.2 分区并格式化硬盘
# 重新分区 [root@gluterfs ~]# fdisk /dev/sdb Command (m for help): n Select (default p): p Partition number (1-4, default 1): First sector (2048-20971519, default 2048): Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): Command (m for help): w # 格式化硬盘 [root@gluterfs ~]# mkfs.xfs /dev/sdb1
1.1.1.3 挂载并创建共享目录
# 挂载硬盘 [root@gluterfs ~]# mkdir -p /data/ [root@gluterfs ~]# mount /dev/sdb1 /data/ # 创建共享目录 [root@gluterfs01 ~]# mkdir -p /data/{distributed01,replicated01,striped01} [root@gluterfs01 ~]# mkdir -p /data/distributed-replicated0{1,2} [root@gluterfs02 ~]# mkdir -p /data/{distributed02,replicated02,striped02} [root@gluterfs02 ~]# mkdir -p /data/distributed-replicated0{3,4}
1.1.1.4 制作本地yum源文件
[root@gluterfs ~]# cat > /etc/yum.repos.d/CentOS-Gluster.repo << EOF [centos-gluster] name=centos-gluster baseurl=http://10.0.0.1/centos/7/storage/x86_64/gluster-3.10/ enabled=1 gpgcheck=0 EOF
1.1.2 安装部署软件
1.1.2.1 安装软件
[root@gluterfs ~]# yum install -y centos-release-gluster # 如果没有本地源时安装此源 [root@gluterfs ~]# yum install -y glusterfs-server
1.1.2.2 启动服务
[root@gluterfs ~]# systemctl enable glusterd [root@gluterfs ~]# systemctl start glusterd
1.2 配置信任池(即所有节点地址池)
[root@gluterfs01 ~]# gluster peer probe 10.0.0.212 [root@gluterfs02 ~]# gluster peer probe 10.0.0.211
1.3 创建卷
提示:glusterfs的基础卷分为分布式卷、复制卷、条带卷,并且可以由这三种基础卷组合为其他扩展卷。
1.3.1 创建分布式卷
1.3.1.1 语法及示例
# gluster volume create [transport tcp | rdma | tcp,rdma] # gluster volume create test-volume server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4
1.3.1.2 命令实例
[root@gluterfs01 ~]# gluster volume create distributed-volume 10.0.0.211:/data/distributed01 10.0.0.212:/data/distributed02
1.3.2 创建复制卷(RAID 1)
1.3.2.1 语法及示例
# gluster volume create [replica ] [transport tcp | rdma | tcp,rdma] # gluster volume create test-volume replica 2 transport tcp server1:/exp1 server2:/exp2
1.3.2.2 命令实例
[root@gluterfs01 ~]# gluster volume create replicated-volume replica 2 10.0.0.211:/data/replicated01 10.0.0.212:/data/replicated02
1.3.3 创建条带卷(RAID 0)
1.3.3.1 语法及示例
# gluster volume create [stripe ] [transport tcp | rdma | tcp,rdma] # gluster volume create test-volume stripe 2 transport tcp server1:/exp1 server2:/exp2
1.3.3.2 命令实例
[root@gluterfs01 ~]# gluster volume create striped-volume stripe 2 10.0.0.211:/data/striped01 10.0.0.212:/data/striped02
1.3.4 创建分布式条带卷
1.3.4.1 语法及示例(必须偶数个节点)
# gluster volume create [stripe ] [transport tcp | rdma | tcp,rdma] # gluster volume create test-volume stripe 4 transport tcp server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4 server5:/exp5 server6:/exp6 server7:/exp7 server8:/exp8
1.3.5 创建分布式复制卷
1.3.5.1 语法及示例(必须偶数个节点,生产中常用方式)
# gluster volume create [replica ] [transport tcp | rdma | tcp,rdma] # gluster volume create test-volume replica 2 transport tcp server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4 server5:/exp5 server6:/exp6
1.3.5.2 命令实例(此处使用两个节点模拟四个节点时的情况)
[root@gluterfs01 ~]# gluster volume create distributed-replicated-volume replica 2 transport tcp 10.0.0.211:/data/distributed-replicated01 10.0.0.212:/data/distributed-replicated03 10.0.0.211:/data/distributed-replicated02 10.0.0.212:/data/distributed-replicated04 force
提示:复制的顺序和创建的顺序有关:server1:/exp1 server2:/exp2互为复制关系,server1:/exp1 server2:/exp2互为复制关系,以此类推。
1.3.5.3 问题解决
【问题现象】:
[root@gluterfs02 ~]# gluster volume create distributed-replicated-volume replica 2 transport tcp 10.0.0.211:/data/distributed-replicated01 10.0.0.211:/data/distributed-replicated02 10.0.0.212:/data/distributed-replicated03 10.0.0.212:/data/distributed-replicated04 volume create: distributed-replicated-volume: failed: Multiple bricks of a replicate volume are present on the same server. This setup is not optimal. Use 'force' at the end of the command if you want to override this behavior.
【解决办法】:
gluster volume create distributed-replicated-volume replica 2 transport tcp 10.0.0.211:/data/distributed-replicated01 10.0.0.212:/data/distributed-replicated03 10.0.0.211:/data/distributed-replicated02 10.0.0.212:/data/distributed-replicated04 force
提示:如果执行命令时提示“Use 'force' at the end of the command if you want to override this behavior.”时,在命令最后添加force参数执行即可。
1.3.6 创建分布式条带复制卷
1.3.6.1 语法及示例
# gluster volume create [stripe ] [replica ] [transport tcp | rdma | tcp,rdma] # gluster volume create test-volume stripe 2 replica 2 transport tcp server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4 server5:/exp5 server6:/exp6 server7:/exp7 server8:/exp8
1.4 启动卷并查看状态
1.4.1 启动卷
[root@gluterfs01 ~]# gluster volume start distributed-volume [root@gluterfs01 ~]# gluster volume start replicated-volume [root@gluterfs01 ~]# gluster volume start striped-volume [root@gluterfs01 ~]# gluster volume start distributed-replicated-volume
1.4.2 查看卷状态
[root@gluterfs01 ~]# gluster volume info Volume Name: distributed-replicated-volume Type: Distributed-Replicate Volume ID: 60b17fbc-309e-4893-82ab-208f946b7b74 Status: Started Snapshot Count: 0 Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.0.0.211:/data/distributed-replicated01 Brick2: 10.0.0.212:/data/distributed-replicated03 Brick3: 10.0.0.211:/data/distributed-replicated02 Brick4: 10.0.0.212:/data/distributed-replicated04 Options Reconfigured: transport.address-family: inet nfs.disable: on Volume Name: distributed-volume Type: Distribute Volume ID: 4bb1530e-99a1-44fd-9fd1-b90e387262dc Status: Started Snapshot Count: 0 Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: 10.0.0.211:/data/distributed01 Brick2: 10.0.0.212:/data/distributed02 Options Reconfigured: transport.address-family: inet nfs.disable: on Volume Name: replicated-volume Type: Replicate Volume ID: 10517a05-7fdd-47d5-8018-d1788c0482b5 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.0.0.211:/data/replicated01 Brick2: 10.0.0.212:/data/replicated02 Options Reconfigured: transport.address-family: inet nfs.disable: on Volume Name: striped-volume Type: Stripe Volume ID: 50be9ee6-f8c5-424b-ba2d-9812d3cd6270 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.0.0.211:/data/striped01 Brick2: 10.0.0.212:/data/striped02 Options Reconfigured: transport.address-family: inet nfs.disable: on
1.5 使用gluterfs
1.5.1 创建共享目录
[root@gluterfs01 ~]# mkdir -p /mnt/{distributed,replicated,striped,distributed-replicated}
1.5.2 挂载共享卷
[root@gluterfs01 ~]# mount.glusterfs 10.0.0.211:/distributed-volume /mnt/distributed/ [root@gluterfs01 ~]# mount.glusterfs 10.0.0.211:/replicated-volume /mnt/replicated/ [root@gluterfs01 ~]# mount.glusterfs 10.0.0.211:/striped-volume /mnt/striped/ [root@gluterfs01 ~]# mount.glusterfs 10.0.0.211:/distributed-replicated-volume /mnt/distributed-replicated/
1.5.3 查看挂载情况
[root@gluterfs01 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda3 98G 2.3G 96G 3% / devtmpfs 228M 0 228M 0% /dev tmpfs 237M 0 237M 0% /dev/shm tmpfs 237M 4.6M 233M 2% /run tmpfs 237M 0 237M 0% /sys/fs/cgroup /dev/sda1 1014M 110M 905M 11% /boot tmpfs 48M 0 48M 0% /run/user/0 /dev/sdb1 10G 33M 10G 1% /data 10.0.0.211:/distributed-volume 20G 66M 20G 1% /mnt/distributed 10.0.0.211:/replicated-volume 10G 33M 10G 1% /mnt/replicated 10.0.0.211:/striped-volume 20G 66M 20G 1% /mnt/striped 10.0.0.211:/distributed-replicated-volume 20G 66M 20G 1% /mnt/distributed-replicated
1.6 测试卷
1.6.1 测试分布式卷
1.6.1.1 写入文件
[root@gluterfs01 ~]# echo "distributed01" > /mnt/distributed/distributed01.txt [root@gluterfs01 ~]# echo "distributed02" > /mnt/distributed/distributed02.txt
1.6.1.2 查看写入的文件
[root@gluterfs01 ~]# ll /data/distributed01/* -rw-r--r-- 2 root root 14 Dec 14 15:30 /data/distributed01/distributed02.txt [root@gluterfs02 ~]# ll /data/distributed02/* -rw-r--r-- 2 root root 14 Dec 14 15:30 /data/distributed02/distributed01.txt
1.6.2 测试复制卷
1.6.2.1 写入文件
[root@gluterfs01 ~]# echo "replicated" > /mnt/replicated/replicated.txt
1.6.2.2 查看写入的文件
[root@gluterfs01 ~]# ll /data/replicated01/* -rw-r--r-- 2 root root 11 Dec 14 15:34 /data/replicated01/replicated.txt [root@gluterfs02 ~]# ll /data/replicated02/* -rw-r--r-- 2 root root 11 Dec 14 15:34 /data/replicated02/replicated.txt
1.6.3 测试条带卷
1.6.3.1 写入文件
[root@gluterfs01 ~]# ll -h /var/log/messages-20171214 -rw-------. 1 root root 736K Dec 14 15:19 /var/log/messages-20171214 [root@gluterfs01 ~]# cat /var/log/messages-20171214 >> /mnt/striped/striped.txt
1.6.3.2 查看写入的文件
[root@gluterfs01 ~]# ll -h /data/striped01/* -rw-r--r-- 2 root root 384K Dec 14 15:40 /data/striped01/striped.txt [root@gluterfs02 ~]# ll -h /data/striped02/* -rw-r--r-- 2 root root 352K Dec 14 15:40 /data/striped02/striped.txt
1.6.4 测试分布式复制卷
1.6.4.1 写入文件
[root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated01.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated02.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated03.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated04.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated05.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated05.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated06.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated07.txt [root@gluterfs01 ~]# cat /var/log/messages-20171214 > /mnt/distributed-replicated/distributed-replicated08.txt
1.6.4.2 查看写入的文件
# gluterfs01节点查看 [root@gluterfs01 ~]# tree /data/distributed-replicated0* /data/distributed-replicated01 ├── distributed-replicated01.txt ├── distributed-replicated02.txt ├── distributed-replicated03.txt └── distributed-replicated04.txt /data/distributed-replicated02 ├── distributed-replicated05.txt ├── distributed-replicated06.txt ├── distributed-replicated07.txt └── distributed-replicated08.txt 0 directories, 8 files # gluterfs02节点查看 [root@gluterfs02 ~]# tree /data/distributed-replicated0* /data/distributed-replicated03 ├── distributed-replicated01.txt ├── distributed-replicated02.txt ├── distributed-replicated03.txt └── distributed-replicated04.txt /data/distributed-replicated04 ├── distributed-replicated05.txt ├── distributed-replicated06.txt ├── distributed-replicated07.txt └── distributed-replicated08.txt 0 directories, 8 files

我的微信
如果有技术上的问题可以扫一扫我的微信