ICode9
精准搜索请尝试： 精确搜索
首页
编程语言>
数据库
系统相关
互联网
其他分享
Python
Java
JavaScript
android
PHP
首页 > 其他分享> 文章详细

replication与shard分片结合使用

2022-03-05 16:35:38  阅读：203  来源： 互联网

标签：shop 10.0 -- userid shard replication mongos 分片 rs3


部署脚本

#!/bin/bash
#复制集配置
IP='10.0.0.12'  #主机ip
NA='rs3'  #复制集名称

if [ "$1" = "reset" ]
then
    kill -2 `ps -ef|grep -v grep |grep logpath|awk '{print $2}'`
    sleep 2
    rm -rf /mongodb/m{17,18,19,log}/*
    exit
fi

if [ "$1" = "install" ]
then
  mkdir -p /mongodb/m{17,18,19,log}/
  mongod --dbpath /mongodb/m17 --logpath /mongodb/mlog/m17.log --logappend --fork --port 27017 --replSet ${NA}  --smallfiles
  mongod --dbpath /mongodb/m18 --logpath /mongodb/mlog/m18.log --logappend --fork --port 27018 --replSet ${NA}  --smallfiles
  mongod --dbpath /mongodb/m19 --logpath /mongodb/mlog/m19.log --logappend --fork --port 27019 --replSet ${NA}  --smallfiles
   exit
 fi
 
 if [ "$1" = "repl" ]
then
mongo --port 27017 <<EOF

use admin;
var rsconf = {
_id:'${NA}',
members:[
{_id:0,host:'${IP}:27017'},
{_id:1,host:'${IP}:27018'},
{_id:2,host:'${IP}:27019'},
]
};
rs.initiate(rsconf);
<<
EOF
fi

部署情况

提前部署好两个复制集，部署脚本如上
现在部署情况如下
rs3:
  10.0.0.12  27017   
  10.0.0.12  27018
  10.0.0.12  27019

rs4:
  10.0.0.13  27017
  10.0.0.13  27018
  10.0.0.13  27019

configsvr :
  10.0.0.11  27020

mongos：
  10.0.0.11  30000

部署configsvr和mongos

mongod --dbpath /mongodb/m20 --logpath /mongodb/mlog/m20.log --logappend --fork --port 27020 --configsvr 
mongos --logpath /mongodb/mlog/m30.log --logappend --fork --port 30000 --configdb 10.0.0.11:27020 



[mongod@mcw01 ~]$ mongod --dbpath /mongodb/m20 --logpath /mongodb/mlog/m20.log --logappend --fork --port 27020 --configsvr 
about to fork child process, waiting until server is ready for connections.
forked process: 20138
child process started successfully, parent exiting
[mongod@mcw01 ~]$ mongos --logpath /mongodb/mlog/m30.log --logappend --fork --port 30000 --configdb 10.0.0.11:27020 
2022-03-05T15:18:39.286+0800 W SHARDING [main] Running a sharded cluster with fewer than 3 config servers should only be done for testing purposes and is not recommended for production.
about to fork child process, waiting until server is ready for connections.
forked process: 20158
child process started successfully, parent exiting
[mongod@mcw01 ~]$

添加分片

复制集和分片结合的。配置上没太多区别。就是在添加分片的时候，一个加上复制集名称，一个不加。一个复制集就是一个整体，当成分片的一个单元。复制集里的每个副本都是一个备份，其中写入的时候写到主副本就行了吧，复制集会实现其它备副本的复制。复制集中的副本是不是也可以设置在不同的主机上呢，这里设置的是同一台主机上，备份的安全性上考虑，肯定备份不要在同一个主机才安全。
sh.addShard('rs3/10.0.0.12:27017');

添加分片只需要添加复制集的主17就可以了，然后查看分片，可以看到主副本，以及其它副本的。
[mongod@mcw01 ~]$ mongo 10.0.0.11:30000
MongoDB shell version: 3.2.8
connecting to: 10.0.0.11:30000/test
mongos> sh.addShard('rs3/10.0.0.12:27017');
{ "shardAdded" : "rs3", "ok" : 1 }
mongos> sh.addShard('rs4/10.0.0.13:27017');
{ "shardAdded" : "rs4", "ok" : 1 }
mongos> sh.status();
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("62230ecfe5cc911b886d1023")
}
  shards:  #写数据，分片上的数据是独有的，而同一个副本集不同副本中的数据是相同的，是个复制的数据。当数据分配存储到rs3分片时，18,19两个副本会从17主上复制一份过来作为备份用
    {  "_id" : "rs3",  "host" : "rs3/10.0.0.12:27017,10.0.0.12:27018,10.0.0.12:27019" }
    {  "_id" : "rs4",  "host" : "rs4/10.0.0.13:27017,10.0.0.13:27018,10.0.0.13:27019" }
  active mongoses:
    "3.2.8" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:

mongos>

设置分片库表，分片规则，插入数据，手动预分片

给shop开启分片，给shop下的user分片，根据userid分片。当userid是1000的时候切割一下，2000的时候切割一下......
mongos> show dbs;
config  0.000GB
mongos> sh.enableSharding('shop')
{ "ok" : 1 }
mongos> sh.shardCollection('shop.user',{userid:1})
{ "collectionsharded" : "shop.user", "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:10000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:1000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:2000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:3000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:4000})
{ "ok" : 1 }
mongos> 



查看已经是预分片了，并且res3复制集有，rs4复制集也有分片，各有三个，还是均衡的
mongos> sh.status();
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("62230ecfe5cc911b886d1023")
}
  shards:
    {  "_id" : "rs3",  "host" : "rs3/10.0.0.12:27017,10.0.0.12:27018,10.0.0.12:27019" }
    {  "_id" : "rs4",  "host" : "rs4/10.0.0.13:27017,10.0.0.13:27018,10.0.0.13:27019" }
  active mongoses:
    "3.2.8" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        3 : Success
  databases:
    {  "_id" : "shop",  "primary" : "rs4",  "partitioned" : true }
        shop.user
            shard key: { "userid" : 1 }
            unique: false
            balancing: true
            chunks:
                rs3    3
                rs4    3
            { "userid" : { "$minKey" : 1 } } -->> { "userid" : 1000 } on : rs4 Timestamp(3, 0) 
            { "userid" : 1000 } -->> { "userid" : 2000 } on : rs4 Timestamp(4, 0) 
            { "userid" : 2000 } -->> { "userid" : 3000 } on : rs3 Timestamp(4, 1) 
            { "userid" : 3000 } -->> { "userid" : 4000 } on : rs3 Timestamp(3, 4) 
            { "userid" : 4000 } -->> { "userid" : 10000 } on : rs3 Timestamp(3, 5) 
            { "userid" : 10000 } -->> { "userid" : { "$maxKey" : 1 } } on : rs4 Timestamp(2, 1) 

mongos>


我们在mongos上进入shop库,插入数据
mongos> use shop
switched to db shop
mongos> for(var i=1;i<=4000;i++){db.user.insert({userid:i,intro:'i am mcw, from china'})}
WriteResult({ "nInserted" : 1 })
mongos>

查看插入数据的效果

首先看下rs3上的主副本：有两千到四千的数据
[mongod@mcw02 ~]$ mongo --port 27017
rs3:PRIMARY> show dbs;
local  0.000GB
shop   0.000GB
rs3:PRIMARY> use shop;
switched to db shop
rs3:PRIMARY> show tables;
user
rs3:PRIMARY> db.user.find().count();
2001
rs3:PRIMARY> db.user.find().skip(1999);
{ "_id" : ObjectId("622314a446d8d9b89fc9e710"), "userid" : 3999, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("622314a446d8d9b89fc9e711"), "userid" : 4000, "intro" : "i am mcw, from china" }
rs3:PRIMARY> db.user.find().limit(3);
{ "_id" : ObjectId("6223149d46d8d9b89fc9df41"), "userid" : 2000, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df42"), "userid" : 2001, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df43"), "userid" : 2002, "intro" : "i am mcw, from china" }
rs3:PRIMARY> 

看下rs3上的备副本：数据跟主副本时一致的
[mongod@mcw02 ~]$ mongo --port 27018
rs3:SECONDARY> use shop
switched to db shop
rs3:SECONDARY> db.user.find().count();
2001
rs3:SECONDARY> db.user.find().skip(1999);
{ "_id" : ObjectId("622314a446d8d9b89fc9e710"), "userid" : 3999, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("622314a446d8d9b89fc9e711"), "userid" : 4000, "intro" : "i am mcw, from china" }
rs3:SECONDARY> db.user.find().limit(3);
{ "_id" : ObjectId("6223149d46d8d9b89fc9df41"), "userid" : 2000, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df42"), "userid" : 2001, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df43"), "userid" : 2002, "intro" : "i am mcw, from china" }
rs3:SECONDARY> 


查看rs4上主副本：发现数据跟rs.status()是一样的。从1-1999都在rs4副本上
[mongod@mcw03 ~]$ mongo --port 27017
rs4:PRIMARY> use shop
switched to db shop
rs4:PRIMARY> db.user.find().count();
1999
rs4:PRIMARY> db.user.find().skip(1997);
{ "_id" : ObjectId("6223149d46d8d9b89fc9df3f"), "userid" : 1998, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df40"), "userid" : 1999, "intro" : "i am mcw, from china" }
rs4:PRIMARY> db.user.find().limit(3);
{ "_id" : ObjectId("6223149446d8d9b89fc9d772"), "userid" : 1, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149446d8d9b89fc9d773"), "userid" : 2, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149446d8d9b89fc9d774"), "userid" : 3, "intro" : "i am mcw, from china" }
rs4:PRIMARY>

 

标签：shop,10.0,--,userid,shard,replication,mongos,分片,rs3
来源： https://www.cnblogs.com/machangwei-8/p/15968441.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

关于我们 | 联系我们 | 留言反馈

专注分享技术，共同学习，共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有