网页站点怎么命名,wordpress图片文件目录,7天酒店网站建设优势,昆明手机网站开发背景#xff1a;
尝试解决如下问题#xff1a;单分片存在过多文档#xff0c;超过lucene限制 分析
1.一般为日志数据或者OLAP数据#xff0c;直接删除索引重建
2.尝试保留索引#xff0c;生成新索引 - 数据写入新索引#xff0c;查询时候包含 old_index,new_index
3.…背景
尝试解决如下问题单分片存在过多文档超过lucene限制 分析
1.一般为日志数据或者OLAP数据直接删除索引重建
2.尝试保留索引生成新索引 - 数据写入新索引查询时候包含 old_index,new_index
3.尝试split split index API
如果需要将当前index的primary shard数量增加时可以使用split index api。
会生成一个新index但会保留原来的index。 步骤
确保source index只读
PUT source_index/_settings
{settings: {index.blocks.write: true }
}
spilt API修改primary shard数量
POST source_index/_split/new_index
{settings: {index.number_of_shards: 10}
}
监控执行进度
GET _cat/recovery/new_index 测试
版本 7.17.5
# 新建测试索引
PUT test_split
{}# 关闭source索引的写入
PUT /test_split/_settings
{settings: {index.blocks.write: true }
}# 执行split API
POST /test_split/_split/test_split_new
{settings: {index.number_of_shards: 12}
}
遇到报错并解决在split API执行阶段
1. source 索引必须是 read-only 的{error: {root_cause: [{type: illegal_state_exception,reason: index test_split must be read-only to resize index. use \index.blocks.writetrue\}],type: illegal_state_exception,reason: index test_split must be read-only to resize index. use \index.blocks.writetrue\},status: 500
}2. source分片数3必须是target分片数的因子所以target不能为11可以为12{error: {root_cause: [{type: illegal_argument_exception,reason: the number of source shards [3] must be a factor of [11]}],type: illegal_argument_exception,reason: the number of source shards [3] must be a factor of [11]},status: 400
} 应用
集群版本 6.8.5
设置source索引 index.blocks.write: true 之后执行split API异常
{error: {root_cause: [{type: remote_transport_exception,reason: [es-log-all-2][10.xx.x.xx:9300][indices:admin/resize]}],type: illegal_state_exception,reason: the number of routing shards [5] must be a multiple of the target shards [20]},status: 500
}
即目标索引的主分片个数必须是index.number_of_routing_shards的因数
注意number_of_routing_shards 不可以动态修改
结论ES6.8无法通过split API解决索引分片过少的问题
官方docSplit index API | Elasticsearch Guide [8.9] | Elastic Shrink index API
如果需要将当前index的primary shard数量减少时可以使用shrink index api。
会生成一个新index但会保留原来的index。
Shrinks an existing index into a new index with fewer primary shards.
POST /my-index-000001/_shrink/shrunk-my-index-000001
步骤
# 新建index
PUT test_shrink
{}# 查看索引的shard在哪些node
GET _cat/shards/test_shrink?v# 将所有主分片分配到node1副本设置为0设置readOnly
PUT test_shrink/_settings
{settings: {index.number_of_replicas: 0,index.routing.allocation.require._name: node-es-0,index.blocks.write: true}
}# 执行shrink API
POST /test_shrink/_shrink/new_test_shrink
{settings: {index.number_of_replicas: 1,index.number_of_shards: 1, index.codec: best_compression },aliases: {my_search_indices: {}}
}
如果上述命令修改成
POST /test_shrink/_shrink/new_test_shrink
{settings: {index.number_of_replicas: 1,index.number_of_shards: 2, index.codec: best_compression },aliases: {my_search_indices: {}}
}
新的number_of_shards不是source index的number_of_shards的因子那么出现如下错误
{error: {root_cause: [{type: illegal_argument_exception,reason: the number of source shards [3] must be a multiple of [2]}],type: illegal_argument_exception,reason: the number of source shards [3] must be a multiple of [2]},status: 400
} 官方docShrink index API | Elasticsearch Guide [8.9] | Elastic
