Introduction An Elasticsearch mapping explosion occurs when an index accumulates thousands of unique field names, typically from unstructured JSON documents with dynamic mapping enabled. Each field consumes heap memory for its mapping metadata, and excessive mappings cause GC pressure, slow cluster state updates, and potential cluster instability.
Symptoms - Cluster state updates take minutes instead of milliseconds - `GET /_cluster/health` shows increasing `task_max_waiting_in_queue_millis` - Master node heap usage consistently near 100% - `GET /my_index/_mapping` returns a massive JSON document - Index creation and template updates become extremely slow
Common Causes - Dynamic mapping enabled on indices receiving arbitrary JSON documents - User-generated metadata fields (e.g., custom attributes, tags) creating unique field names - Nested objects with unbounded key names (e.g., `{"attributes": {"color": "red", "size": "L", ...}}`) - No `total_fields` limit configured on indices - Log ingestion pipelines creating timestamp-based or host-based field names
Step-by-Step Fix 1. **Check current mapping field count": ```bash curl -s localhost:9200/my_index/_mapping | \ python3 -c " import json, sys mapping = json.load(sys.stdin) for idx, data in mapping.items(): fields = data.get('mappings', {}).get('properties', {}) count = len(json.dumps(fields).replace('{}', '').replace('"{', '').replace('}"', '')) print(f'{idx}: {len(fields)} top-level fields') " ```
- 1.**Check the total_fields limit and current usage":
- 2.```bash
- 3.curl -s localhost:9200/_cluster/settings?include_defaults=true | \
- 4.jq '.defaults.index.mapping.total_fields'
- 5.# Default limit: 1000 fields
- 6.
` - 7.**Temporarily increase the limit if the index is still needed":
- 8.```bash
- 9.curl -X PUT localhost:9200/my_index/_settings -H 'Content-Type: application/json' -d '{
- 10."index.mapping.total_fields.limit": 2000
- 11.}'
- 12.
` - 13.**Disable dynamic mapping to prevent further growth":
- 14.```bash
- 15.curl -X PUT localhost:9200/my_index/_mapping -H 'Content-Type: application/json' -d '{
- 16."dynamic": "strict"
- 17.}'
- 18.
` - 19.**Reindex with a controlled mapping":
- 20.```bash
- 21.# Create new index with explicit mapping
- 22.curl -X PUT localhost:9200/my_index_v2 -H 'Content-Type: application/json' -d '{
- 23."mappings": {
- 24."dynamic": "strict",
- 25."properties": {
- 26."timestamp": { "type": "date" },
- 27."message": { "type": "text" },
- 28."level": { "type": "keyword" },
- 29."metadata": {
- 30."type": "object",
- 31."enabled": false
- 32.}
- 33.}
- 34.}
- 35.}'
# Reindex data curl -X POST localhost:9200/_reindex -H 'Content-Type: application/json' -d '{ "source": { "index": "my_index" }, "dest": { "index": "my_index_v2" } }' ```