Fix Elasticsearch Index Not Created

What's Actually Happening

Elasticsearch fails to create a new index. Index creation request returns error or index does not appear in cluster.

The Error You'll See

```bash $ curl -X PUT "http://elasticsearch:9200/myindex"

{ "error": { "type": "index_already_exists_exception", "reason": "index [myindex] already exists" }, "status": 400 } ```

Resource limit error:

```bash $ curl -X PUT "http://elasticsearch:9200/newindex"

{ "error": { "type": "validation_exception", "reason": "index limit exceeded" }, "status": 400 } ```

Mapping error:

```bash $ curl -X PUT "http://elasticsearch:9200/myindex" -d '{"mappings": {...}}'

{ "error": { "type": "mapper_parsing_exception", "reason": "mapping definition has unsupported parameters" }, "status": 400 } ```

Why This Happens

1. 1.Index already exists - Index name already in use
2. 2.Invalid index name - Name format not allowed
3. 3.Mapping error - Invalid field mapping definition
4. 4.Resource limit - Cluster reached index limit
5. 5.Disk space - No space for new index
6. 6.Cluster state issues - Cluster unhealthy or blocked

Step 1: Check Cluster Health

```bash # Check cluster health: curl http://elasticsearch:9200/_cluster/health

# Output: # {"status":"green"} or {"status":"red"}

# Check cluster state: curl http://elasticsearch:9200/_cluster/state

# Check nodes: curl http://elasticsearch:9200/_cat/nodes?v

# Check disk space: curl http://elasticsearch:9200/_cat/allocation?v

# Check indices: curl http://elasticsearch:9200/_cat/indices?v

# If cluster red: curl http://elasticsearch:9200/_cluster/allocation/explain ```

Step 2: Check Index Exists

```bash # Check if index exists: curl -I http://elasticsearch:9200/myindex

# HTTP 200 = exists # HTTP 404 = does not exist

# List all indices: curl http://elasticsearch:9200/_cat/indices?v

# Check specific index: curl http://elasticsearch:9200/myindex

# Check index settings: curl http://elasticsearch:9200/myindex/_settings

# Check index mapping: curl http://elasticsearch:9200/myindex/_mapping

# Delete existing index: curl -X DELETE http://elasticsearch:9200/myindex

# Create with different name: curl -X PUT http://elasticsearch:9200/myindex_v2 ```

Step 3: Check Index Name Validity

```bash # Valid index name rules: # Lowercase only # No spaces # No special chars: \, /, *, ?, ", <, >, |, space, comma, # # Cannot start with: -, _, + # Cannot be: . or .. # Max length: 255 bytes

# Test index name: curl -X PUT "http://elasticsearch:9200/MyIndex" # Error: Invalid index name [MyIndex] must be lowercase

curl -X PUT "http://elasticsearch:9200/my-index-1" # OK - lowercase, hyphen, numbers allowed

curl -X PUT "http://elasticsearch:9200/_myindex" # Error: cannot start with _

# Valid examples: curl -X PUT "http://elasticsearch:9200/myindex" curl -X PUT "http://elasticsearch:9200/my_index_2024" curl -X PUT "http://elasticsearch:9200/logs-2024-01"

# Use index template for naming: curl -X PUT "http://elasticsearch:9200/_template/mytemplate" -d '{"index_patterns":["logs-*"]}' ```

Step 4: Check Mapping Configuration

```bash # Create index with mapping: curl -X PUT "http://elasticsearch:9200/myindex" -H 'Content-Type: application/json' -d ' { "mappings": { "properties": { "title": {"type": "text"}, "date": {"type": "date"}, "price": {"type": "float"} } } }'

# Check mapping errors: # Common issues: # Unknown type: {"type": "unknown"} # Wrong format: missing properties wrapper

# Valid mapping format: { "mappings": { "properties": { "field": {"type": "text"} } } }

# For nested objects: { "mappings": { "properties": { "user": { "type": "nested", "properties": { "name": {"type": "text"}, "email": {"type": "keyword"} } } } } }

# Use index templates: curl -X PUT "http://elasticsearch:9200/_index_template/mytemplate" -d ' { "index_patterns": ["myindex-*"], "template": { "mappings": { "properties": { "field": {"type": "text"} } } } }' ```

Step 5: Check Settings Configuration

```bash # Create index with settings: curl -X PUT "http://elasticsearch:9200/myindex" -d ' { "settings": { "number_of_shards": 3, "number_of_replicas": 1 } }'

# Check shard settings: # number_of_shards must be 1-1022

curl -X PUT "http://elasticsearch:9200/myindex" -d '{"settings":{"number_of_shards": 2000}}' # Error: too many shards

# Check replicas: # Can be 0 or more

# Recommended settings: { "settings": { "number_of_shards": 1, # Single shard for small index "number_of_replicas": 1 # At least one replica } }

# Check index settings after creation: curl http://elasticsearch:9200/myindex/_settings | jq .

# Update settings: curl -X PUT "http://elasticsearch:9200/myindex/_settings" -d '{"number_of_replicas": 2}' ```

Step 6: Check Disk Space

```bash # Check disk allocation: curl http://elasticsearch:9200/_cat/allocation?v

# Output: # disk.used disk.avail disk.total disk.percent # 50GB 100GB 150GB 33

# Check disk watermarks: curl http://elasticsearch:9200/_cluster/settings?include_defaults=true | jq .defaults.cluster.routing.allocation.disk

# Default watermarks: # low: 85% - warn # high: 90% - prevent allocation # flood_stage: 95% - block index creation

# Check if disk full: curl http://elasticsearch:9200/_cat/allocation?v | awk '$5 > 90 {print}'

# Free disk space: # Delete old indices: curl -X DELETE "http://elasticsearch:9200/old-logs-*"

# Or adjust watermarks: curl -X PUT "http://elasticsearch:9200/_cluster/settings" -d ' { "transient": { "cluster.routing.allocation.disk.watermark.low": "90%", "cluster.routing.allocation.disk.watermark.high": "95%" } }' ```

Step 7: Check Index Limits

```bash # Check number of indices: curl http://elasticsearch:9200/_cat/indices | wc -l

# Check shards per node limit: curl http://elasticsearch:9200/_cat/shards | wc -l

# Default max shards per node: 1000

# Check cluster settings: curl http://elasticsearch:9200/_cluster/settings?include_defaults=true | jq .defaults.cluster.max_shards_per_node

# Increase limit: curl -X PUT "http://elasticsearch:9200/_cluster/settings" -d ' { "transient": { "cluster.max_shards_per_node": 2000 } }'

# Check total shards: curl http://elasticsearch:9200/_cat/shards?v | awk '{sum+=1} END {print "Total shards:", sum}'

# If limit exceeded, delete indices: curl -X DELETE "http://elasticsearch:9200/unused-*" ```

Step 8: Check Cluster Block

```bash # Check for blocks: curl http://elasticsearch:9200/_cluster/state | jq .blocks

# Block types: # read-only # read-only-allow-delete

# Check index blocks: curl http://elasticsearch:9200/myindex/_settings | jq .myindex.settings.index.blocks

# Remove read-only block: curl -X PUT "http://elasticsearch:9200/myindex/_settings" -d '{"index.blocks.read_only_allow_delete": false}'

# Or for all indices: curl -X PUT "http://elasticsearch:9200/_all/_settings" -d '{"index.blocks.read_only_allow_delete": null}'

# Check flood stage watermark block: # Automatically set when disk > 95% # Must manually clear after freeing space

curl -X PUT "http://elasticsearch:9200/_cluster/settings" -d ' { "transient": { "cluster.routing.allocation.disk.watermark.low": "85%", "cluster.routing.allocation.disk.watermark.high": "90%", "cluster.routing.allocation.disk.watermark.flood_stage": "95%" } }' ```

Step 9: Check Elasticsearch Logs

```bash # Check Elasticsearch logs: tail -f /var/log/elasticsearch/elasticsearch.log

# Check for index creation errors: grep -i "index" /var/log/elasticsearch/elasticsearch.log | grep -i "error|failed"

# Check for mapping errors: grep -i "mapper" /var/log/elasticsearch/elasticsearch.log | tail -20

# Check for disk errors: grep -i "disk" /var/log/elasticsearch/elasticsearch.log

# Check for resource errors: grep -i "limit|resource" /var/log/elasticsearch/elasticsearch.log

# Journalctl: journalctl -u elasticsearch -f

# Check slow logs: tail -f /var/log/elasticsearch/elasticsearch_index_indexing_slowlog.json ```

Step 10: Elasticsearch Index Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-es-index.sh #!/bin/bash

ES_URL="http://localhost:9200" INDEX=$1

echo "=== Cluster Health ===" curl -s $ES_URL/_cluster/health | jq .

echo "" echo "=== Disk Allocation ===" curl -s $ES_URL/_cat/allocation?v

echo "" echo "=== Indices Count ===" curl -s $ES_URL/_cat/indices?v | wc -l

echo "" echo "=== Shards Count ===" curl -s $ES_URL/_cat/shards?v | wc -l

echo "" echo "=== Index $INDEX Status ===" if [ -n "$INDEX" ]; then curl -s -I $ES_URL/$INDEX | head -1 curl -s $ES_URL/$INDEX/_settings | jq . fi

echo "" echo "=== Recent Errors ===" tail -20 /var/log/elasticsearch/elasticsearch.log | grep -i "error|failed|exception"

echo "" echo "=== Cluster Blocks ===" curl -s $ES_URL/_cluster/state | jq .blocks

echo "" echo "=== Create Test Index ===" curl -s -X PUT "$ES_URL/test-index" -H 'Content-Type: application/json' -d '{"settings":{"number_of_shards":1,"number_of_replicas":0}}' | jq . EOF

chmod +x /usr/local/bin/check-es-index.sh

# Usage: /usr/local/bin/check-es-index.sh myindex

# Create index with validation: curl -X PUT "http://elasticsearch:9200/myindex?pretty" -H 'Content-Type: application/json' -d ' { "settings": { "number_of_shards": 1, "number_of_replicas": 1 }, "mappings": { "properties": { "title": {"type": "text"}, "date": {"type": "date"} } } }' ```

Elasticsearch Index Checklist

Verify the Fix

```bash # After fixing index creation

# 1. Create index curl -X PUT "http://elasticsearch:9200/myindex" // {"acknowledged": true}

# 2. Check index exists curl http://elasticsearch:9200/_cat/indices/myindex?v // Index listed with health green

# 3. Check mapping curl http://elasticsearch:9200/myindex/_mapping // Mapping applied correctly

# 4. Index document curl -X POST "http://elasticsearch:9200/myindex/_doc" -d '{"title":"test"}' // Document indexed

# 5. Search index curl "http://elasticsearch:9200/myindex/_search" // Returns results

# 6. Check cluster health curl http://elasticsearch:9200/_cluster/health // Status green ```

Related Issues

• [Fix Elasticsearch Cluster Red](/articles/fix-elasticsearch-cluster-red)
• [Fix Elasticsearch Query Timeout](/articles/fix-elasticsearch-query-timeout)
• [Fix Elasticsearch Node Not Joining](/articles/fix-elasticsearch-node-not-joining)