Fix Magento 2 Indexer Stuck Processing Forever

What's Actually Happening

You're running Magento 2 and suddenly products don't appear in search results, category pages are empty or outdated, and prices are wrong. When you check the indexer status, you see one or more indexers stuck in "processing" state indefinitely. Running bin/magento indexer:reindex doesn't help - it either hangs or completes but the status remains "processing". The indexer is locked and can't complete its job.

This is a common Magento 2 issue that affects stores during high traffic, bulk imports, server crashes, or when indexers are running via cron jobs that were interrupted. The indexer state becomes corrupted and Magento thinks an indexing operation is still in progress when it's actually not.

The Error You'll See

Magento 2 indexer issues manifest in several ways:

# Reindex hangs without completion $ bin/magento indexer:reindex catalog_category_product Catalog Product indexer is locked and cannot be reindexed. # Command hangs here indefinitely...

# Reindex with timeout errors $ bin/magento indexer:reindex SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction

# Cron indexer stuck $ bin/magento cron:run [Magento\Framework\Exception\LocalizedException] Cannot run indexer process because it's already running

# Products not appearing in search # Frontend search returns no results # Category pages show "We can't find products matching the selection"

# Deadlock errors during reindex SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

# Memory exhaustion during reindex PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 134217728 bytes) ```

Additional symptoms: - Products added yesterday don't appear in search - Category page shows old product count - Price changes not reflected on frontend - Stock status incorrect - "Processing" status never changes to "Valid" - Cron jobs failing with indexer locks - Admin showing outdated product counts - Search returning zero results for valid queries

Why This Happens

1. 1.Stale Indexer Lock: An indexing process was interrupted - server crash, PHP timeout, process killed, or MySQL restart - leaving the indexer in "processing" state. The m2_indexer_state table wasn't updated to reflect the process termination.
2. 2.MySQL Table Locks: Another process has a lock on tables the indexer needs - running imports, other indexers, backup processes, or long-running queries. The indexer waits indefinitely for locks to release.
3. 3.Deadlock During Reindex: Multiple indexers or processes trying to access the same tables simultaneously cause database deadlocks. MySQL detects the deadlock but Magento doesn't properly handle the retry.
4. 4.Insufficient PHP Memory: Large catalogs require significant memory for indexing. PHP memory_limit is exceeded during index, process dies, but lock remains.
5. 5.Cron Job Overlap: Multiple cron instances running indexer jobs simultaneously when lock isn't properly checked. Each tries to reindex, causing conflicts.
6. 6.Mview Subscription Issues: Magento's Mview (materialized view) has corrupted subscription data in mview_state table, causing indexers to never complete their scheduled updates.
7. 7.Large Dataset Timeout: Catalog has grown beyond what the indexer can process within PHP max_execution_time or MySQL wait_timeout. Process times out but lock persists.
8. 8.Corrupted Index Tables: The index tables themselves have corrupted data or inconsistent state from previous failed reindex attempts.

Step 1: Identify Which Indexers Are Stuck

First, check the status of all indexers and identify which are problematic.

```bash # SSH into your Magento server cd /var/www/magento

# Check all indexer status bin/magento indexer:status

# Check specific indexer status bin/magento indexer:status catalog_category_product bin/magento indexer:status catalogsearch_fulltext bin/magento indexer:status catalog_product_price

# List all indexers bin/magento indexer:info

# Check indexer state in database mysql -u magento_user -p -e " SELECT indexer_id, status, updated, hash_key FROM m2_indexer_state; " magento_db

# Check mview state as well mysql -u magento_user -p -e " SELECT view_id, mode, status, updated FROM mview_state; " magento_db

# Check for locked tables mysql -u magento_user -p -e "SHOW OPEN TABLES WHERE In_use > 0;" magento_db

# Check for long-running queries mysql -u magento_user -p -e " SELECT Id, User, Time, State, LEFT(INFO, 100) as Query FROM information_schema.PROCESSLIST WHERE Time > 30 ORDER BY Time DESC; "

# Check if any indexer processes are actually running ps aux | grep -E "indexer|reindex" | grep -v grep

# Check cron processes ps aux | grep cron | grep -v grep

# Check recent indexer logs tail -100 var/log/indexer.log tail -100 var/log/system.log | grep -i indexer tail -100 var/log/debug.log | grep -i indexer

# Check if there are any PHP processes running indexing pgrep -af "bin/magento.*indexer" ```

Step 2: Clear Stale Indexer Locks in Database

Reset the stuck indexer state directly in the database.

```bash # Backup indexer state table first mysql -u magento_user -p magento_db > /tmp/indexer_state_backup_$(date +%Y%m%d).sql

# Check current state mysql -u magento_user -p -e "SELECT * FROM m2_indexer_state;" magento_db

# Option 1: Reset all indexer states mysql -u magento_user -p magento_db << 'EOF' UPDATE m2_indexer_state SET status = 'valid', updated = NOW() WHERE status IN ('processing', 'working'); EOF

# Option 2: Reset specific indexer mysql -u magento_user -p magento_db << 'EOF' UPDATE m2_indexer_state SET status = 'valid', updated = NOW() WHERE indexer_id = 'catalog_category_product'; EOF

# Reset mview states as well mysql -u magento_user -p -e " UPDATE mview_state SET status = 'idle' WHERE status = 'working'; " magento_db

# Clear semaphore locks mysql -u magento_user -p magento_db << 'EOF' TRUNCATE TABLE m2_semaphore_locks; EOF

# Or if semaphore table doesn't exist, check: mysql -u magento_user -p -e "SHOW TABLES LIKE '%semaphore%';" magento_db mysql -u magento_user -p -e "SHOW TABLES LIKE '%lock%';" magento_db

# Verify state was reset mysql -u magento_user -p -e "SELECT indexer_id, status FROM m2_indexer_state;" magento_db

# Check status via CLI bin/magento indexer:status ```

Step 3: Kill Stuck MySQL Processes

Terminate long-running MySQL queries that are blocking indexers.

```bash # Find long-running queries mysql -u magento_user -p -e " SELECT Id, User, Time, State, LEFT(INFO, 150) as Query FROM information_schema.PROCESSLIST WHERE db = 'magento_db' AND Time > 60; "

# Kill specific process by ID mysql -u magento_user -p -e "KILL 123;"

# Kill all long-running queries (be careful!) mysql -u magento_user -p -e " SELECT CONCAT('KILL ', Id, ';') FROM information_schema.PROCESSLIST WHERE db = 'magento_db' AND Time > 300 AND Command = 'Query'; " | mysql -u magento_user -p

# Kill all Magento queries (nuclear option) mysql -u magento_user -p -e " SELECT GROUP_CONCAT(CONCAT('KILL ',Id) SEPARATOR '; ') FROM information_schema.PROCESSLIST WHERE db = 'magento_db'; " | mysql -u magento_user -p

# Check if locks are cleared mysql -u magento_user -p -e "SHOW OPEN TABLES WHERE In_use > 0;" magento_db

# Check InnoDB lock status mysql -u magento_user -p -e " SELECT * FROM information_schema.INNODB_LOCKS; SELECT * FROM information_schema.INNODB_LOCK_WAITS; "

# If still locked, restart MySQL # Only if safe to do so: sudo systemctl restart mysql # Or: sudo systemctl restart mariadb ```

Step 4: Clear Magento Cache and Locks

Clear Magento's internal locks and caches.

```bash # Clear Magento cache bin/magento cache:clean bin/magento cache:flush

# Clear specific caches bin/magento cache:clean config layout block_html collections bin/magento cache:clean catalog_category catalog_product

# Clear FPC (Full Page Cache) bin/magento cache:clean full_page

# Remove lock files manually rm -rf var/locks/* rm -rf var/cache/* rm -rf var/page_cache/* rm -rf generated/code/* rm -rf generated/metadata/*

# Clear Redis cache if used redis-cli FLUSHDB # Or for specific Magento database: redis-cli -n 0 FLUSHDB redis-cli -n 1 FLUSHDB

# Clear Magento's file locks rm -f var/.lock rm -f var/*.lock

# Check for cron locks ls -la var/ rm -f var/.cron_lock

# Clear compiled DI rm -rf generated/code/Magento rm -rf generated/code/*/ bin/magento setup:di:compile

# Regenerate static content rm -rf pub/static/frontend/* rm -rf pub/static/adminhtml/* bin/magento setup:static-content:deploy -f

# Clear view_process locks rm -rf var/view_preprocessed/*

# Verify directories are writable chmod -R 775 var/ chmod -R 775 pub/ chmod -R 775 generated/ chown -R www-data:www-data var/ pub/ generated/ ```

Step 5: Run Reindex with Increased Resources

Run indexer with proper memory and timeout settings.

```bash # Check current PHP memory limit php -i | grep memory_limit

# Increase memory for the command: php -d memory_limit=4G bin/magento indexer:reindex

# Or set in .user.ini or .htaccess echo "memory_limit = 4G" > .user.ini

# Run specific indexer php -d memory_limit=4G bin/magento indexer:reindex catalog_category_product

# Run with no timeout php -d memory_limit=4G -d max_execution_time=0 bin/magento indexer:reindex

# If using multiple PHP versions: /usr/bin/php8.1 -d memory_limit=4G bin/magento indexer:reindex

# Run in background for large catalogs: nohup php -d memory_limit=4G bin/magento indexer:reindex > var/log/reindex.log 2>&1 &

# Monitor progress: tail -f var/log/reindex.log

# Check running process: ps aux | grep indexer

# Run with strace to debug: strace -f -o /tmp/indexer-trace.log php bin/magento indexer:reindex catalog_category_product

# Run specific indexer dimension: bin/magento indexer:reindex catalog_category_product --dimension-mode=none

# For Magento Commerce (Cloud): php -d memory_limit=8G bin/magento indexer:reindex

# Run indexers in parallel (if safe for your configuration): bin/magento indexer:reindex catalog_category_product & bin/magento indexer:reindex catalog_product_entity & wait

# Use queue consumers for async indexing (M2.4+): bin/magento queue:consumers:start index.local.catalog.category.product bin/magento queue:consumers:start index.local.catalog.product.entity

# Check consumer status: bin/magento queue:consumers:list ```

Step 6: Reset Specific Indexers Completely

For stubborn indexers, do a complete reset and rebuild.

```bash # Reset specific indexer (invalidates and clears) bin/magento indexer:reset catalog_category_product

# Reset multiple indexers bin/magento indexer:reset catalog_category_product catalog_product_price catalogsearch_fulltext

# Reset all indexers bin/magento indexer:reset

# Verify reset bin/magento indexer:status

# Now reindex bin/magento indexer:reindex

# If reset doesn't work, truncate index tables manually # WARNING: This removes all indexed data - only use during maintenance

mysql -u magento_user -p magento_db << 'EOF' -- Backup first CREATE TABLE catalog_category_product_index_tmp AS SELECT * FROM catalog_category_product_index; -- Truncate TRUNCATE TABLE catalog_category_product_index; -- Or for fulltext TRUNCATE TABLE catalogsearch_fulltext_scope1; TRUNCATE TABLE catalogsearch_fulltext_scope2; EOF

# After truncating, run reindex bin/magento indexer:reindex catalog_category_product

# Check table sizes before/after mysql -e " SELECT table_name, ROUND(((data_length + index_length) / 1024 / 1024), 2) AS size_mb FROM information_schema.TABLES WHERE table_schema = 'magento_db' AND table_name LIKE 'catalog_%index%' ORDER BY size_mb DESC; " ```

Step 7: Fix MySQL Configuration for Large Catalogs

Optimize MySQL settings for indexing large catalogs.

```bash # Check current MySQL settings mysql -e "SHOW VARIABLES LIKE '%timeout%';" | grep -i timeout mysql -e "SHOW VARIABLES LIKE '%buffer%';" mysql -e "SHOW VARIABLES LIKE '%innodb%';" | head -20

# Common issues: # - innodb_lock_wait_timeout too low # - innodb_buffer_pool_size too small # - max_allowed_packet too small

# Edit MySQL configuration: sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

# Add or increase these settings: # For 8GB RAM server: ```

```ini [mysqld] # InnoDB settings innodb_buffer_pool_size = 4G innodb_buffer_pool_instances = 4 innodb_log_file_size = 512M innodb_log_buffer_size = 64M innodb_flush_log_at_trx_commit = 2 innodb_flush_method = O_DIRECT innodb_file_per_table = 1 innodb_lock_wait_timeout = 120

# Connection settings max_connections = 500 max_connect_errors = 1000

# Buffer sizes read_buffer_size = 2M read_rnd_buffer_size = 4M sort_buffer_size = 4M join_buffer_size = 4M

# Query cache (if enabled) query_cache_size = 64M query_cache_type = 1

# Timeout settings wait_timeout = 28800 interactive_timeout = 28800 lock_wait_timeout = 86400

# Binary log (if needed) max_allowed_packet = 256M ```

```bash # Restart MySQL sudo systemctl restart mysql

# Verify settings applied mysql -e "SHOW VARIABLES LIKE 'innodb_lock_wait_timeout';" mysql -e "SHOW VARIABLES LIKE 'innodb_buffer_pool_size';"

# For temporary changes (no restart): mysql -e "SET GLOBAL innodb_lock_wait_timeout = 120;" mysql -e "SET GLOBAL lock_wait_timeout = 86400;" ```

Step 8: Switch Indexer Mode for Debugging

Change indexer mode to isolate issues.

```bash # Check current mode bin/magento indexer:show-mode

# Switch to Update on Save (immediate indexing) bin/magento indexer:set-mode realtime catalog_category_product

# Switch to Update by Schedule (cron-based) bin/magento indexer:set-mode schedule catalog_category_product

# Switch all to schedule mode bin/magento indexer:set-mode schedule

# For debugging, use realtime mode (shows errors immediately) bin/magento indexer:set-mode realtime

# Run reindex with schedule mode bin/magento cron:run --group=index

# Check schedule table mysql -e "SELECT * FROM cron_schedule WHERE status = 'pending' LIMIT 20;" magento_db

# Clear cron schedule if stuck mysql -e "DELETE FROM cron_schedule WHERE status = 'running' AND created_at < DATE_SUB(NOW(), INTERVAL 1 HOUR);" magento_db

# Run with verbose output bin/magento indexer:reindex -v

# Check for dimension mode issues bin/magento indexer:show-dimensions-mode

# Set dimension mode bin/magento indexer:set-dimensions-mode catalog_product_price website ```

Step 9: Verify and Fix Product Data Integrity

Check for corrupted product data causing index failures.

```bash # Check for products with missing required data mysql -e " SELECT entity_id, sku, attribute_set_id, type_id FROM catalog_product_entity WHERE attribute_set_id = 0 OR type_id = ''; " magento_db

# Check for orphaned products (no website assignment) mysql -e " SELECT cpe.entity_id, cpe.sku FROM catalog_product_entity cpe LEFT JOIN catalog_product_website cpw ON cpe.entity_id = cpw.product_id WHERE cpw.product_id IS NULL; " magento_db

# Fix orphaned products (assign to default website) mysql -e " INSERT INTO catalog_product_website (product_id, website_id) SELECT entity_id, 1 FROM catalog_product_entity WHERE entity_id NOT IN (SELECT product_id FROM catalog_product_website); " magento_db

# Check for duplicate SKUs mysql -e " SELECT sku, COUNT(*) as count FROM catalog_product_entity GROUP BY sku HAVING count > 1; " magento_db

# Check category product associations mysql -e " SELECT COUNT(*) FROM catalog_category_product_index; " magento_db

# Check for NULL values in indexed columns mysql -e " SELECT COUNT(*) FROM catalog_product_entity_int WHERE value IS NULL; SELECT COUNT(*) FROM catalog_product_entity_varchar WHERE value IS NULL; SELECT COUNT(*) FROM catalog_product_entity_decimal WHERE value IS NULL; " magento_db

# Run integrity check bin/magento catalog:images:resize

# Check for url_key issues mysql -e " SELECT entity_id, sku, value FROM catalog_product_entity_varchar WHERE attribute_id = (SELECT attribute_id FROM eav_attribute WHERE attribute_code = 'url_key') AND value = ''; " magento_db ```

Step 10: Set Up Monitoring and Prevent Future Issues

Create monitoring and prevention measures.

```bash # Create indexer monitoring script cat > /usr/local/bin/monitor-magento-indexers.sh << 'EOF' #!/bin/bash # Magento 2 Indexer Monitor

MAGENTO_DIR="/var/www/magento" LOG_FILE="/var/log/magento/indexer-monitor.log" ALERT_EMAIL="ops@company.com"

cd $MAGENTO_DIR

# Check for processing indexers PROCESSING=$(bin/magento indexer:status 2>/dev/null | grep -c "Processing")

if [ "$PROCESSING" -gt 0 ]; then PROCESSING_TIME=$(mysql -N -e " SELECT TIMESTAMPDIFF(MINUTE, updated, NOW()) FROM m2_indexer_state WHERE status = 'processing' ORDER BY updated ASC LIMIT 1; " magento_db 2>/dev/null)

# If processing for more than 30 minutes, likely stuck if [ "$PROCESSING_TIME" -gt 30 ]; then echo "$(date): WARNING - $PROCESSING indexers stuck for $PROCESSING_TIME minutes" >> $LOG_FILE

# Reset stuck indexers mysql -e " UPDATE m2_indexer_state SET status = 'valid', updated = NOW() WHERE status = 'processing'; " magento_db

echo "$(date): Reset stuck indexers" >> $LOG_FILE

# Trigger reindex bin/magento indexer:reindex >> $LOG_FILE 2>&1

# Alert echo "Magento indexers were stuck and have been reset. Check logs." | \ mail -s "Magento Indexer Alert" $ALERT_EMAIL fi fi

# Check for table locks LOCKED_TABLES=$(mysql -N -e "SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA='magento_db' AND TABLE_ROWS > 0;" 2>/dev/null) LOCK_COUNT=$(mysql -N -e "SELECT COUNT(*) FROM information_schema.INNODB_LOCK_WAITS;" 2>/dev/null)

if [ "$LOCK_COUNT" -gt 0 ]; then echo "$(date): WARNING - $LOCK_COUNT table locks detected" >> $LOG_FILE fi

echo "$(date): Indexer check complete. Processing: $PROCESSING, Stuck minutes: ${PROCESSING_TIME:-0}" >> $LOG_FILE EOF

chmod +x /usr/local/bin/monitor-magento-indexers.sh

# Add to cron (crontab -l; echo "*/5 * * * * /usr/local/bin/monitor-magento-indexers.sh") | crontab -

# Create reindex cron optimization cat > /etc/cron.d/magento-indexers << 'EOF' # Run indexers every 5 minutes with staggered schedule */5 * * * * www-data cd /var/www/magento && php -d memory_limit=2G bin/magento cron:run --group=index 2>&1 | logger -t magento-index

# Full reindex daily at 2 AM 0 2 * * * www-data cd /var/www/magento && php -d memory_limit=4G bin/magento indexer:reindex 2>&1 | logger -t magento-full-reindex EOF

# Create maintenance script cat > /usr/local/bin/magento-indexer-maintenance.sh << 'EOF' #!/bin/bash # Weekly Magento Indexer Maintenance

MAGENTO_DIR="/var/www/magento" cd $MAGENTO_DIR

echo "$(date): Starting weekly indexer maintenance" >> /var/log/magento/indexer-maintenance.log

# Clear old cron schedules mysql -e "DELETE FROM cron_schedule WHERE created_at < DATE_SUB(NOW(), INTERVAL 7 DAY);" magento_db

# Reset any stuck indexers mysql -e "UPDATE m2_indexer_state SET status = 'valid' WHERE status IN ('processing', 'working');" magento_db mysql -e "UPDATE mview_state SET status = 'idle' WHERE status = 'working';" magento_db

# Clear cache bin/magento cache:flush >> /var/log/magento/indexer-maintenance.log 2>&1

# Reindex php -d memory_limit=4G bin/magento indexer:reindex >> /var/log/magento/indexer-maintenance.log 2>&1

# Clear cache again bin/magento cache:flush >> /var/log/magento/indexer-maintenance.log 2>&1

echo "$(date): Maintenance complete" >> /var/log/magento/indexer-maintenance.log EOF

chmod +x /usr/local/bin/magento-indexer-maintenance.sh

# Add weekly cron (crontab -l; echo "0 3 * * 0 /usr/local/bin/magento-indexer-maintenance.sh") | crontab - ```

Checklist for Fixing Magento 2 Indexer Issues

Verify the Fix

After fixing Magento indexer issues, verify everything works:

```bash # 1. All indexers show valid status bin/magento indexer:status # All should show "Valid" or "Reindex required" (not Processing)

# 2. Products appear in search # Test search on frontend curl -s "https://yourstore.com/catalogsearch/result/?q=test" | grep -c "product-item"

# 3. Category pages show products curl -s "https://yourstore.com/category-page.html" | grep -c "product-item"

# 4. Reindex completes without hanging time bin/magento indexer:reindex # Should complete in reasonable time

# 5. No database locks mysql -e "SHOW OPEN TABLES WHERE In_use > 0;" magento_db # Should return empty result

# 6. No long-running queries mysql -e "SELECT * FROM information_schema.PROCESSLIST WHERE Time > 60;" magento_db # Should return empty

# 7. Cron jobs running bin/magento cron:run --group=index # Should complete without errors

# 8. Products count matches mysql -e "SELECT COUNT(*) FROM catalog_product_entity;" magento_db mysql -e "SELECT COUNT(*) FROM catalog_category_product_index;" magento_db # Numbers should be reasonable

# 9. Frontend search returns results # Test various search terms

# 10. Monitoring script runs successfully /usr/local/bin/monitor-magento-indexers.sh # Exit code 0 ```

Related Issues

• [Fix Symfony Cache Corrupted](/articles/fix-symfony-cache-corrupted) - Symfony cache issues
• [Fix Drupal Module Update Failed](/articles/fix-drupal-module-update-failed) - Drupal update problems
• [Fix Laravel Migration Locked](/articles/fix-laravel-migration-locked) - Laravel database locks
• [Fix PHP Memory Limit Exhausted](/articles/fix-php-memory-limit-exhausted) - PHP memory issues
• [Fix MySQL Lock Wait Timeout](/articles/fix-mysql-lock-wait-timeout) - MySQL lock issues
• [Fix ElasticSearch Index Not Updating](/articles/fix-elasticsearch-index-not-updating) - Elasticsearch index problems
• [Fix Magento 2 Cron Not Running](/articles/fix-magento-2-cron-not-running) - Magento cron issues