🎉 DevOps Interview Prep Bundle is live — 1000+ Q&A across 20 topicsGet it →
All Articles

AWS SQS Message Not Being Consumed: Fix in 5 Minutes

Messages sitting in an SQS queue forever, never picked up by your consumer? Here is exactly how to diagnose visibility timeout misconfiguration, IAM permission gaps, dead-letter queue redirects, and consumer polling issues.

Shubham4 min read
Share:Tweet

A message sitting in SQS with ApproximateNumberOfMessages stuck above zero means something specific is blocking consumption — SQS itself is almost never the actual bug, the consumer side or configuration around it usually is.

Step 1: Confirm Where the Message Actually Is

bash
aws sqs get-queue-attributes --queue-url $QUEUE_URL \
  --attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible
 
# ApproximateNumberOfMessages: 42          ← visible, waiting to be consumed
# ApproximateNumberOfMessagesNotVisible: 8  ← currently "in flight" (being processed or stuck in visibility timeout)

If messages are stuck in NotVisible rather than Messages, they were already picked up by a consumer that never deleted or released them — a different problem than "never consumed at all."

Cause 1: Consumer Isn't Actually Polling (IAM Permission or Wrong Queue URL)

bash
# Check if your consumer even has permission to receive messages
aws sqs receive-message --queue-url $QUEUE_URL --max-number-of-messages 1
# If this fails for you manually, your consumer's IAM role likely has the same problem
json
// Required IAM permissions for a consumer — commonly people grant
// SendMessage but forget ReceiveMessage/DeleteMessage
{
  "Effect": "Allow",
  "Action": ["sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes"],
  "Resource": "arn:aws:sqs:us-east-1:123456789012:my-queue"
}

Fix — verify the consumer's actual IAM role has all three, and confirm it's polling the correct queue URL (not confusing dev/staging/prod queue URLs, a very common copy-paste mistake):

bash
aws sts get-caller-identity    # confirm you're checking the right identity
aws iam simulate-principal-policy \
  --policy-source-arn arn:aws:iam::123456789012:role/consumer-role \
  --action-names sqs:ReceiveMessage \
  --resource-arns arn:aws:sqs:us-east-1:123456789012:my-queue

Cause 2: Visibility Timeout Too Short for Processing Time

bash
aws sqs get-queue-attributes --queue-url $QUEUE_URL --attribute-names VisibilityTimeout
# VisibilityTimeout: 30    ← if your consumer takes longer than 30s to process,
#                             the message becomes visible again mid-processing

If processing takes longer than the visibility timeout, another consumer instance picks up the SAME message while the first is still working on it — this often looks like "messages aren't being consumed" when actually they're being processed repeatedly and never successfully deleted because processing never completes before the timeout expires.

bash
# Check your consumer's actual average processing time against this

Fix — set visibility timeout to comfortably exceed your consumer's max processing time (AWS recommends 6x your average, at minimum):

bash
aws sqs set-queue-attributes --queue-url $QUEUE_URL \
  --attributes VisibilityTimeout=300    # if processing genuinely takes up to ~1-2 minutes

For long-running processing, use ChangeMessageVisibility to extend the timeout dynamically instead of just setting one large fixed value:

python
import boto3
sqs = boto3.client('sqs')
 
sqs.change_message_visibility(
    QueueUrl=queue_url,
    ReceiptHandle=receipt_handle,
    VisibilityTimeout=120    # Extend while still processing, before the original timeout expires
)

Cause 3: Messages Silently Redirected to Dead-Letter Queue

bash
aws sqs get-queue-attributes --queue-url $QUEUE_URL \
  --attribute-names RedrivePolicy
# {"deadLetterTargetArn":"arn:aws:sqs:...:my-queue-dlq","maxReceiveCount":"3"}
 
# Check if messages are actually landing in the DLQ instead
aws sqs get-queue-attributes --queue-url $DLQ_URL --attribute-names ApproximateNumberOfMessages

If your consumer receives a message, fails to process it (throws an exception without deleting it), the message becomes visible again — after maxReceiveCount failed attempts, SQS moves it to the DLQ automatically. If nobody's monitoring the DLQ, this looks exactly like "messages disappearing" from the main queue's perspective.

Fix — check the DLQ and fix the underlying processing error, then optionally redrive messages back:

bash
# Peek at what's actually in the DLQ to understand the failure
aws sqs receive-message --queue-url $DLQ_URL --max-number-of-messages 1
 
# Once the consumer bug is fixed, move messages back to the main queue
aws sqs start-message-move-task --source-arn arn:aws:sqs:...:my-queue-dlq

Set up a CloudWatch alarm on the DLQ's message count — this is the fix that prevents "silently disappearing messages" from happening unnoticed again.

Cause 4: Long Polling Not Configured, Consumer Missing Messages Between Polls

bash
aws sqs get-queue-attributes --queue-url $QUEUE_URL --attribute-names ReceiveMessageWaitTimeSeconds
# ReceiveMessageWaitTimeSeconds: 0    ← short polling, can miss messages if
#                                        your consumer's poll interval is too spaced out

Short polling (0s wait) only checks a subset of servers per request and returns immediately even if no message is found — combined with an infrequent polling loop, messages can sit for longer than expected between poll attempts, or occasionally get missed by a given poll due to SQS's distributed architecture.

Fix — enable long polling:

bash
aws sqs set-queue-attributes --queue-url $QUEUE_URL \
  --attributes ReceiveMessageWaitTimeSeconds=20    # Max value, reduces empty responses and missed messages

Cause 5: Consumer Application Crashed or Scaled to Zero

bash
# The simplest cause, worth checking first before anything queue-side
kubectl get pods -n production -l app=sqs-consumer
# Or for Lambda-based consumers:
aws lambda get-function --function-name my-sqs-consumer --query 'Configuration.State'

Fix — obvious once found, but worth checking early:

bash
kubectl rollout restart deployment/sqs-consumer -n production
# Or check Lambda concurrency limits/throttling if using Lambda as the consumer
aws lambda get-function-concurrency --function-name my-sqs-consumer

Diagnostic Checklist

bash
aws sqs get-queue-attributes --queue-url $QUEUE_URL --attribute-names All
kubectl get pods -n production -l app=sqs-consumer                 # is the consumer even running
aws sqs get-queue-attributes --queue-url $DLQ_URL --attribute-names ApproximateNumberOfMessages
aws iam simulate-principal-policy --policy-source-arn ... --action-names sqs:ReceiveMessage,sqs:DeleteMessage

More AWS troubleshooting? Read our AWS Lambda timeout error fix and AWS IAM AssumeRole access denied fix.

🔧

Today I Fixed

Short real fixes from production — posted daily

Browse fixes
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments