Robust Document Processing: Handling Unsupported Formats with AWS Textract and Lambda

In the justifai project, our document processing pipeline relies heavily on AWS Textract for extracting information from various file types. However, a critical issue emerged: when Textract encountered an unsupported document format, such as a PDF that Textract's synchronous API couldn't process, our AWS Lambda function would fail. This led to multiple retries, unnecessary resource consumption, and eventually, CloudWatch alarms that obscured genuine operational issues.

The Problem: Unhandled Exceptions

The core of the problem lay in how our Lambda function handled UnsupportedDocumentException from Textract. Instead of gracefully managing these cases, the exception would cause the Lambda to crash, triggering the retry mechanism. This not only consumed compute cycles but also delayed the processing of other valid documents and created noise in our monitoring systems.

try {
  const textractResult = await textractClient.analyzeDocument({
    // ... textract parameters
  }).promise();
  // Process successful result
} catch (error) {
  // If not explicitly handled, this error would crash the Lambda
  throw error; // Leads to retries and alarms for unsupported formats
}

The Solution: Graceful Degradation and Centralized Logic

To address this, we implemented a two-pronged approach:

1. Exception Handling for Unsupported Formats: We modified the Lambda to specifically catch UnsupportedDocumentException. Instead of rethrowing, we now classify such documents for REVIEW. This means the document is marked for manual human review, ensuring no data is lost and the automated pipeline continues without interruption. True operational errors continue to be rethrown, maintaining the integrity of our alarm system.

2. Centralized Persistence and Notification: To avoid code duplication and ensure consistent behavior, we extracted the logic for persisting document status to DynamoDB and sending notifications via SQS into a shared helper function. This helper is now utilized by both the successful OCR path and the new unsupported-format path. This ensures that regardless of the processing outcome, the document's status is updated, and downstream services are notified.

This approach ensures that while synchronous Textract might not always support a specific format, our system remains robust. Documents are never silently dropped; they are either fully processed or flagged for manual intervention.

// Simplified helper function
const updateAndNotify = async (documentId, status, details) => {
  await dynamoDbClient.updateItem({
    TableName: 'DocumentStatus',
    Key: { id: documentId },
    UpdateExpression: 'SET #status = :s, #details = :d',
    ExpressionAttributeNames: { '#status': 'status', '#details': 'details' },
    ExpressionAttributeValues: { ':s': status, ':d': details }
  }).promise();

  await sqsClient.sendMessage({
    QueueUrl: 'NotificationQueueUrl',
    MessageBody: JSON.stringify({ documentId, status })
  }).promise();
};

// Inside the Lambda handler
try {
  const textractResult = await textractClient.analyzeDocument({
    // ... textract parameters
  }).promise();
  await updateAndNotify(document.id, 'PROCESSED', textractResult);
} catch (error) {
  if (error.code === 'UnsupportedDocumentException') {
    console.warn(`Unsupported document format for ${document.id}. Flagging for review.`);
    await updateAndNotify(document.id, 'REVIEW', { reason: 'Unsupported Format' });
  } else {
    console.error(`Processing error for ${document.id}:`, error);
    throw error; // Re-throw other errors for retry/alarm
  }
}

The Takeaway

Building resilient serverless applications requires meticulous error handling, especially when integrating with external services like AWS Textract. By anticipating potential failures, implementing graceful degradation strategies, and centralizing side-effect logic (like database persistence and notifications), you can significantly improve system stability, reduce operational overhead, and provide clearer visibility into exceptional cases. Always consider how external service limitations impact your workflow and design specific recovery paths.

Generated with Gitvlg.com