Skip to main content

CloudWatch Integration Guide

Integrate MCP Express with AWS CloudWatch to build custom tools that can seamlessly query and interact with your log data. This powerful integration enables you to execute flexible log queries against your CloudWatch log groups, retrieve relevant log entries, and use the results in your MCP tools..

Configuration​

Connection Parameters​

ParameterRequiredDescription
RegionYesAWS region where your CloudWatch logs are stored (e.g., us-east-1, eu-west-1)
AWS Access KeyYesAWS access key ID for authentication
AWS Secret KeyYesAWS secret access key for authentication
Log GroupYesThe log group name to search logs in (supports templating)
Start TimeYesISO Time UTC to start the search from (supports templating)
End TimeYesISO Time UTC till when the logs are searched (supports templating)
QueryYesCloudWatch Insights query to search and filter logs (supports templating)

Setting Up CloudWatch Integration​

  1. Select CloudWatch Integration: In your MCP server dashboard, choose "CloudWatch" from the available integrations
  2. Configure AWS Region: Select the AWS region where your CloudWatch logs are stored
  3. Set AWS Credentials: Enter your AWS Access Key and AWS Secret Key (these values are kept confidential)
  4. Define Log Group: Enter the log group name or use templating for dynamic log group selection
  5. Set Time Range: Specify the start and end time for log searches using ISO Time UTC format
  6. Create Query Template: Define your CloudWatch Insights query using templating for dynamic parameters
  7. Test Connection: Use the built-in connection test to verify your setup works correctly

CloudWatch Configuration

Query Templates​

The CloudWatch integration uses templating to create dynamic log queries based on tool inputs. This allows you to safely parameterize your log searches while maintaining flexibility. Multiple fields support templating: Log Group, Start Time, End Time, and Query.

Template Syntax​

Variables from tool inputs can be referenced using {{ variable_name }} syntax:

Example:

/aws/lambda/{{functionName}}

Example Usages​

For all examples below, we assume the tool accepts the following input parameters based on the input schema:

Example CloudWatch Input

  • functionName (string): Function name of log group
  • startTime (string): Start time of logs
  • endTime (string): End time of logs
  • limit (string): Limits of logs

Static Queries (No Input Parameters Required)​

The CloudWatch integration can execute fixed queries, useful for monitoring specific log groups. For example, searching for errors in a specific Lambda function:

Log Group:

/aws/lambda/my-production-function

Query:

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 100

Start Time:

2025-11-10T00:00:00Z

End Time:

2025-11-10T23:59:59Z

Dynamic Template Queries​

The CloudWatch integration supports parameterized queries that accept user input. For example, searching logs for a specific function name and time range:

Log Group:

/aws/lambda/{{functionName}}

Start Time:

{{startTime}}

End Time:

{{endTime}}

Query:

fields @timestamp, @message, @logStream
| sort @timestamp desc
| limit {{limit}}

Advanced CloudWatch Insights Queries​

The integration supports complex CloudWatch Insights query syntax for sophisticated log analysis:

Query:

fields @timestamp, @message, @requestId, @duration, @billedDuration, @memorySize
| filter @type = "REPORT"
| stats avg(@duration), max(@duration), min(@duration) by bin(5m)

CloudWatch Logs Insights Query Syntax​

For detailed information about writing CloudWatch Logs Insights queries, refer to the official AWS documentation:

Common Query Commands​

  • fields - Select specific fields to display in results
  • filter - Filter log events based on conditions
  • stats - Calculate aggregate statistics
  • sort - Sort results by specified fields
  • limit - Limit the number of results returned
  • parse - Extract data from log fields using patterns

Useful Query Functions​

  • like - Pattern matching with wildcards
  • =~ and !~ - Regular expression matching
  • avg(), sum(), min(), max(), count() - Aggregation functions
  • earliest(), latest() - Time-based functions

For comprehensive examples and advanced query techniques, visit the AWS CloudWatch Logs Insights documentation.

Security Features​

AWS Credentials Protection​

The CloudWatch integration securely handles AWS credentials:

  • AWS Access Keys and Secret Keys are encrypted and never displayed in plain text after creation
  • Credentials are transmitted securely and stored with encryption at rest
  • Use IAM users with minimal required permissions for enhanced security

CloudWatch Query Limits​

While CloudWatch doesn't have "dangerous keywords" like SQL databases, you should be aware of query constraints:

  • Queries that return large result sets can be expensive and slow
  • Always use limit clauses to control result set sizes
  • Be cautious with queries that scan long time ranges
  • Monitor CloudWatch Logs Insights query costs

Best Practices for Query Templates​

  1. Use parameterized queries with variable placeholders for dynamic values
  2. Always specify time ranges to limit query scope and costs
  3. Use appropriate limit values to prevent excessive data retrieval
  4. Validate user inputs before they reach the query template
  5. Use IAM policies to restrict CloudWatch access to specific log groups
  6. Implement rate limiting for user-initiated queries to control costs

Common Use Cases​

1. Application Monitoring​

Search and analyze application logs for insights:

  • Monitor Lambda function errors and exceptions
  • Track API Gateway request patterns and response times
  • Analyze ECS container logs for application issues
  • Identify performance bottlenecks from application metrics

2. Troubleshooting and Debugging​

Debug production issues efficiently:

  • Search for specific error messages or stack traces
  • Filter logs by request ID to trace request flows
  • Analyze log patterns around specific timestamps
  • Identify correlation between different service logs

3. Security and Compliance​

Monitor security events and maintain compliance:

  • Search for unauthorized access attempts
  • Track authentication failures and security events
  • Monitor API calls for suspicious activity
  • Generate audit reports from CloudWatch logs

4. Performance Analysis​

Analyze system and application performance:

  • Calculate average response times and latencies
  • Identify slow queries or operations
  • Monitor resource utilization patterns
  • Track cold start frequencies for serverless functions

Best Practices​

Security​

  • Use IAM roles with minimal required permissions following the principle of least privilege
  • Create dedicated IAM users for MCP CloudWatch integration rather than using root credentials
  • Enable CloudTrail to audit CloudWatch API calls made by the integration
  • Rotate AWS access keys regularly according to your security policy
  • Use AWS Secrets Manager or Parameter Store for credential management in production environments
  • Restrict log group access using IAM policies to prevent unauthorized data access

Performance​

  • Always specify time ranges to limit query scope and reduce costs
  • Use appropriate limit clauses to control result set sizes
  • Optimize queries by filtering early in the query pipeline
  • Use specific log groups rather than searching multiple groups simultaneously
  • Consider CloudWatch data retention policies to manage costs
  • Cache frequently accessed query results when appropriate

Reliability​

  • Implement proper error handling for connection failures and timeout issues
  • Test all query templates thoroughly before deployment to production
  • Monitor CloudWatch Logs Insights query execution times and optimize slow queries
  • Set reasonable timeout values based on expected query complexity
  • Use exponential backoff for retrying failed queries
  • Implement query result pagination for large datasets

Additional Resources​

AWS Documentation​

Useful Tools​

Troubleshooting​

Connection Issues​

  • Verify the AWS region is correct and matches your log group location
  • Check that AWS credentials (Access Key and Secret Key) are valid and not expired
  • Ensure the IAM user or role has logs:StartQuery and logs:GetQueryResults permissions
  • Verify network connectivity to AWS CloudWatch service endpoints
  • Check if any VPC endpoints or network policies are blocking CloudWatch access
  • Confirm that the specified log group exists in the configured region

Authentication Problems​

  • Confirm the AWS Access Key ID and Secret Access Key are correct
  • Verify the IAM user has not been deleted or disabled
  • Check IAM policy attachments for required CloudWatch permissions
  • Ensure no Service Control Policies (SCPs) are blocking CloudWatch access
  • Verify MFA requirements are met if enforced on the IAM user
  • Check for any temporary security credentials that may have expired

Query Execution Issues​

  • Review the CloudWatch Insights query syntax for formatting errors
  • Verify all template variables are provided by the tool input schema
  • Check if the time range (Start Time to End Time) is valid and not too large
  • Ensure the log group name is correctly formatted and exists
  • Verify the query doesn't exceed CloudWatch Insights query size limits
  • Check CloudWatch service quotas for concurrent queries and data scanned
  • Review CloudWatch Logs Insights query results for error messages or warnings
  • Confirm the time range contains log data (logs may not exist for the specified period)

Permission Errors​

  • Ensure IAM policy includes logs:DescribeLogGroups permission
  • Verify logs:FilterLogEvents permission if using filter queries
  • Check that resource-level permissions allow access to specific log groups
  • Confirm IAM policy doesn't have explicit deny statements for CloudWatch actions
  • Review CloudWatch resource-based policies if applicable

Performance Issues​

  • Reduce the time range to scan less log data
  • Add more specific filters earlier in the query pipeline
  • Increase the limit parameter gradually to find optimal result sizes
  • Consider using CloudWatch Logs Insights' aggregation functions for large datasets
  • Check AWS service health dashboard for any CloudWatch service disruptions
  • Monitor query execution times and optimize queries that consistently run slow