AWS SQS

5 min

Write events directly to an Amberflo supplied and secured AWS SQS queue

Amberflo provides your account with a FIFO (First-In-First-Out) AWS SQS queue that includes the necessary access rights and permissions to submit meter records.

Meter records are automatically processed as they arrive in the queue.
No additional configuration is needed on your end for ingestion.

To get your SQS queue and S3 bucket provisioned, please contact us.

Format

The meter records sent to the SQS queue must follow the same standardized format as used by the Amberflo Ingest API. Here are some examples:

[{
     "customerId": "customer-123",
     "meterApiName": "ComputeHours",
     "meterValue": 5,
     "meterTimeInMillis": 1619445706909,
     "dimensions": {
        "region": "us-west-2",
        "az": "az1"
     }
}]

We also support NDJSON format (JSON separated by a newline)

{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 5, "meterTimeInMillis": 1619445706909 }
{ "customerId": "customer-321", "meterApiName": "ComputeHours", "meterValue": 4, "meterTimeInMillis": 1619445712341 }
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 1, "meterTimeInMillis": 1619445783456 }

Code Example

import json
from uuid import uuid1
from datetime import date

import boto3

records_to_send = [{
     'customerId': 'customer-123',
     'meterApiName': 'ComputeHours',
     'meterValue': 5,
     'meterTimeInMillis': 1619445706909,
     'dimensions': {
        'region': 'us-west-2',
        'az': 'az1'
     }
}]

queue_url = 'https://sqs.us-west-2.amazonaws.com/624335419252/62-ingest.fifo'

sqs = boto3.client('sqs')

sqs.send_message(
    QueueUrl=queue_url,
    MessageBody=json.dumps(records_to_send),
    MessageGroupId=str(uuid1()),
    MessageDeduplicationId=str(uuid1()),
)

Troubleshooting

Meter records sent to the SQS queue are first saved in an S3 bucket before being processed. (See the AWS S3 integration.)

If there's a failure during processing:

A failure report file will be created in the S3 bucket with the reason for the error.
A dead-letter queue is also set up to capture messages that could not be written to S3.

This setup ensures reliable troubleshooting and traceability for all ingestion issues.