Amazon goes Queuing

A colleague of mine passed on a surprise announcement from Amazon: They’ve just launched the v1.0 beta of their Simple Queue Service. It is an online queuing service with some very simple SOAP and REST-style APIs for message queuing.

For the beta, they’ve limited message size to 4KB, and a maximum of 4000 messages across all queues for a single subscription ID. According to the FAQ, messages are kept for up to 30 days.

The API is quite simple – methods for creating, updating, deleting and querying queues, as well as three methods for queue interaction: Enqueue, Read, and Dequeue.

The messaging read process is somewhat intriguing. A client application issues a ‘read’ request against the queue. Pending message(s) are returned (you can request more than one message in a read request) in response to the request, and a lock is placed on those messages. This means that successive read requests within the lock period will not get the same series of messages. The client application then explicitly dequeues the messages by issuing a ‘dequeue’ request against the queue, supplying the QueueEntryId for each message to remove.

One interesting caveat is mentioned in the documentation for the Read operation:

A message may be returned by the Read operation even though it has already been dequeued, and concurrent Read calls may return the same message to multiple readers. This behavior is a result of prioritizing reliable data storage (even in the face of hardware failures), and we expect such events to occur very infrequently.

It seems that SQS is optimized for reliability, but not for once-and-only-once delivery.

Applications utilizing SQS would need to ensure that they handle duplicate scenarios. The documentation basically implies that the appropriate solution is to ensure that messages are idempotent; retrieving the same message twice should have the same effect as receiving it once.

Currently security is pretty lax. One only needs to know the Subscriber ID to be able to retrieve a list of queues, and from there it would be possible to retrieve messages from those queues. I’m sure that this is something that Amazon would address as they further enhance the service. A simple password-based mechanism would take the security up a level, and, in the future, certificate-based mechanisms could be employed.

With this foray into more ‘Enterprise Level’ services, it does make me wonder where Amazon is headed. They’ve certainly mastered the eCommerce world, and have provided some interesting Web Services focused on that domain. Now it seems that they’re venturing out into some more horizontally focused services, and exposing some of their platform-level capabilities. I wonder what’s coming next…

This entry was posted in Web Services. Bookmark the permalink.

4 Responses to Amazon goes Queuing

  1. Daniel says:

    Hmmm…verrry interesting. I appreciate your conjectures as to why, but I still can’t figure out why, like, really…just a public exposure of an in-house thing, right? “Hey we have this queuing thing, let’s make it public.” “Uh, ok!” My main ? is “Who’s going to use this?” Secondary is “Who’s going to provide services?” One’s chance to become “the premier provider of SQS consulting?”

  2. Jason says:

    It’s certainly in the realm of ‘technology preview.’ I’m waiting to see what use-cases or demos Amazon will provide for this feature. The current samples provide a C#-based SQS API exerciser, and an XSLT example that is really using the queue as a collection container.

  3. joe jackson says:

    weird. how does one obtain a subscription id?

  4. Jason says:

    You’ll need to register at Amazon’s Web Services site in order to obtain an AWS Subscription ID.
    Just go to the ‘Register for AWS’ link in the top left of this page:
    http://www.amazon.com/gp/browse.html/102-0146935-4245750?node=3435361

Comments are closed.