The Amazon Simple Queue Service (SQS) is a highly scalable, reliable and elastic queuing service that ‘just works’. Customers from various verticals (media, social gaming, mobile, news, advertisement) such as Netflix, Shazam and Scopely have used SQS in variety of use-cases requiring loose coupling and high performance. For example, AWS customers use SQS for asynchronous communication pipelines, buffer queues for databases, asynchronous work queues, and moving latency out of highly responsive requests paths.

Today, the SQS team is launching two important features – Long Polling and richer client functionality in the SQS SDK – that we believe will extend the reach of SQS to new use cases by reducing the cost of high scale messaging for our customers.

Long polling reduces extraneous polling to help you receive new messages as quickly as possible. Customers tell us they poll SQS quickly because they want to retrieve messages as soon as they become available. But when the message rate fluctuates, it produces empty receives, meaning extra work and extra cost.

With long polling, SQS instead waits for a message to become available and sends it to the client if the message arrives within a customer-defined time period. By reducing extraneous polling, we expect this feature to lower the cost of using SQS for any given volume of messages, while still delivering messages quickly for customers who would otherwise poll their queues rapidly. It can eliminate the need for ‘back-off algorithms’ that dynamically adjust SQS polling frequencies.

In addition to Long Polling, we are also launching richer client functionality in the Java SDK. This rich client extends the existing AmazonSQSAsyncClient interface to provide batching of outgoing messages, and also pre-fetching of incoming messages. Rich client also uses long-polling under the hood.

When rich client receives a new outgoing message, it waits a short, configurable period of time to see if other outgoing messages arrive. If they do, they are added to the buffer. This enables your applications to take advantage of batch pricing more easily, without custom development. Rich client can prefetch batches of incoming messages, so that your application can process the new messages immediately once it’s through with a current batch. Similar to long polling, we expect rich client to improve the performance of SQS for our customers while decreasing their costs.

Historically, messaging has been an important building block for building highly reliable distributed systems. Within Amazon’s e-commerce platform, messaging systems have always been a key part of our service-oriented architecture to build an asynchronous communication pipeline between different services. Today, SQS is a key part of this architecture and is used in mission critical backend systems for a myriad of use-cases in the Kindle platform, Amazon Retail Ordering workflow, Amazon Fulfillment technologies, etc.

Similarly, AWS customers have been using SQS in interesting ways. For example, Netflix uses SQS for a variety of use-cases such as monitoring and encoding workflows. Netflix’s Chief Architect Adrian Cockcroft blogged “Simple Queue Service (SQS) is very useful, easy to use, scalable and reliable. We ran our own message queue service in our datacenter for many years, and it wasn’t a happy experience. I’m very glad it’s someone else’s problem, and we use SQS heavily in our architecture.”

Shazam, developers of the mobile discovery app, report using SQS as a buffer for DynamoDB. Amazon SQS provides highly scalable ‘eventual throughput’ for cases when Shazam’s message rate exceeds the throughput they provision for DynamoDB.

Scopely, the social mobile games developer, buffers most operations via SQS to maximize performance for gamers who may play in very short bursts of activity. “The absolute minimum of activity happens synchronously,” reports Scopely CTO Ankur Bulsara, “everything else happens via SQS – asynchronously, but still very quickly.

Higher performance at lower cost means customers can use SQS for even more demanding use cases. So we’ve worked hard to ensure these new features are as easy to adopt as possible. For more information on the use of Long Polling and Rich Client, please see the appropriate topics in the Amazon SQS Developer Guide.

If you have an interesting SQS use case that you’d like to tell us about, please let me know in the comments below.For more information visit the SQS detail page.