HOW NASA USES AWS SQS
Amazon Web Services are leading the Cloud market with nearly 40% of the market with them. AWS provides over 200 services, thus AWS is capable of providing unified solutions for most companies. One of the many services provided by AWS for Message Queue is AWS SQS
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
What is a Messaging Queue Service?
A message queue is a form of asynchronous service-to-service communication used in serverless and microservices architectures. Messages are stored on the queue until they are processed and deleted. Each message is processed only once, by a single consumer. Message queues can be used to decouple heavyweight processing, to buffer or batch work, and to smooth spiky workloads.
Note: The size of the message data is a minimum 1Kb and a maximum of 256Kb.
Why Do we need a Messaging Queue?
Consider the above scenario of Order confirmation and mailing the customer an acknowledgment of the confirmed order. If a consumer buys an item, the records are to be stored in the Database and to be sent to a mail server to get the confirmation of the order via email.
In the tightly coupled architecture (The red path), if the server goes down there would be a loss of data transmitted at the moment. A tightly coupled system is a concept of system design and computing where every hardware and software component are linked together in such a manner that each component is dependent upon the other. So if the customer confirms the order and if the database server is down (even for a fraction of a second) the data of the order confirmation would not be recorded in the system but the email for the order confirmation would be received by the customer. This is inadmissible.
Whereas, a loosely coupled system is one in which each of its components has, or makes use of, little or no knowledge of the definitions of other separate components. In the above architecture, the green path follows a loosely coupled system architecture. Amazon SQS is a component of the loosely coupled architecture. When the customer confirms the order the data is sent to the Message Queue. The Message Queue keeps the data until the data is been received by the servers in a specific order if the servers go down due to some reason. The eliminates the potential data loss as compared to a tightly coupled architecture. System Components and Services like Amazon SQS are termed as middleware.
Amazon SQS
Amazon SQS is a distributed queuing webservice for exchanging messages between software systems asynchronously. SQS is used to develop large-scale distributed systems with decoupled components without worrying about creating and managing the queue.
SQS comes in two modes: Standard and FIFO (First-In-First-Out).
FIFO queue maintains the order of the messages that are produced and consumed providing a once-only delivery mechanism. Use Case: guaranteed order, once only delivery.
Standard queue attempts to preserve the order but is flexible, and it provides at least once delivery. The message in the Standard queue can be delivered more than once and standard Queue offers high throughput. Use Case: Faster messaging, can be duplicates, messages sent out of order.
General Use Case: One to one messaging, sending emails, form and file processing.
Message Availability Limit: By default, the messages in SQS will be deleted after 4 days, but this can be extended up to 14 days
Amazon SQS provides common middleware constructs such as dead-letter queues and poison-pill management.
A dead letter queue is a queue that other (source) queues can target for messages that can’t be processed successfully. You can set aside and isolate these messages in the dead letter queue to determine why their processing did not succeed. This feature makes troubleshooting simple.
Poison pills are special messages that can be received, but not processed. They are a mechanism used in order to signal a consumer to end its work so it is no longer waiting for new inputs.
How Nasa uses Amazon SQS
“We now have an agile, scalable foundation on which to do all kinds of amazing things. Much like with the exploration of space, we’re just starting to imagine all that we can do with it.” — Bryan WallsImagery Experts Deputy Program Manager, NASA
The NASA Image and Video Library is a cloud-native solution, with the front-end web app separated from the backend API. It runs as immutable infrastructure in a fully automated environment, with all infrastructure defined in code to support continuous integration and continuous deployment (CI/CD).
All components of the NASA Image and Video Library are built to scale on-demand, as needed to handle usage spikes. Amazon Simple Queue Service (SQS), which is used to decouple incoming jobs from pipeline processes at Nasa. Notification service helps to trigger the events when new data come up from the developer side at the server-side.
Thank You All for your Time!