How Lambda@Edge and AWS Greengrass Enable Functions on *Anything* on the IoT
In the past few years, serverless has been one of the hottest topics in public cloud technologies, and AWS Lambda is the absolute flagship product among the different service offerings.
Since its launch, AWS have kept adding useful features and integrations to promote wider adoption of serverless technologies. Every new feature adds great value in terms of tackling security, operations and deployment challenges. Two of the announcements at 2016 re:Invent were absolute corkers in my opinion: Lambda@Edge and Greengrass.
Why are these so awesome? Historically, Lambda functions could only run in AWS cloud. No longer. Lambda@Edge enables AWS users to run Lambda functions on Cloudfront edge locations and Greengrass enables Internet of Things devices to run Lambda functions locally.
This combination means that Lambda can be run on pretty much anything, be it a mobile phone, smart TV (or toilet?), or a bog-standard server! Brilliant!
How does it work?
Greengrass is an extension of AWS IoT Device SDK, and is intended to enable field devices with intermittent network connectivity to communicate with each other and AWS’s cloud.
Historically with AWS IoT, devices did not have a standard message queue for temporarily storing messages in the case of an internet outage. Now, however, a device running a Greengrass Core can act as a local hub for the other devices so that, if there is no internet connection to AWS, the local hub can still discover, securely communicate, process and store device data via the local network on its local CPU and storage. Once connectivity to AWS has been restored, messages persisted in the local message queue can be sent to the cloud for long-term storage and analytics. Lambda functions, being event driven, can be very usefully deployed on the device hub for processing events from other devices. Greengrass Core devices can also follow the state of field devices by the device shadow, so the device’s desired state is known by the hub if the device is restarted or reset for any reason.
A Greengrass Group consists of one or more Greengrass Core devices and other field devices, which could represent the floor of a building, a vehicle or a home. Since the Core devices can discover other devices - and keep track of other’s state in the device shadow - local hubs can be designed to run with high availability to prevent any loss of field device data.
Since bandwidth can be one limitation on sending masses of data from field devices, local processing and filtering of datasets can greatly increase the efficiency of communications between Edge locations and the cloud. Once the messages have been stripped down as much as possible, they can be sent to AWS S3 data lake for further cleansing, data mining and big data analytics by cloud services.
Testing and releasing
Just like for any other software, we need to think about the testing and release process for applications running on field devices. Luckily, the Greengrass Core functions will be standard Lambda functions, which can be tested easily in AWS. We can build continuous integration and delivery (CI/CD) pipelines using the standard AWS building blocks to automate the testing and release process for our Greengrass Core code changes.
As usual, code stored and managed in CodeCommit or Github is a good starting point for a pipeline. We can compose CodePipeline steps to read our code from Git, build it in CodeBuild, preferably alongside some unit tests, and run some more functional and integrations tests before deploying to the field devices. You can easily test the functions to be run on the Edge by deploying them to AWS and giving them some test workloads.
When it comes to integration and full end-to-end system testing of your IoT applications, you can launch an AWS test environment for the components hosted in the cloud, while Greengrass functions can also be deployed to AWS for testing. At this point, it’s only the edge devices or sensors that are missing from the testing. Here, you can build a simulation of your field devices that are sending data to your Greengrass Core back to AWS ecosystem - full end-to-end testing in a serverless fashion, with Lambda!
Finally, you can deploy the code changes, if all tests passed, with AWS CodeDeploy, as the last step of your CodePipeline. CodeDeploy can roll out your changes to many target devices with different deployment strategies.
I’ve had a play with the Cloudformation template from this AWS blog post from last year, it's a great example of simulating sensors for IoT applications in AWS Lambda.
Adoption from small startups to the enterprise
Today, we can see rapid innovation in the space of smart appliances. Whether they are operating in our homes, in vehicles or in the field, these devices aim to optimise efficiency in spaces which they inhabit.
There are many small startups in this competition, each trying to revolutionize everyday life through IoT projects, but we can also see some large players also in this space. Probably the most well known and advanced is John Deere’s Intelligent Solutions Group, which uses AWS IoT for precision agriculture, which makes the most of limited natural resources to feed more people. The foundation of precision agriculture is highly automated farming machines guided by software, GPS technology and satellites. With sub-inch accuracy, farmers control the precise placement of seeds and chemicals. They spray precisely the right amount of fertilizer and harvest precisely. Sensors and IoT make those things possible.
And this is only the start. Over the last few years we have heard of energy suppliers, car manufacturers and other big enterprises making big plans and investing in IoT projects.
IoT is a rapidly growing market and, in my opinion, the new extensions for AWS Lambda@edge processing will take AWS adoption to a new level. Even though we cannot say it’s ‘serverless’ anymore, as Lambda is running on on-premise devices, the level of reusability, integration and support with the AWS ecosystem is incredibly powerful and can greatly boost new or existing IoT projects.
Interested in how you can use Lambda? My blog ‘5 killer use cases for AWS Lambda’ has proved very popular amongst our readers and customers! Check it out! Read the blog.
