Showing posts with label Serverless. Show all posts
Showing posts with label Serverless. Show all posts

March 2, 2019

Serverless Series - Demystifying Serverless Architecture


In the blog series Demystifying Serverless Architecture, will be covering the evolution of Serverless Architecture with an introduction to The AWS Serverless Platform followed by walkthrough of sample use cases to build serverless web application using Amazon S3, Amazon API Gateway, AWS Lambda & Amazon CloudWatch.


Serverless Blog Series 

Reference Links




Serverless Series - Building a Serverless WebApp

 Serverless Series HomeAs part of the Serverless WebApp Demo, we will be building 2 use cases and 1 for DIY. All of these use cases will involve Amazon API Gateway, Amazon S3, Amazon CloudWatch & AWS Lambda. Familiarise yourself a bit about these services before you dive into the below sessions. Source code for the Serverless WebApp can be found on Github.

High-level steps to setup Serverless WebApplication using API Gateway & Lambda

  1. Create LambdaFunction with content from BasicLambda.py code, create a new role with `Simple Micro Services` permission and add API Gateway Trigger
  2. Create & Configure API Gateway GET Method to invoke BasicLambda.py
  3. Create S3 Bucket, make its objects public, convert it to a static website.
  4. upload error.html & index.html to S3 bucket created in #3, make these objects public
  5. Launch the S3 Static Website URL to see the web application is working
  6. Copy StopRunningInstances.py code to replace existing Lambda Function
  7. Add `AmazonEC2FullAccess` Lambda Execute Role created in #1
  8. Create one or more EC2 instances with tag name `AutoOff` with the value `yes` (case sensitive)
  9. Launch the S3 Static Website URL to test the running EC2 instances with the tag AutoOff=yes are shutdown.


MindMap covering Use Case 1 & Use Case 2 - View this for before proceeding..

Serverless WebApp Build - Use Case 1
Serverless WebApp Build - Use Case 2
Serverless WebApp Build - Use Case 3 - DIY

Serverless Series - Serverless WebApp - Use Case 3

 Serverless Series HomeHere is the network diagram to build Serverless WebApp - Use Case 3. It is just an increment of Serverless Webapp - Use Case 2 with Route 53 integration. So DIY :)

Network Diagram


Serverless Series - Serverless WebApp - Use Case 2

 Serverless Series HomeThis blog post will be taking you through the details to build Serverless WebApp - Use Case which will get the running EC2 instances with tag AutoOff=yes through lambda function and stop those instances, display the details in webpage and add to cloudwatch logs.

Network Diagram

Output  Webpage


Steps to AutoOff EC2

  • Copy StopRunningInstances.py code to replace existing Lambda Function 
  • Add `AmazonEC2FullAccess` LambdaExecutionRole created earlier
  • Create one or more EC2 instances with tag name `AutoOff` with the value `yes` (case sensitive)
  • Launch the S3 Static Website URL to test the running EC2 instances with the tag AutoOff=yes are shutdown.

Cleanup
  • CloudWatch console Delete the log group /aws/lambda/ServerlessWebAppDemo under CloudWatch Logs
  • S3 console -> Delete S3 Bucket serverlesswebappdemo
  • Lambda consolue -> Delete Lambda Function - ServerlessWebAppDemo
  • API Gateway console -> LambdaMicroservice -> Resources -> ServerlessWebAppDemo
  • IAM console -> Roles -> Delete Role - LambdaExecutionRole


Serverless Series - Serverless Services used in Demo

 Serverless Series HomeLet's have a closer look the 4 Serverless Services that will be used in today's Demo. 




Amazon S3

S3 is the massive storage and one of the back bones of AWS. Infact this was first service launched by AWS
How much data can I store?
The total volume of data and number of objects you can store are unlimited. Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 terabytes. The largest object that can be uploaded in a single PUT is 5 gigabytes. For objects larger than 100 megabytes, customers should consider using the Multipart Upload capability.

What storage classes does Amazon S3 offer?
Amazon S3 offers a range of storage classes designed for different use cases. There are three highly durable storage classes including 
  • S3 Standard for general-purpose storage of frequently accessed data
  • S3 Intelligent-Tiering storage class is designed to optimize costs by automatically moving data to the most cost-effective access tier, without performance impact or operational overhead. 
  • S3 Standard - Infrequent Access for long-lived, but less frequently accessed data, 
  • S3 One Zone-IA is for data that is accessed less frequently, but requires rapid access when needed. Unlike other S3 Storage Classes which store data in a minimum of three Availability Zones (AZs), S3 One Zone-IA stores data in a single AZ and costs 20% less than S3 Standard-IA. 
  • S3 Glacier  is a secure, durable, and low-cost storage class for data archiving.
  • S3 Glacier Deep Archive is Amazon S3’s lowest-cost storage class and supports long-term retention and digital preservation for data that won’t be regularly accessed. It is designed for customers — particularly those in highly-regulated industries, such as the Financial Services, Healthcare, and Public Sectors — that retain data sets for 7-10 years or longer to meet regulatory compliance requirements. 

S3 Storage Classes can be configured at the object level and a single bucket can contain objects stored in S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA. For further reading on S3 Storage Classes, refer https://aws.amazon.com/s3/storage-classes/.


 Amazon CloudWatch

CloudWatch sits under Management Tools in AWS Console and can monitor any AWS Services – ALB, EBS, EC2, ELB, S3 etc
  • Metrics - For example, basic host level metrics like CPU Utilization, DiskOps etc are monitored where OS level metrics like memory would need you to write custom metrics. Documentations are available for the same.
  • Dashboards – Creates awesome dashboards to see what is happening with your AWS Environment. 
  • Alarms – Allows you to set Alarms that notify you when particular thresholds are hit. 
  • Events – CloudWatch events helps you to respond to state changes in your AWS resources  
  • Logs – CloudWatch Logs helps you to aggregate, monitor, and store logs. 

Amazon API Gateway
Amazon API Gateway is an AWS service that enables you to create, publish, maintain, monitor, and secure your own APIs at any scale. You can create robust, secure, and scalable APIs that access AWS or other web services, as well as data stored in the AWS Cloud
  • Support for stateful (WebSocket) and stateless (REST) APIs
  • Integration with AWS services such as AWS Lambda, Amazon Kinesis, and Amazon DynamoDB
  • There are two kinds of developers who use API Gateway: API developers and app developers.
    • An API developer creates and deploys an API to enable the required functionality in API Gateway. The API developer must be an IAM user in the AWS account that owns the API.
    • As an API developer, you can create and manage an API by using the API Gateway console, CLI, SDK or Cloud Formation Template
    • An app developer builds a functioning application to call AWS services by invoking a WebSocket or REST API created by an API developer in API Gateway.
    • The app developer is the customer of the API developer. The app developer does not need to have an AWS account, provided API is designed that way.
Together with AWS Lambda, API Gateway forms the app-facing part of the AWS serverless infrastructure

AWS Lambda

AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time you consume - there is no charge when your code is not running. With AWS Lambda, you can run code for virtually any type of application or backend service - all with zero administration. AWS Lambda runs your code on a high-availability compute infrastructure and performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring and logging. 
  • All you need to do is supply your code in one of the languages that AWS Lambda supports (currently Node.js, Java, C# and Python). 
  • You can use AWS Lambda  
    • (1) to run your code in response to events (Event Driven), such as changes to data in an Amazon S3 bucket or an Amazon DynamoDB table;  
    • (2) to run your code in response to HTTP requests using Amazon API Gateway; or invoke your code using API calls made using AWS SDKs.  
  • Lambda functions are independent but one lambda function can trigger other lambda functions 
  • Lambda scales automatically and also can perform actions globally like copying files from S3 in one region to other region.

Pricing at high-level:

  • S3 - new AWS customers receive 5 GB of Amazon S3 Standard storage, 20,000 Get Requests, 2,000 Put Requests, 15GB of data transfer in, and 15GB of data transfer out each month for one year.
    • ~ 0.23/per GB update 50GB
  • API Gateway 
    • Rest API 
      • Pay only for the API calls you receive and the amount of data transferred out
      • For better performance and faster API execution, you can optionally provision a dedicated cache for each stage of your APIs. After you specify the size of the cache you require, you will be charged the following hourly rates for each stage’s cache, without any long-term commitments.
      • Free Tier - 1M API CALLS RECEIVED | 1M MESSAGES | 750,000 CONNECTION MINUTES
    • WebSocket APIs
      • Pay only for messages sent and received and the total number of connection minutes. You may send and receive messages up to 128 kilobytes (KB) in size. Messages are metered in 32 KB increments. So, a 33 KB message is metered as two messages.
      • For WebSocket APIs, the API Gateway free tier includes one million messages (sent or received) and 750,000 connection minutes for up to 12 months.
  • AWS Lambda
    • Lambda counts a request each time it starts executing in response to an event notification or invoke call, including test invokes from the console. You are charged for the total number of requests across all your functions.  
  • Amazon CloudWatch
    • Basic Monitoring Metrics (at 5-minute frequency) where detailed monitoring Metrics (at 1-minute frequency) is charged
    • Example: EC2 Detailed Monitoring is priced at $2.10 per instance per month and goes down to $0.14 per instance at the lowest priced tier.) 

For the Sample that we are going to try today, trust me that it won’t incur any charge if you clean-up diligently :).