Hosting a Simple WebApp on AWSCloud

We are going to host a simple web application on Linux & Windows EC2 instances and here are the Key requirements:

Requirement #1

•  Create a VPC with Internet Gateway, two route tables, two subnets in two availability zones
•  Define separate Network Access Control List (NACL) and Security Group for the two EC2 instances
•  Setup two EC2 instances - Linux & Windows on public subnet with Apache & IIS configured on port 80 
•  Ensure that application use custom index page named webindex.html
•  Website should be served from the 2 GiB additional EBS Volumes
•  Setup an Application Load Balancer to distribute traffic to the Linux and Windows servers  in a round-robin fashion which means that requests to the Application Load Balancer on port 80 will get re-directed to the Apache and IIS web servers listening on port 80.
• Code the website to fetch the static content like images / videos from S3 Bucket. 
• Validate website is being served over ALB public DNS.

Requirement #2

• Create an AMI out of Linux & Windows EC2 Instances
• Create 2 Launch Configurations with the AMIs created in previous step with the same instance specification as in RFE #1
• Create Auto Scaling Group (ASG) with the above Launch Configurations to scale in when CPU > 80% and scale out when CPU < 80%

Requirement #3

• Create a CloudFront distribution -> WebDistribution and point to ALB public endpoint

Services Used

• EC2 - Linux & Windows 2019, EBS, ALB, ASG, S3
• Region used: Mumbai

Requirement #1 Architecture Diagram

Step 1: Network & Security Group Setup

VPC

• Switch to ap-south-1 region (Mumbai)
• Create a VPC with a Name tag WebVPC with IPv4 CIDR block 10.0.0.0/16 leaving IPv6 CIDR block and Tenancy as default.
• Create Internet Gateway as WebIGW and attach to the VPC - WebVPC
• Create two route tables WebRT-Public and WebRT-Private with WebVPC selected
• Add a route to WebRT-Public pointing to the Internet Gateway - WebIGW

Subnet

• Create two Subnets with Name tag as WebSubnet1-Public & WebSubnet2-Public for two availability zones in in ap-south-1 region with WebVPC 
• Set the CIDR block to 10.0.1.0/24 & 10.0.2.0/24 respectively
• Go to public route table WebRT-Public, click on Subnet Association and select both the subnets

Network Access Control Lists (NACL)

• Create two Network ACLs with Name tags as WebNACL1 & WebNACL2. 
• Since NACLs are state-less, inbound and outbound rules have to be enabled explicitly
• Add the following rules for both inbound and outbound for WebNACL1
  

  Rule #	Protocol	Port	Source
  100	SSH	22	0.0.0.0/0
  110	HTTP	80	0.0.0.0/0

  
  
• Select Subnet Associations of WebNACL1 and pick WebSubnet1-Public
• Add the following rules for both inbound and outbound for WebNACL2
  

  Rule #	Protocol	Port	Source
  120	RDP	3389	0.0.0.0/0
  130	HTTP	80	0.0.0.0/0

  
  
• Select Subnet Associations of WebNACL2 and pick WebSubnet2-Public

Security Groups

• Create two Security Groups with Name tag as WebSG-Linux & WebSG-Win and set the VPC as WebVPC
• Since Security Groups are stateful, enabling inbound is sufficient
• Add the following inbound rules for WebSG-Linux
  

  Protocol	Port	Source
  SSH	22	0.0.0.0/0
  HTTP	80	0.0.0.0/0

• 
  
• Add the following inbound rules for WebSG-Win
  

  Protocol	Port	Source
  RDP	3389	0.0.0.0/0
  HTTP	80	0.0.0.0/0

• Note: Best practise is to use specific port rage or specific port instead of 0.0.0.0/0
• Please note by default all outbound connections are allowed.
  
  

Step 2: Setup Linux EC2 with Apache & host custom index page

Instance Configuration

Create an EC2 instance with the following configuration

               

Instance Spec	Values
AMI	Amazon Linux v2
EBS Volume - Root	8 GiB
EBS Volume - Additional	2 GiB
VPC	WebVPC
Subnet	WebSubnet1-Public
Security Group	WebSG-Linux
Create New Key Pair	webkeypair
Name tag	WebLinuxServer

User Data

#!/bin/bash

# Install Apache Web Server
sudo yum install -y httpd

# Turn on web server
sudo chkconfig httpd on   # httpd service comes up on reboot
sudo service httpd start

# Setup web server
cd /var/www/html

echo "<html><h1>Hello AWS Aspirants – I am running on Linux over port 80</h1></html> " > webindex.html 

Default Web Page setting

vi /etc/httpd/conf/httpd.conf to view the default document

to change the default document edit the following line 

DirectoryIndex  webindex.html index.html

Step 3: Setup Windows EC2 with IIS & host custom index

Create an EC2 instance with the following configuration

               

Instance Spec	Values
AMI	Amazon Windows 2019 Base Image
EBS Volume - Root	30 GiB
EBS Volume - Additional	2 GiB
VPC	WebVPC
Subnet	WebSubnet1-Public
Security Group	WebSG-Linux
Create New Key Pair	webkeypair
Name tag	WebWinServer

User Data

# Install & Configure IIS

<powershell>
Set-ExecutionPolicy Unrestricted -Force
New-Item -ItemType directory -Path 'C:\temp'
 
# Install IIS and Web Management Tools
Import-Module ServerManager
install-windowsfeature web-server, web-webserver -IncludeAllSubFeature
install-windowsfeature web-mgmt-tools

# Create custom index.html

Set-Location -path C:\inetpub\wwwroot

$htmlcode = " <html><h1> Hello AWS Aspirants - I am running on Windows Server Over Port 80 </h1></html>" 

$webindex | ConvetTo-Html - Head $htmlcode | Out-File .\webindex.html

</powershell>


Tips: <persist>true</persist>

By default user data commands are run once when the instance is first launched. If you would like your commands to run every time the instance is started you need to include the <persist>true</persist> at the end in your user data.

Manual IIS Setup

IIS Installation on Windows 2016

Create Windows 2019 EC2, RDP, Install & Configure IIS with this instructions

 https://help.w-systems.com/wmobile/howto_install_iis_win2016.html

Go to C:\inetpub\wwwroot
rename iisstart.htm to iisstart_original.htm

create webindex.html and place
<html><h1>Hello AWS Aspirants – running on Windows & IIS Server – on port 80</h1></html>

Save the file and run http://localhost on same EC2 or use public URL from outside

Default index.html Setting: 
Search -> iis -> Default Web Site -> Default Content (double click this to view the default documents) 

Add webindex.html as default page.

Step 3: Create an Application Load Balancer and point to EC2 public endpoints

Step 4: Create S3 Bucket and upload Static Images for Website

• Create a S3 Bucket named simplewebapp101
• Upload the static files that you want to place in the webpage rendered from Linux or Windows EC2
• Make the objects public
• Create an Instance Role with S3 Read-Only access and apply to both Windows & Linux EC2
• Edit the webindex.html to update with the new image URL stored in S3 Bucket and refresh the web page using ELB endpoint

 

Requirement #2 Architecture Diagram

Requirement #3 Architecture Diagram
DIY

Amazon SNS & SQS Simplified

A overview of Amazon SNS & SQS with introduction, creating configuring & testing sample workflow using Amazon SNS, SQS & S3 through AWS Console.

Amazon SNS

Amazon Simple Notification Service (SNS) is a highly available, durable, secure, fully managed pub/sub messaging service that enables you to decouple microservices, distributed systems, and serverless applications. Amazon SNS provides topics for high-

throughput, push-based, many-to-many messaging. Using Amazon SNS topics, your publisher systems can fan out messages to a large number of subscriber endpoints for parallel processing, including Amazon SQS queues, AWS Lambda functions, and HTTP/S webhooks. Additionally, SNS can be used to fan out notifications to end users using mobile push, SMS, and email. 

Amazon SQS

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available. Get started with SQS in minutes using the AWS console, Command Line Interface or SDK of your choice, and three simple commands.
SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.

What is Dead Letter Queue?

A dead-letter queue is a queue that other (source) queues can target for messages that can't be processed (consumed) successfully. In this tutorial you learn how to create an Amazon SQS source queue and to configure a second queue as a dead-letter queue for it. For more information, see Amazon SQS Dead-Letter Queues.

Important

When you designate a queue to be a source queue, a dead-letter queue is not created automatically. You must first create a normal standard or FIFO queue before designating it a dead-letter queue.

The dead-letter queue of a FIFO queue must also be a FIFO queue. Similarly, the dead-letter queue of a standard queue must also be a standard queue.

 Simple Workflow - Amazon SNS & SQS:

Assume a scenario you are placing an order for a notebook, SNS sends an email to designated email address about the order request plus action to be taken and also adds the message to the orders content queue in SQS.

Setup SNS Topic

• Login to AWS Console & setup an SNS Topic named Order and two subscription for Email & Amazon SQS
• Create a Subscription using Email protocol, configure your email id and verify the email address by clicking the Subscription link received to your email
• Create a Subscription using Amazon SQS protocol, copy the ARN of the ContentQ that you will be creating under Setup SQS Queues

Setup SQS Queues

• Create a standard queue named ContentQ , leave everything to default and change message retention period to 1 day (range is 1 to 14 days).
• Go to Permission tab of the newly created Q enable Allow Everybody for All SQS Actions (SQS:*)
• Redrive policy will be empty at this point in time
• Create a standard queue named ContentRedriveQ , leave everything to default and change message retention period to 5 day (range is 1 to 14 days).
• No change to permission and redrive policy will be empty
• Go to ContentQ, enable Dead Letter Queue and select ContentRedriveQ, leave Maximum receives as 1.
• Testing SQS 
• Select ContentQ -> Queue Actions -> Send a Message -> Type a message and Click Send
• Select ContentQ -> Queue Actions -> View / Delete Messages -> Click Start polling for messages.  This action reads the message from the queue but does not send a confirmation of successful processing of the message. This turns to be failed or unprocessed category, which then falls into Dead Letter Queue. Now you will be able to see this message under ContentRedriveQ, which is mapped as a Dead Letter Queue of ContentQ.

Testing SNS & SQS Flow

• Go to Amazon SNS ->Topic -> Orders -> Publish Message -> Enter Subject & Message Body, leave rest to default and click `Publish Message`.
• This message will be delivered to your configured email via Email protocol subscription to Order topic and will be delivered to SQS - ContentQ via SQS protocol subscription to Order topic.
• Select ContentQ -> Queue Actions -> View / Delete Messages -> Click Start polling for messages.  This action reads the message from the queue but does not send a confirmation of successful processing of the message. This turns to be failed or unprocessed category, which then falls into Dead Letter Queue. Now you will be able to see this message under ContentRedriveQ, which is mapped as a Dead Letter Queue of ContentQ.

Enhanced Workflow: Amazon S3, SNS & SQS Events:

Now lets enhance the above workflow a bit by triggering message as part of S3 upload / put event. You will see how well the messages fan-out from SNS Topic to email & SQS queues.
Message fan-out is nothing but broadcasting messages from one to many.

Setup S3 Event

• Create an S3 bucket named ordersbucket, go to Properties -> Events -> Add Notification -> Enter Details for New Event
  Event Name: ForSNSNotificaiton, Events: PUT, Send to: SNS Topic, SNS: Orders and try to save
  You will get the following error as your S3 bucket do not have permission to SNS Topic
  Unable to validate the following destination configurations. Permissions on the destination topic do not allow S3 to publish notifications from this bucket. (arn:aws:sns:us-east-1:<account id>:Orders)
• To grant permission to the S3 bucket events to trigger SNS Topic, go to SNS Topic -> Orders -> Edit Access Policy ->
  replace the blue text by green 

"Condition": {
        "StringEquals": {
          "AWS:SourceOwner": "<account id>"
        }

"Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:s3::: ordersbucket"
        } 

• Now go back & Click Save for S3, it should work.
• This configuration itself triggers the S3 event, which would sent the below email to your configured email and a message to ContentQ

{  "Type" : "Notification",  "MessageId" : "f0c66789-543d-5527-9bfd-328a83cbd237",  "TopicArn" : "arn:aws:sns:us-east-1:544638597657:Orders",  "Subject" : "Amazon S3 Notification",  "Message" : "{\"Service\":\"Amazon S3\",\"Event\":\"s3:TestEvent\",\"Time\":\"2019-03-14T04:16:54.038Z\",\"Bucket\":\"ordersbucket\",\"RequestId\":\"37475229D2AF3B2E\",\"HostId\":\"csuC6wL1zh8now9VybrB8LUju2Nc1z1sF1C1TN0HU15tHR6KvI2x4lIBqrM9pMXiz5wgSQkgyYczk=\"}",  "Timestamp" : "2019-03-14T04:16:54.145Z",  "SignatureVersion" : "1",  "Signature" : "LBIZeIodCe6Y1Irx8ZBLifqrEPbUw+tEFAwVygDszoMDyKZSrMD5kwKsJ0kZzjuaXvOeYhdITIuwgWMNnrRJpLhH9EtlhbHV0g/GT/pDaNZb52JV6vRB8zO0de8DC2AVDgQ7TyxS7Vx6TuqBPuxsRX0mdD1H+UPxc3+1ory7UAXggcT0h7zKVQkT7BrT+9dJs8+RfUQ/1YODYNZCR0qJMHQIqUDbx4KR0KtuobZ+wTwT60hJgUYcM/13VL7cZgckMNGuYv8qNJc4hEwb591V8C5nnvx7JEksLJkP91PfQJsCzoGaGvh+UhDWmjVI6fHMZNo+zmyMe8I0sEw==",  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-6aad65c2f991xxxxxxxxda11f913f9.pem",  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:<accountid>:Orders:925ec4b2-1333-4b69-b82b-569c1eb7dd94"}

Testing S3, SNS & SQS Flow

• To test the whole flow upload a file into ordersbucket, which would trigger S3 PUT Event -> SNS Topic - Orders and watch the rest to happen seamlessly
• This message will be delivered to your configured email with subject - Amazon S3 Notification via Email protocol subscription to Order topic and will be delivered to SQS - ContentQ via SQS protocol subscription to Order topic.

• Select ContentQ -> Queue Actions -> View / Delete Messages -> Click Start polling for messages.  This action reads the message from the queue but does not send a confirmation of successful processing of the message. This turns to be failed or unprocessed category, which then falls into Dead Letter Queue. Now you will be able to see this message under ContentRedriveQ, which is mapped as a Dead Letter Queue of ContentQ.

Cleanup

Let's clean the resources that we have created as part of this demo

• Delete S3 Bucket named ordersbucket
• Delete SNS Topic Subscriptions that you created for email protocol & SQS
• Delete SNS Topic Orders
• Delete SQS - ContentQRedrive & ContentQ
• You will get a waring that there are messages in the Queue, if you still want to delete. You can either choose to delete or purge the messages and then come back for deletion.

How do you protect the data stored in Amazon S3?

Well, its not just the data stored in Amazon S3 needs to be safe but during the onward & return journey as well.

That broadly classifies  the data protection into categories  as protecting data while in-transit (as it travels to and from Amazon S3) and at rest (while it is stored on disks in Amazon S3 data centers).

· Server-Side Encryption – User requests Amazon S3 to encrypt the object before saving it on disks in its data centers and decrypt it when object are downloaded. Has 3 different types as listed in the image based on who manages the keys.

· Client-Side Encryption – The data is encrypted on the client-side and the encrypted data is uploaded to Amazon S3. In this case, user manage the encryption process, the encryption keys, and related tools.

Manage the S3 Data till end of life, efficiently

•AWS S3 provides Lifecycle management process to manage the data till end of life in a cost efficient way.

•Lifecycle of a object in the bucket or entire bucket can be managed using Lifecycle rules

•Lifecycle rules enable you to automatically transition objects to the less expensive S3 storage option say Standard - Infrequent Access Storage Class, and/or archive objects to the Glacier Storage Class, and/or remove objects after a specified time period.

•Each rule has an action either Transaction action or Expiration actions.

•You can apply rules to either all the objects in the bucket or all the objects that share the specified prefix.

•Moving the object directly from S3 to Glacier and S3 to Expire / Delete is also possible.

•The object should be in Glacier for min of 90 days before delete; even otherwise it will be charged for 90 days as per the Glacier pricing model.

•Old UI for Lifecycle management is pretty self-explanatory than the New UI, hence you may switch to Old UI & try for better clarity.