Credentials Management with AWS

Storing & Managing the credentials should be handled cautiously be it on-premise or on the cloud.
With AWS you can handle that in two ways viz. 1. credstash  2. parameter store

CredStash - KMS + DynamoDB

CredStash is a very simple credential management and distribution system announced in Re:invent 2014, which uses

• Dynamo DB to store secrets
• KMS to hold the master keys
• IAM Policies for Access Control

Here you go with the steps for CredStash setup

1. Create an IAM user or Role

You need to create an IAM user or Role, which has minimum permission to generate KMS Data Key and put / get from from the Dynamo DB Table. Hence the following permissions in the IAM policies are mandatory - kms:GenerateDataKey and dynamodb:PutItem.

2. Create a KMS Key 

• Login to AWS Console
• Go to IAM Console -> Encryption Keys and Click Create Key
• Select Region  under Encryption Keys
  [Note: Master keys are defined and stored regionally. Master keys cannot be shared across regions. You can use the same alias and/or description for keys across regions, but the underlying key itself will be unique.]
• Enter Key Name: credstash  with appropriate Description and Choose Key Material Origin as KMS under Advanced Options.
• tag it as per your standard
• Key Administrators: Choose the IAM users and roles that can administer this key through the KMS API
• Key Users: Choose the IAM users and roles that can use this key to encrypt and decrypt data from within applications.

3.  Setting up CredStash

# Make sure to run with elevated privileges before you run the below commands
$ sudo su -

CredStash - Installation on mac:
# Ensure Python version as 2.7 or above
$ python --version   

# Install pip
$ sudo easy_install pip 

# Install credstash
$ sudo pip install credstash 

CredStash - Installation on Linux:
# Ensure Python version as 2.7 or above
$ python --version   

# Ensure pip version as 9.0.1 or above
$ pip --version   

# upgrading pip from 6.1.1 to 9.0.1 to install credstash moves /usr/bin/pip to /usr/local/bin/pip # Ensure pip version as 9.0.1 or above

$ sudo pip install --upgrade pip  

$ /usr/local/bin/pip --version

# Install credstash
$ sudo pip install credstash 

CredStash - Setup: 

# credstash setup - basically creates a Dynamo DB table named `credential-store` by default and you can optionally special custom table name with `--table` option  

# Default
$ credstash setup 

# Custom
$ credstash --table my-credential-store setup

4. CredStash - Store & Manage Credential:

#1 Storing the Secret Key Value through parameter
$ credstash --table my-credential-store put 'secret_key_name' 'secret_key_value' ['context_key'='context_val']

'context_key' can be used to pass an 'encryption context' like environment and if specified during encryption should be specified during decryption as well. Can be used for extra security as defined here.   

[Note: Watch out the above single quote (can be double quote as well) during copy + paste, otherwise you might get the following error
An error occurred (ResourceNotFoundException) when calling the PutItem operation: Requested resource not found.] 

         
          # Reading the Secret Key Value
          $ credstash --table my-credential-store get 'secret_key_name' 

          # Assigning the Secret Key Value to a variable
          $ my_secret_key_value = $(credstash --table my-credential-store get 'secret_key_name')

          # Deleting the Secret Key from Dynamo DB Table
            $ credstash --table my-credential-store delete 'secret_key_name'

          # You can clean your credstash setup by just deleting the table
          $ aws dynamodb delete-table --table-name <table name>

           <table_name> = credential-store by default and custom as you have defined.

          #2 Storing Secret Key Value from a file via stdin.
          # Storing @ as first character in secret_key_value errors outs
          # basically this is due to the way credstash.py utility is written    
            $ credstash --table my-credential-store put 'my_complex_key_name' '@abc$de*fg'
                usage: credstash put [-h] [-k KEY] [-v VERSION] [-a]                          [-d               {SHA,MD5,RIPEMD,SHA384,SHA224,SHA256,SHA512,WHIRLPOOL}]                          credential value [context [context ...]]           
               credstash put: error: argument value: Unable to read file abc$de*fg       
       
          # Alternate option to have @ as first character in secret_key_value
          # Store key value in a file (say pwd.txt)
            $ cat pwd.txt | credstash --table my-credential-store put 'my_complex_key_name' -
               my_complex_key_name has been stored
            [Note: hypen in the end tells credstash to take the value from stdin.]

            $ credstash --table my-credential-store get 'my_complex_key_name'
             @abc$de*fg

            $ export my_key_value=$(credstash --table my-credential-store get 'my_complex_key_name')
            $ echo $test
              @abc$de*fg

#3 Storing Secret Key Value from a file by specifying file name as parameter prefixed with @   

# basically this is due to the way credstash.py utility is written    

$ credstash --table my-credential-store put 'my_complex_key_name' '@pwd.txt'     

   my_complex_key_name has been stored 

$ credstash --table my-credential-store get 'my_complex_key_name'

               @abc$de*fg

Parameter Store under SSM

Parameter Store is a relatively new service launched by AWS in Re:invent 2016 as part of EC2 Systems, which helps you to create a parameter, to store sensitive information which can be accessed from application.

Here you go with the steps for Parameter Store setup

1. Create an IAM user or Role

You need to create an IAM user or Role and assign
ssm:DescribeParameter, ssm:GetParameters permission if you want the user / role to read the parameter & related information. However you will have to provide ssm:PutParameter & ssm:DeleteParameter if you want the user / role to administer Parameter Store.

         Refer  here  for details about Controlling Access to System Manager Parameters.

2. Create a KMS Key 

• Login to AWS Console
• Go to IAM Console -> Encryption Keys and Click Create Key
• Select Region  under Encryption Keys
  [Note: Master keys are defined and stored regionally. Master keys cannot be shared across regions. You can use the same alias and/or description for keys across regions, but the underlying key itself will be unique.]
• Enter Key Name:  database_keys  with appropriate Description and Choose Key Material Origin as KMS under Advanced Options.
• tag it as per your standard
• Key Administrators: Choose the IAM users and roles that can administer this key through the KMS API
• Key Users: Choose the IAM users and roles that can use this key to encrypt and decrypt data from within applications.

3. Store & Manage Credential:

    Create Parameter through Console

• Login to AWS Account, go to AWS EC2 Console -> System Manager -> Parameter Store
• Click Get Started Now or Create Parameter, enter Parameter Name, Description, Type (choose SecureString for storing passwords), KMS Key ID followed by the actual value to be set for the parameter.
  This value will be encrypted using the master key from the given KMS Key ID and stored.

    Create Parameter through AWS Cli

         # You have option to store 3 types of parameters - viz String, StringList and SecureString where this blog focuses only on SecureString.
         # Storing the Secret Key
          $ aws ssm put-parameter --name 'secret_key_name' --type 'SecureString' --value 'secret_key_value' --key-id 'kms_key_id'
               [Note:
                 1. By default aws ssm uses alias/aws/ssm key which is created by AWS when the account is setup. You can pass custom KMS key using --key-id option.
                 2. --overwrite can be used to replace the existing value
                ]
          # Reading the Secret Key Value - Encrypted
          $ aws ssm get-parameters --name 'secret_key_name' --with-decryption
                 the output is in json format as listed below 
                      {
                            "InvalidParameters": [],
                            "Parameters": [
                                 {
                                       "Type": "SecureString",
                                       "Name": "secret_key_name",
                                       "Value": "encrypted_secret_key_value"
                                 }

                             ]                        }  
          # Reading the Secret Key Value - decrypted original value
          $ aws ssm get-parameters --name 'secret_key_name' --with-decryption
                 the output is in json format as listed below 
                      {
                            "InvalidParameters": [],
                            "Parameters": [
                                 {
                                       "Type": "SecureString",
                                       "Name": "secret_key_name",
                                       "Value": "secret_key_value"
                                 }

                             ]                        }  
          # Assigning the Secret Key Value to a variable
          $ my_secret_key_value = $(aws ssm get-parameters --name 'secret_key_name' --region='region_code' --with-decryption --query Parameters[0].Value --output text)
               [Note:
                   If the parameter is not defined in the specific region or not able to retrieve due to any other region None will be returned.]

          # Deleting the Parameter / Secret Key
          $ aws ssm delete-parameter --name 'secret_key_name'

CredStash vs Parameter Store

Refer AWS KMS Pricing and DynamoDB Pricing for more details.

CredStash involves both AWS KMS & DynamoDB pricing whereas Parameter Store involves only AWS KMS pricing. But for storing database credentials, we would still remain with the free limit of DynamoDB usage. 

Getting started with AWS

Here is the pictorial representation of 5 steps on Getting started with AWS

1. Create an AWS account

2. Create an IAM user using root account

3. Create VPC with Single Subnet using new user

4. Create a Security Group

5. Create an Instance, link to VPC, Subnet & Security Group

RDS supported Database Engines vs RDS Features

As on 2017 RDS supports 6 Database Engines viz. MySQL, Oracle, SQL Server, PostgreSQL, MariaDB & Aurora. The below table depicts the comparison of 4 salient features of RDS - Automated Backup, Encryption, Multizone Availability and Read Replica

RDS Database	Automated Backup  & Snapshot	Encryption @ Rest	Multizone Availability	Read Replica
MySQL	Yes	Yes	Yes	Yes
Oracle	Yes	Yes	Yes	No
SQL Server	Yes	Yes	SQL Server Mirroring	No
PostgreSQL	Yes	Yes	Yes	Yes
MariaDB	Yes	Yes	Yes	Yes
Aurora	Yes	No	No	No