January 10, 2018

Introduction to Salt States

    What is Salt?

    • Why Salt before what is Salt? 
      • Salt is very similar to Chef, Puppet, Ansible but very Powerful. 
      • Using ssh is substantially slow and not scalable, hence TCP based Salt stands out. 
    • Thomas Hatch’s first commit was on Feb 2011 for SaltStack and the company was founded in 2012. Linkedin was the first one to use Salt and now there are 10,000+ users. 
    • Salt is a remote execution and configuration management tool.  
    • The foundation of Salt is remote execution, which is very unique and enables extremely fast & reliable Remote Control Systems.  
    • Salt is designed to be secure using AES encryption and public-key authentication; incredibly scalable using an advanced ZeroMQ topology, so that salt daemon includes a viable and transpired AMQ broker.Further Salt takes advantage of communication via msgpack, which makes it fast and light network traffic. 
    • Salt is open source developed under Apache 2.0 license. 
    • Salt is flexible enough to support Agent (minion) or Agentless (via ssh). 
    • Salt is written in phython, hence it’s very easy to contribute. 
    • The salt-master daemon runs on the designated Salt master and performs functions such as authenticating minions, sending, and receiving requests from connected minions and sending and receiving requests and replies to the 'salt' CLI. 
    • LocalClient – Salt Client runs on the same machine as of salt master is called LocalClient. This connects to the salt master via – TCP - 4506 

    Salt – Master 

    • Host name of the Salt master should be salt. The salt minion looks defaulting to that name. you can change the host name but the default one is simple  
    • Salt Master connects to Salt Minion via TCP 4505. 
    • ZeroMQ / AMQP broker / Wrapper MQ 

    Salt – Minion 

    • Each salt minion establishes a connection to the master ReqServ via TCP 4506  

    ZeroMQ 

    • High-speed network message passing. 

    Salt States 

    • The folder (/opt/salt/states/base) under which you are planning to create your state files should be added in file_roots master config file (/etc/salt/master) as your root directory. 
    • Create the following file under /opt/salt/states/base
      • Filename:list-processes.sls 
                                #State Id should be unique across all states files. No space after or before: list-processes:| # type the command after two black spaces in the immediate new line   cmd.run: # 4 spaces after new line, once space after -, no space before : and once space after :     - name: ps -ef 
                   Run the file: sudo salt ‘*all-02*’ state.sls list-processes 
                   Output start & end: 

    Pillars 

    • Pillars are tree-like structures of data defined on the Salt Master and passed through to minions.  
    • A pillar is a list of data on the master (in SLS format) that you need to distribute to your minions. Pillar allows you to set variables that the minions can access, for example a database configuration option. 
    • Pillar data is stored on the master and cached on matching minions.   Note: Pillar data is only available to matching minions. 

    Grains 

    • Salt grains are pieces of information gathered by and maintained by a minion (salt-minion daemon) and passed back to master upon request. 
    • In Salt, grains are used for immutable aspects of your minion, such as the cpu, memory, location, time zone, etc. 
    • Thegrainsare stored on each minion in YAML and hold OS and hardware specific information to that minion.  
    • Grains are stored on minions and cached on the master.   Note: All grains data is available to all minions. 

    Difference between Grains & Pillars

    Differences 
    Grains 
    Pillars 
    Info which.. 
    ... Minion knows about itself 
    ... Minion asks Master about 
    Distributed: 
    Yes 
    No 
    Centralized 
    No 
    Yes 
    Computed automatically: 
    Yes (mostly should) 
    No 
    Assigned manually: 
    No (or kept to minimum) 
    Yes 
    Conceptually intrinsic to… 
    … individual minion node 
    … entire system 

    Salt Commands - Tried  & Tested

    • sudo salt-run manage.up – lists the connected salt minions 
    • sudo salt-run manage.status – lists the minions that are up & down 
    • sudo salt-run manage.down – lists the minions that are down 
    • type ls -1 /var/cache/salt/master/minions from master to list the available minions 
    • "sudo salt-key -L" will list all minions that whose public keys you've accepted on your master. 
    • sudo salt ‘*’ cmd.run ‘<daemon  or application name> <parameters>’ runas=<username>  Note: run.cmd does not work. With cmd.run it says ERROR: Sorry, Windows does not support runas functionality. 
    Use grains, specify OS to run on specific set of OS. 
    sudo salt –G ‘os:CentOS’ ‘<daemon  or application name> <parameters>’ runas=<username> 
    or  sudo salt –G ‘kernel:Linux’ ‘<daemon  or application name> <parameters>’ runas=<username> 
    sudo salt –G ‘kernel:windows’ cmd.run 'd:\apps\cygwin64\bin\bash.exe -li <application name> <parameters>'  

    • Find the IP Address of the minions 
      • sudo salt ‘*’ grains.item fqdn_ip4 
      • sudo salt kernel:windows grains.item fqdn_ip4 
      • sudo salt kernel:linux grains.item fqdn_ip4 
      • salt-call --local grains.item fqdn - this one to get fqdn on EC2 instance
    • Command Line reference for Masterless Minion 
      • salt-call -g lists the grains in the system - EC2 instance
      • salt-call --local state.highstate -l debug --retcode-passthrough pillar='{"name": "value"}'
      • salt-call --local state.highstate -l warning --retcode-passthrough pillar='{"name": "value"}'`
      • salt-call --local state.apply <sls filename without  extn> -l debug
      • salt-call --local grains.item fqdn - this one to get fqdn on EC2 instance

    December 23, 2017

    Introduction to AWS & Cloud Migration @ BIT Sathy

    Guest Lecture on Introduction to AWS & Cloud Migration @ BIT Sathy for Information Technology & Computer Science Department Professors.

    Here is the topics covered during the lecture and hope professors did enjoy the session :) 
    • What is cloud computing?
    • What is cloud migration?
    • Different types of Cloud Offerings available in the market
    • Different forms of cloud computing
    • World Leaders in Public Cloud & how AWS stands out!
    • Amazon Web Services vs On-Premise
    • Introduction to Amazon Web Services (AWS)
    • Details on 5 core AWS services that are most widely used during migration
    • Introduction to AWS Tooling
    • A successful use case on Cloud Migration -> Data Centre to AWS.


    Refer the link to certificate of appreciation  from the college for the Guest Lecture.

    Special Award for Contribution to Alma Mater 2017

    Appreciation goes a long way and a recognition from your Alma Mater after 17 yrs of your graduation is something close to your heart.
    I was delighted to receive the Special Award from Bannari Amman Institute of Technology for my contribution towards the technical guidance & lectures to the college as well as to the Bengaluru based BIT Alumnae as a Secretary of  BIT Alumni Chapter, Bengaluru.




    December 13, 2017

    Install CodeDeploy agent on EC2 Instance


     Code Snippet to install code deploy agent

    When you provision EC2 instance for deployment to be done by AWS CodeDeploy, that EC2 instance should have CodeDeploy agent for deployment to proceed. https://docs.aws.amazon.com/codedeploy/latest/userguide/codedeploy-agent-operations-install-linux.html 

    Option 1:


                yum update -y
                yum install -y ruby
                yum install wget

                cd /home/ec2-user

                wget https://aws-codedeploy-${AWS::Region}.s3.amazonaws.com/latest/install

                chmod +x ./install
                ./install auto  

    service codedeploy-agent status 
    service codedeploy-agent start


    Option 2:


    yum update -y 
                yum install -y ruby aws-cli 
      
                cd /home/ec2-user 
                aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . --region us-east-1 
                chmod +x ./install 
    ./install auto

    Note: Use sudo, if elevated privilege is required.

    Overview of Sample Web Application Architecture

    The Sample Web Application depicted below will include Web Servers, App Servers and Database Servers
    -       There are two Availability Zones (AZ) in the Sample Web Application, in order to provide high redundancy and therefore high availability
    -       Subnet is nothing but the range of IP addresses in a VPC
    -       Network ACLs stands for Network Access Control Lists that are applied to subnets
    -       Each AZ has one private subnet and public subnet
    -       All subnets within a VPC is designed to talk to each other freely
    -       Only public subnets are accessible from the internet
    -       Servers in the private subnet can only make outbound calls to the Internet via the NAT server. No inbound traffic is accepted.
    -       NAT have only one purpose here  -> allows instances on private subnets to call out to the Internet to download updates. Traffic from the Internet is not permitted to make inbound connections
    -       Traffic is further restricted via security groups
    -       NAT Instance is relatively old service and we have an alternative now called NAT Gateway, which was introduced in Re-invent 2016
    -       Basically NAT Instance is an EC2 instance with certain configurations where you have to establish ASG to scale-up or down and enable fault tolerance whereas in NAT Gateway both elasticity and failover are handled by AWS.
    -       AWS Internet Gateway - An Internet gateway is a horizontally scaled, redundant, and highly available VPC component that allows communication between instances in your VPC and the Internet.
    -       Amazon Route 53 (Route 53) is a scalable and highly available Domain Name System (DNS)


    When the user access the website www.mycloudapp.com either from his computer or mobile, the request goes to Route 53, passes through Internet Gateway & Elastic Load Balancer before hitting the Web Servers in public subnet. The Application Servers and Database Servers are placed in private subnet which can be accessed only by Web Servers. These Servers in private subnet can make only outbound calls to the Internet where they get their software updates, which happens through VPC NAT Gateway.



    What is an ELB?
    -       ELB stands for Elastic Load Balancing.
    -       Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances & multiple availability zones.
    -       ELB enables you to achieve greater levels of fault tolerance in your applications ensuring that only healthy Amazon EC2 instances receive traffic

    -       ELB can automatically scales its request-handling capacity to meet the demands of application traffic.