AWS User Group Bengaluru had organized the first
ever AWS Community Day in Bengaluru on 28th Oct 2017 which was an all-day event
running two parallel tracks - proven use cases / success stories and workshops.
I also had an opportunity to present one of our companies most successful
use-case "Migration Story: Agile to FDD in-light of AWS"
While moving from Agile to Feature Driven Development (FDD) with geographically
distributed development centre, it is customary to have dedicated light
environment per feature and robust automation to build & deploy at your own
will!
The migration story was on Agile to
FDD from a cloud
based supply chain leader, GTNexus an Infor Company, that unfolds how the AWS Services came as a blessing
in disguise to provide a highly scalable, Elastic & Cost Effective Solution
to facilitate on-demand miniature development environments and independent
Build & Deployment framework.
Outline
- How GTNexus used
Agile Scrum and the paradigm shift in the branching strategy when moved
from scrum to Feature Driven Development
- Development Cycle
& the Timeline
- High-level
architecture of a full-fledged test environment hosted in DataCenter and
the need for on-demand, scalable, miniature development environment for
feature based testing in AWS.
- Highly Elastic
Build & Deployment Framework to cater the on-demand build &
deployment needs of FTEs
While we were talking, our we had ~ 500 on-demand FTEs across 4 regions viz - Mumbai, US West - Oregon, Europe - Frankfurt & US East - North Virginia. - How useful was the
move to AWS in terms of Scalability, Elasticity, Cost
Efficiency and Security.
Miniature FTE in AWS
On the pursuit of light environment for Feature Based Testing,
re-designed the QA Environment with a Single Windows & Linux VM Images with
application services installed and housing the OS specific Data Services (say
SQL Server in Windows; DB2, Riak/KVS & memsql in Linux).
After the light environment is setup in private cloud / Data
Center, the next step is to setup this light Feature Test Environment in AWS.
Windows & Linux VM Images are exported from Data Center & imported into
AWS. A Feature Test Environment in AWS comprises of a VPC, Internet Gateway,
Internet facing subnet, two EC2 instances that are replicated from the Data
Center VMs. These two EC2 instances are stored as pre-fabricated AMIs for
creating on-demand FTEs using CloudFormation templates.
Elasticity in FTEs
With a FrontEnd application developed using AWS SDK for Python,
Engineer aka the Feature Owner is free to create new FTE by providing the
following inputs like
- Who owns this
FTE?
- Where is User’s
office located?
- Which branch
should the FTE be based off?
- What is the
feature development Id as in Jira?
and creates the cloud formation template on the fly, which in turn
creates the Windows & Linux Instance from the region specific AMIs in the
low latency region close to user’s office location and creates RecordSets.
The feature owner can trigger the build & deployment the against the chosen branch to get his changes into the new FTE.
The feature owner can trigger the build & deployment the against the chosen branch to get his changes into the new FTE.
With the FrontEnd application, Engineers are free to create FTEs at their own will, trigger build & deployment as needed, Run Code validation suite like JUnit Tests & SQL Static Code Check.
When the feature development is complete, feature owner promotes
the feature branch to the integration branch and delete the FTE Environment.
And thats how the elasticity is in place for Feature Test
Environments.
Elasticity in Build & Deploy Infra
Yes, the build & deployment framework for FTEs are also
scalable & elastic in nature and that’s implemented using Jenkins + EC2
plugin. With this plugin, if Jenkins notices that the build or deployment
cluster is overloaded, it'll start instances using EC2 API and
automatically connect them as Jenkins slaves. When the load goes down,
excessive EC2 instances will be terminated. This setup allows you to maintain a
small in-house cluster. The FrontEnd application for
creating FTEs have an interface to the Jenkins which helps to trigger build
& deployment at the click of a button.
The Build & Deployment infrastructure is of three pre-fabricated AMIs
- Build
- Deployment
- NAS box which acts
as source code and artifact repository
If there are any failures during the deployment, it can be
triggered independently via Jenkins as well.
Slide Deck for the Tech Talk can be found here.
