Senza¶
Senza is STUPS’ deployment tool to create and execute AWS CloudFormation templates in a sane way.
See the Deployment section for details on how to deploy applications using Senza, Pier One and Taupage.
Installation¶
Install or upgrade to the latest version of Senza () with PIP:
$ sudo pip3 install --upgrade stups-senza
Command Line Usage¶
Senza definitions can be bootstrapped conveniently to get started quickly:
$ senza init myapp.yaml
Cloud Formation stacks are created from Senza definitions with the create
command:
$ senza create myapp.yaml 1 0.1-SNAPSHOT
You can disable the automatic Cloud Formation rollback-on-failure in order to do ‘post-mortem’ debugging (e.g. on an EC2 instance):
$ senza create --disable-rollback myerroneous-stack.yaml 1 0.1-SNAPSHOT
You can pass parameters from yaml file.
$ senza create --parameter-file parameters.yaml myapp.yaml 1 0.1-SNAPSHOT
Stacks can be listed using the list
command:
$ senza list myapp.yaml # list only active stacks for myapp
$ senza list myapp.yaml --all # list stacks for myapp (also deleted ones)
$ senza list # list all active stacks
$ senza list --all # list all stacks (including deleted ones)
$ senza list "suite-.*" 1 # list stacks starting with "suite" and with version "1"
$ senza list ".*" 42 # list all stacks with version "42"
$ senza list mystack ".*test" # list all stacks for "mystack" with version ending in "test"
There are a few commands to get more detailed information about stacks:
$ senza resources myapp.yaml 1 # list all CF resources
$ senza events myapp.yaml 1 # list all CF events
$ senza instances myapp.yaml 1 # list EC2 instances and IPs
$ senza console myapp.yaml 1 # get EC2 console output for all stack instances
$ senza console 172.31.1.2 # get EC2 console output for single instance
Most commands take so-called STACK_REF arguments, you can either use an existing Senza definition YAML file (as shown above) or use the stack’s name and version, you can also use regular expressions to match multiple applications and versions:
$ senza inst # all instances, no STACK_REF argument given
$ senza inst mystack # list instances for all versions of "mystack"
$ senza inst mystack 1 # only list instances for "mystack" version "1"
$ senza inst "suite-.*" 1 # list instances starting with "suite" and with version "1"
$ senza inst ".*" 42 # list all instances with version "42"
$ senza inst mystack ".*test" # list all instances for "mystack" with version ending in "test"
Traffic can be routed via Route53 DNS to your new stack:
$ senza traffic myapp.yaml # show traffic distribution
$ senza traffic myapp.yaml 2 50 # give version 2 50% of traffic
Warning
Some clients use connection pools which - by default - reuse connections as long as there are requests to be processed. In this case senza traffic
won’t result in any redirection of the traffic, unfortunately. To force such clients to switch traffic from one stack to the other you might want to manually disable the load balancer (ELB) of the old stack, e.g. by changing the ELB listener port. This switches traffic entirely. Switching traffic slowly (via weighted DNS records) is only possible for NEW connections.
It is recommended to monitor the behavior of clients during traffic switching and if necessary to ask them to reconfigure their connection pools.
Stacks can be deleted when they are no longer used:
$ senza delete myapp.yaml 1
$ senza del mystack # shortcut: delete the only version of "mystack"
Available Taupage AMIs and all other used AMIs can be listed to check whether old, outdated images are still in-use or if a new Taupage AMI is available:
$ senza images
Tip
All commands and subcommands can be abbreviated, i.e. the following lines are equivalent:
$ senza list
$ senza l
Bash Completion¶
The programmable completion feature in Bash permits typing a partial command, then pressing the [Tab] key to autocomplete the command sequence. If multiple completions are possible, then [Tab] lists them all.
To activate bash completion for the Senza CLI, just run:
$ eval "$(_SENZA_COMPLETE=source senza)"
Put the eval line into your .bashrc
:
$ echo 'eval "$(_SENZA_COMPLETE=source senza)"' >> ~/.bashrc
Controlling Command Output¶
The Senza CLI supports three different output formats:
text
- Default ANSI-colored output for human users.
json
- JSON output of tables for scripting.
tsv
- Print tables as tab-separated values (TSV).
JSON is best for handling the output programmatically via various languages or jq (a command-line JSON processor). The text format is easy for humans to read, and “tsv” format works well with traditional Unix text processing tools, such as sed, grep, and awk:
$ senza list --output json | jq .
$ senza instances my-stack --output tsv | awk -F\\t '{ print $6 }'
Senza Definition¶
Senza definitions are Cloud Formation templates as YAML with added ‘components’ on top. A minimal Senza definition without any Senza components would look like:
Description: "A minimal Cloud Formation stack creating a SQS queue"
SenzaInfo:
StackName: example
Resources:
MyQueue:
Type: AWS::SQS::Queue
Tip
Use senza init
to quickly bootstrap a new Senza definition YAML for most common use cases (e.g. a web application).
During evaluation of the definition, mustache templating is applied with access to the rendered definition, including the SenzaInfo, SenzaComponents and Arguments key (containing all given arguments).
Senza Info¶
The SenzaInfo
key must always be present in the definition YAML and configures global Senza behavior.
Available properties for the SenzaInfo
section are:
StackName
- The stack name (required).
OperatorTopicId
- Optional SNS topic name or ARN for Cloud Formation notifications. This can used for example to send notifications about deployments to a mailing list.
Parameters
- Custom Senza definition parameters. This can be used to dynamically substitute variables in the Cloud Formation template.
SpotinstAccessToken
- The access token required to create Elastigroups (optional). See the Senza::Elastigroup component for more details. This property can be encrypted with KMS. It will be decrypted by the senza client to generate the Cloud Formation template. The expected format is: “senza:kms:AQICAH…r0lbg==” where “senza:kms:” is the chosen prefix and the remainder is the encrypted secret.
Tip
Follow the vendor’s instructions to Create an API token.
SpotinstAccountId
- Optional property that contains the target Spotinst account Id. For ex.
act-123c12d3
Senza will try to discover the target Spotinst account by looking up the one associated with the AWS account where the CloudFormation stack is being created. You can lookup the Spotinst account Id using the Spotinst console.
Warning
Spotinst account discovery is only possible with personal tokens. When using programmatic user tokens this property MUST be present or the deployment will fail.
Note
By default any HTML entities within a parameter will be escaped, this may cause some unexpected behaviour. In the event you need to workaround this use three braces either side of your argument evaluation e.g. {{{Arguments.ApplicationId}}}
# basic information for generating and executing this definition
SenzaInfo:
StackName: hello-world
Parameters:
- ApplicationId:
Description: "Application ID from kio"
- ImageVersion:
Description: "Docker image version of hello-world."
- MintBucket:
Description: "Mint bucket for your team"
- GreetingText:
Description: "The greeting to be displayed"
Default: "Hello, world!"
MinLength: "1"
MaxLength: "16"
# a list of senza components to apply to the definition
SenzaComponents:
# this basic configuration is required for the other components
- Configuration:
Type: Senza::StupsAutoConfiguration # auto-detect network setup
# will create a launch configuration and auto scaling group with scaling triggers
- AppServer:
Type: Senza::TaupageAutoScalingGroup
InstanceType: t2.micro
SecurityGroups:
- app-{{Arguments.ApplicationId}}
IamRoles:
- app-{{Arguments.ApplicationId}}
AssociatePublicIpAddress: false # change for standalone deployment in default VPC
TaupageConfig:
application_version: "{{Arguments.ImageVersion}}"
runtime: Docker
source: "stups/hello-world:{{Arguments.ImageVersion}}"
mint_bucket: "{{Arguments.MintBucket}}"
$ senza create example.yaml 3
Usage: __main__.py create [OPTIONS] DEFINITION VERSION [PARAMETER]...
Error: Missing parameter "ApplicationId"
$ senza create example.yaml 3 example latest mint-bucket
Generating Cloud Formation template.. OK
Creating Cloud Formation stack hello-world-3.. OK
The parameters can also be specified by name, which might come handy in complex scenarios with sizeable number of parameters, and also to make the command line more easily readable, for example:
$ senza create example.yaml 3 example MintBucket=<mint-bucket> ImageVersion=latest
Here, the ApplicationId
is given as a positional parameter, then the two
other parameters follow specified by their names. The named parameters on the
command line can be given in any order, but no positional parameter is allowed
to follow the named ones.
Note
The name=value
named parameters are split on first =
which makes it
possible to still include a literal =
in the value part. This also
means that if you have to include it in the parameter value, you need to
pass this parameter with the name, to prevent senza
from treating the
part of the parameter value before the first =
as the parameter name.
It is possible to pass any of the supported CloudFormation Properties such as AllowedPattern
, AllowedValues
,
MinLength
, MaxLength
and many others. Senza itself will not enforce these
but CloudFormation will evaluate the generated template and raise an exception
if any of the Properties is not met. For example:
$ senza create example.yaml 3 example latest mint-bucket "Way too long greeting"
Generating Cloud Formation template.. OK
Creating Cloud Formation stack hello-world-3.. EXCEPTION OCCURRED: An error occurred (ValidationError) when calling the CreateStack operation: Parameter 'GreetingText' must contain at most 15 characters
Traceback (most recent call last):
[...]
Any parameter may be given a default value using Default
attribute.
If a parameter was not specified on the command line (either as positional or
named one), the default value is used. It makes sense to always put all
parameters which have a default value at the bottom of the parameter
definition list, otherwise one will be forced to specify all the following
parameters using a name=value
as there would be no way to map them to
proper position.
There is an option to pass parameters from file. The file needs to be formatted in yaml.
$ senza create --parameter-file parameters.yaml example.yaml 3 1.0-SNAPSHOT
Here is an example of a parameter file.
ApplicationId: example-app-id
MintBucket: your-mint-bucket
You can also combine parameter file and parameters from command line, but you can’t have same parameter twice. The parameter can’t exist both on file and command line.
$ senza create --parameter-file parameters.yaml example.yaml 3 1.0-SNAPSHOT Param=Example1
AccountInfo¶
The following properties are also available in Senza templates.
{{AccountInfo.Region}}
: the AWS region where the stack is created. Ex: ‘eu-central-1’.
Note: in many places of a template, {“Ref” : “AWS::Region”} can also be used.
{{AccountInfo.AccountAlias}}
: the alias name of the AWS account: ex: ‘super-team1-account’
{{AccountInfo.AccountID}}
: the AWS account id: ex: ‘353272323354’
{{AccountInfo.TeamID}}
: the team ID. Ex: ‘super-team1’.
{{AccountInfo.Domain}}
: the AWS account domain: Ex: super-team1.net
Mappings¶
Mappings are essentially key-value pairs and behave exactly as CloudFormation Mappings. Use Mappings for Images
, ServerSubnets
or LoadBalancerSubnets
. An Example:
Mappings:
Images:
eu-west-1:
MyImage: "ami-123123"
# (..)
Image: MyImage
Senza Components¶
Components are predefined Cloud Formation snippets that are easy to configure and generate all the boilerplate JSON that is required by Cloud Formation.
All Senza components must be configured in a list below the top-level “SenzaComponents” key, the structure is as follows:
SenzaComponents:
- ComponentName1:
Type: ComponentType1
SomeComponentProperty: "some value"
- ComponentName2:
Type: ComponentType2
Note
Please note that each list item below “SenzaComponents” is a map with only one key (the component name).
The YAML “flow-style” syntax would be: SenzaComponents: [{CompName: {Type: CompType}}]
.
Senza::StupsAutoConfiguration¶
The StupsAutoConfiguration component type autodetects load balancer and server subnets by relying on STUPS’ naming convention (DMZ subnets have “dmz” in their name). It also finds the latest Taupage AMIs and defines the images which can be used by the “TaupageAutoScalingGroup” component.
Example usage:
SenzaComponents:
- Configuration:
Type: Senza::StupsAutoConfiguration
This component supports the following configuration properties:
AvailabilityZones
- Optional list of AZ names (e.g. “eu-west-1a”) to filter subnets by. This option is relevant for attaching EBS volumes as they are bound to availability zones.
This components adds the following images:
LatestTaupageImage
- Latest Taupage AMI, for use in production deployments. Selected by default in the “TaupageAutoScalingGroup” component.
LatestTaupageStagingImage
- Staging Taupage AMI, for testing compatibility with the new Taupage releases. Should not be used for production deployments!
LatestTaupageDevImage
- Latest build of the Taupage AMI. Should not be used unless you’re working on Taupage or its components.
Senza::TaupageAutoScalingGroup¶
The TaupageAutoScalingGroup component type creates one AWS AutoScalingGroup resource with a LaunchConfiguration for the Taupage AMI.
SenzaComponents:
- AppServer:
Type: Senza::TaupageAutoScalingGroup
InstanceType: t2.micro
SecurityGroups:
- app-myapp
ElasticLoadBalancer: AppLoadBalancer
TaupageConfig:
runtime: Docker
source: pierone.example.org/foobar/myapp:1.0
ports:
8080: 8080
environment:
FOO: bar
This component supports the following configuration properties:
InstanceType
- The EC2 instance type to use.
SpotPrice
- Maximum amount of US dollars you want to spent per hour for a given instance type. See Spot Instances.
SecurityGroups
- List of security groups to associate the EC2 instances with. Each list item can be either an existing security group name or ID.
IamInstanceProfile
- ARN of the IAM instance profile to use. You can either use “IamInstanceProfile” or “IamRoles”, but not both.
IamRoles
- List of IAM role names to use for the automatically created instance profile.
Image
- AMI to use, defaults to
LatestTaupageImage
. If you want to use a different AMI, you have to create a Mapping for it. ElasticLoadBalancer
- Name of the ELB resource. Specifying the ELB resource will automatically use the “ELB” health check type for the auto scaling group.
This property also allows attaching multiple load balancers to the Auto Scaling Group by using a list instead of string, e.g.
ElasticLoadBalancer: [LB1, LB2]
. HealthCheckType
- How the auto scaling group should perform instance health checks. Value can be either “EC2” or “ELB”.
Default is “ELB” if
ElasticLoadBalancer
is set and “EC2” otherwise. HealthCheckGracePeriod
- The length of time in seconds after a new EC2 instance comes into service that Auto Scaling starts checking its health.
TaupageConfig
- Taupage AMI config, see Taupage for details.
At least the properties
runtime
(“Docker”) andsource
(Docker image) are required. Usually you will want to specifyports
andenvironment
too. AssociatePublicIpAddress
- Whether to associate EC2 instances with a public IP address. This boolean value (true/false) is false by default.
BlockDeviceMappings
- Specify additional EBS Devices you want to attach to the nodes. See for Option Map below.
AutoScaling
- Map of auto scaling properties, see below.
AutoScaling
AutoScaling
properties are:
Minimum
- Minimum number of instances to spawn.
Maximum
- Maximum number of instances to spawn.
DesiredCapacity
- Desired number of instances to spawn.
SuccessRequires
- During startup of the stack, define when your ASG is considered healthy by CloudFormation. Defaults to one healthy instance within 15 minutes. To change it to 4 healthy instances within 1 hour, 20 minutes and 30 seconds pass “4 within 1h20m30s” (you can omit hours/minutes/seconds as you please). Values that look like integers will be used as healthy instance count, e.g. “2” would be interpreted as 2 healthy instances within the default timeout of 15 minutes.
MetricType
- Metric to do auto scaling on. This will create automatic Alarms in Cloudwatch for you. If supplied, must be either
CPU
,NetworkIn
orNetworkOut
. If not supplied, you’re Auto Scaling Group will not dynamically scale and you have to define you’re own alerts. ScaleUpThreshold
- On which value of the metric to scale up. For the “CPU” metric: a value of 70 would mean 70% CPU usage. For network metrics a value of 100 would mean 100 bytes, but you can pass the unit (KB/GB/TB), e.g. “100 GB”.
ScaleDownThreshold
- On which value of the metric to scale down. For the “CPU” metric: a value of 40 would mean 40% CPU usage. For network metrics a value of 2 would mean 2 bytes, but you can pass the unit (KB/GB/TB), e.g. “2 GB”.
ScalingAdjustment
- How many instances are added/removed per scaling action. Defaults to 1.
Cooldown
:- After a scaling action occured, do not scale again for this amount of time in seconds. Defaults to 60 (one minute).
Statistic
- Which statistic to track in order to decide when scaling thresholds are met. Defaults to “Average”, can also be “SampleCount”, “Sum”, “Minimum”, “Maximum”.
Period
- Period over which statistic is calculated (in seconds), defaults to 300 (five minutes). It can be 10, 30 or multiples of 60 seconds.
EvaluationPeriods
- The number of periods over which data is compared to the specified threshold. Defaults to 2.
BlockDeviceMappings
BlockDeviceMappings
properties are:
DeviceName
- For example: /dev/xvdk
Ebs
- Map of EBS Options, see below.
Ebs
properties are:
VolumeSize
- How Much GB should this EBS have?
Spot Instances¶
To save money you can choose to use AWS spot instance, instead of using on demand instances. To choose the right instance type and pay up to the current price of an on demand instance you can search AWS instance prices list. This block will buy a c4.large instance for up to $0.134 per hour.
SenzaComponents:
- AppServer:
Type: Senza::TaupageAutoScalingGroup
InstanceType: c4.large
SpotPrice: 0.134
Senza also supports Spotinst’s Elastigroup. For details how that works with senza read the section Senza::Elastigroup
Senza::Elastigroup¶
The Elastrigoup component type creates an Elastigroup. It’s the equivalent of an Auto Scaling Group, but managed externally by a third party - Spotinst.
Tip
To be able to use Elastigroups you need to specify the SpotinstAccessToken
property from the Senza Info section.
Quote from the vendor: “Spotinst Elastigroup predicts EC2 Spot behavior, capacity trends, pricing, and interruptions rate. Whenever there’s a risk of interruption, Elastigroup acts accordingly to balance capacity up to 15 minutes ahead of time, ensuring 100% availability.”
SenzaComponents:
- AppServer:
Type: Senza::Elastigroup
InstanceType: "c5.large"
SpotAlternatives:
- "m5.large"
- "c5.xlarge"
SecurityGroups: app-hello-world
IamRoles:
- app-hello-world
ElasticLoadBalancerV2: AppLoadBalancer
TaupageConfig:
runtime: Docker
source: pierone.example.org/foobar/myapp:1.0
ports:
8080: 8080
environment:
FOO: bar
This component accepts ALL of the properties of the Senza::TaupageAutoScalingGroup
component. They are mapped to the corresponding attributes of the Elastigroup.
This includes the AutoScaling
properties.
It adds the following additional properties:
SpotAlternatives
- The EC2 instance types that should be used as Spot instead of the On-Demand
InstanceType
. The selection of which ones are effectivelly used is controlled by the Elastigroup cluster orientation. The default setting is “Balanced”. If this property is not set, the same instance type as the On-DemandInstanceType
is set as the single Spot alternative.
Note
If this property is set, the instance type specified in InstanceType
will not be considered for the Spot alternatives. This is by design, in order to allow
users to specify a type on instance different from the ones used for Spot. To allow
the same instance type in Spot or On-Demand that type will have to be present in both
properties.
Elastigroup
- This is the, optional, raw specification of the Elastigroup. Please refer to the
vendor documentation for the full specification of the Elastigroup Create API.
The content of this property is copied to the
group
attribute of the API. Please read below for details about precedence of conflicting settings.
Senza will try to mix and match Senza::TaupageAutoScalingGroup properties with the
Elastigroup. Raw definitions inside the Elastigroup
property take precedence
and will be left untouched. For example, if the Senza file contains:
SenzaComponents:
- AppServer:
Type: Senza::Elastigroup
InstanceType: "c5.large"
SpotAlternatives:
- "m5.large"
- "c5.xlarge"
Elastigroup:
compute:
instanceTypes:
ondemand: "m4.large"
spot:
- "m4.xlarge"
- "m4.2xlarge"
The effective setting will be to use m4.large as On-Demand and m4.xlarge and
m4.2xlarge as the spot alternatives. The InstanceType
and SpotAlternatives
properties are ignored.
This is the behavior of all the remaining properties that can be also set in the
Elastigroup
property.
Senza::WeightedDnsElasticLoadBalancer¶
The WeightedDnsElasticLoadBalancer component type creates one HTTPs ELB resource with Route 53 weighted domains.
The SSL certificate name used by the ELB can either be given (SSLCertificateId
) or is autodetected.
You can specify the main domain (MainDomain
) or the default Route53 hosted zone is used for the domain name.
By default, an internal load balancer is created. This is different from the AWS default behaviour. To create an internet-facing
ELB, explicitly set the Scheme
to internet-facing
.
SenzaComponents:
- AppLoadBalancer:
Type: Senza::WeightedDnsElasticLoadBalancer
HTTPPort: 8080
SecurityGroups:
- app-myapp-lb
The WeightedDnsElasticLoadBalancer component supports the following configuration properties:
HTTPPort
- The HTTP port used by the EC2 instances.
HealthCheckPath
- HTTP path to use for health check (must return 200), e.g. “/health”
HealthCheckPort
- Optional. Port used for the health check. Defaults to
HTTPPort
. SecurityGroups
- List of security groups to use for the ELB. The security groups must allow SSL traffic.
MainDomain
- Main domain to use, e.g. “myapp.example.org”
VersionDomain
- Version domain to use, e.g. “myapp-1.example.org”. You can use the usual templating feature to integrate the stack version, e.g.
myapp-{{SenzaInfo.StackVersion}}.example.org
. Scheme
- The load balancer scheme. Either
internal
orinternet-facing
. Defaults tointernal
. SSLCertificateId
- Name or ARN ID of the uploaded SSL/TLS server certificate to use, e.g.
myapp-example-org-letsencrypt
orarn:aws:acm:eu-central-1:123123123:certificate/abcdefgh-ijkl-mnop-qrst-uvwxyz012345
. You can check available IAM server certificates withaws iam list-server-certificates
. For ACM Certificate you must useaws acm list-certificates
Additionally, you can specify any of the valid AWS Cloud Formation ELB properties (e.g. to overwrite Listeners
).
Cross-Stack References¶
Traditional CloudFormation templates only allow to reference resouces that are located in the same template. This can be quite limiting. To compensate Senza selectively supports special cross-stack references in some places in your template, e.g. in SecurityGroups and IamRoles:
AppServer:
Type: Senza::TaupageAutoScalingGroup
InstanceType: c4.xlarge
SecurityGroups:
- Stack: base-1
LogicalId: ApplicationSecurityGroup
IamRoles:
- Stack: base-1
LogicalId: ApplicationRole
These references allow for having an additional special stack per application that defines common security groups and IAM roles that are shared across different versions (in contrast to using senza init).
Another use case for cross-stack references if one needs to access outputs from other stacks inside the TaupageConfig:
# database.yaml
..
Outputs:
DatabaseHost:
Value:
"Fn::GetAtt": [Database, Endpoint.Address]
# service.yaml
..
TaupageConfig:
environment:
DB_HOST:
Stack: exchange-rate-database-2
Output: DatabaseHost