Gitlab and Cloudformation – quotes and subnets

The Scenario

Having used Teamcity and Octopus to deploy our AWS infrastructure and applications for the last few years we wanted to re evaluate the market.  This particular article focus on an issue we had with GitLab and passing subnets to a cloudformation script.

The Challenge

To date we have used called ansible playbooks from Octopus to run our Cloudformation scripts.  We had issues with this approach using Gitlab runners in that we could not install Ansible modules, but that’s another story.  For our Proof Of Concept we reverted to calling Cloudformation natively from gitlab-ci.yml.

Using gitlab variables we wanted to pass all our key parameters including the subnet ranges.  Using the parameter to define subnet groups for ELB’s or RDS we need to specify a minimum of 2 subnets.  The two values are passed as a string for example

subnet-1234567890abcdef, subnet-fedcba0987654321

However passing above in Gitlab as variable converted the string to a list which was rejected by cloudformation.

Consider the script


infra:
stage: deploy
image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
variables:
VPC: "vpc-01234567891115566"
DATABASE_PORT: 1433
WEB_SERVER_PORT: 80
LOCAL_SUBNETS: subnet-1234567890abcdef,subnet-fedcba0987654321
DB_INSTANCE_TYPE: "t3.large"
script:
- aws --version
- echo "Create Infrastructure Natively with AWSCLI"
- echo "Assumes testgitlab already exists"
- aws s3 cp deploy/infrastructure/cloudformation/deployer.yml s3://testgitlab
- echo "Run Cloudformation Script"
- aws cloudformation create-stack --stack-name gitlab-ci --capabilities CAPABILITY_NAMED_IAM --template-url https://testgitlab.s3.eu-west-2.amazonaws.com/deployer.yml --parameters ParameterKey=VPC,ParameterValue=$VPC ParameterKey=DBPort,ParameterValue=$DATABASE_PORT ParameterKey=WebServerPort,ParameterValue=$WEB_SERVER_PORT ParameterKey=DatabaseSubnets,ParameterValue=$LOCAL_SUBNETS ParameterKey=DBInstanceType,ParameterValue=$DB_INSTANCE_TYPE

Gave the following error

Parameter validation failed:
Invalid type for parameter Parameters[7].ParameterValue, value: ['subnet-1234567890abcdef', 'subnet-fedcba0987654321c'], type: <class 'list'>, valid types: <class 'str'>

The Solution

The solution was a combination of double and single quotes, first its good to understand one of the differences between single and double quotes.  Single quotes are considered ‘stronger’ and if used on variables will pass them  literally.  For this reason we needed to use double quotes when passing the parameter.

First enclose the passed variable in double quotes so it is passed as a string to CloudFormation, then enclose the whole thing in single quotes so the enclosed double quotes are passed and now it is treated as a string as opposed to a list

LOCAL_SUBNETS: '"subnet-1234567890abcdef,subnet-fedcba0987654321"'

 

References

We found the following blog posts and official guides invaluable for resolving this issue

What is the difference between single and double quotes

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *