The Scenario
Having used Teamcity and Octopus to deploy our AWS infrastructure and applications for the last few years we wanted to re evaluate the market. This particular article focus on an issue we had with GitLab and passing subnets to a cloudformation script.
The Challenge
To date we have used called ansible playbooks from Octopus to run our Cloudformation scripts. We had issues with this approach using Gitlab runners in that we could not install Ansible modules, but that’s another story. For our Proof Of Concept we reverted to calling Cloudformation natively from gitlab-ci.yml.
Using gitlab variables we wanted to pass all our key parameters including the subnet ranges. Using the parameter to define subnet groups for ELB’s or RDS we need to specify a minimum of 2 subnets. The two values are passed as a string for example
subnet-1234567890abcdef, subnet-fedcba0987654321
However passing above in Gitlab as variable converted the string to a list which was rejected by cloudformation.
Consider the script
infra:
stage: deploy
image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
variables:
VPC: "vpc-01234567891115566"
DATABASE_PORT: 1433
WEB_SERVER_PORT: 80
LOCAL_SUBNETS: subnet-1234567890abcdef,subnet-fedcba0987654321
DB_INSTANCE_TYPE: "t3.large"
script:
- aws --version
- echo "Create Infrastructure Natively with AWSCLI"
- echo "Assumes testgitlab already exists"
- aws s3 cp deploy/infrastructure/cloudformation/deployer.yml s3://testgitlab
- echo "Run Cloudformation Script"
- aws cloudformation create-stack --stack-name gitlab-ci --capabilities CAPABILITY_NAMED_IAM --template-url https://testgitlab.s3.eu-west-2.amazonaws.com/deployer.yml --parameters ParameterKey=VPC,ParameterValue=$VPC ParameterKey=DBPort,ParameterValue=$DATABASE_PORT ParameterKey=WebServerPort,ParameterValue=$WEB_SERVER_PORT ParameterKey=DatabaseSubnets,ParameterValue=$LOCAL_SUBNETS ParameterKey=DBInstanceType,ParameterValue=$DB_INSTANCE_TYPE
Gave the following error
Parameter validation failed:
Invalid type for parameter Parameters[7].ParameterValue, value: ['subnet-1234567890abcdef', 'subnet-fedcba0987654321c'], type: <class 'list'>, valid types: <class 'str'>
The Solution
The solution was a combination of double and single quotes, first its good to understand one of the differences between single and double quotes. Single quotes are considered ‘stronger’ and if used on variables will pass them literally. For this reason we needed to use double quotes when passing the parameter.
First enclose the passed variable in double quotes so it is passed as a string to CloudFormation, then enclose the whole thing in single quotes so the enclosed double quotes are passed and now it is treated as a string as opposed to a list
LOCAL_SUBNETS: '"subnet-1234567890abcdef,subnet-fedcba0987654321"'
References
We found the following blog posts and official guides invaluable for resolving this issue
What is the difference between single and double quotes