This
section serves as a guide line for some usage of terraform to make life
easier. This is built from hard experience using terraform in a
multi-member team and across multiple environments in production.
Modules
are the reusable construct in terraform. In most cases, we want to make
a module out of ANY provisioning process that we expect to reuse.
In addition to reuse, modules provide a repeatable structure and versioning mechanism.
For instance, let’s look at our VPC module. It does the following:
- Creates a VPC
- Creates two subnets in that VPC in each AZ - one public/one private
- Creates a private Route53 zone for that VPC
- Creates basic security groups for that VPC
- Creates dhcp options for nodes provisioned in that VPC
- Provisions NAT nodes in each AZ for outbound traffic from the private subnets
This
is a codified and repeatable mechanism for our base AWS environment and
best practices for that environment. By using this module, you get all
of the basics required to create an isolated environment in AWS for
free.
Additionally, as this is a git repo for this module, you can
get VERSIONED environments. Consider the following module declaration
in terraform code:
By specifically putting the ref in there, you can be sure
that FUTURE changes to the module will not unintentionally propogate
down to existing environments. You can easily test changes by spinning
up the old version, bumping the rev, calling
terraform get --update=true
and then
terraform plan
.
While
terraform TRIES to do the right thing here, being strict in your
versions lets you know for sure that even if you unintentially issued
--update=true
, you would not accidentally tear down a stack.
Note
Always use versioned module sources in terraform. Always. This is required for determinism and repeatability.
Terraform provides another mechanism for a form of reusability -
remote state.
Looking
back out our pushbutton setup, we leverage remote state to help isolate
independent infrastucture components. There is no reason creating a new
instance needs to modify a VPC. There is no reason, creating an RDS
instance needs to modify dhcp options.
You _can_ use non-remote
state files but those are not currently allowed in terraform for reuse.
We use artifactory as our universal remote state mechanism because it
provides built in versioning for us.
In our pushbutton runs, EACH
subdirectory pushes its state up to artifactory at the end of the run.
That makes it usable for downstream modules. The following is an example
of using the VPC remote state for an environment downstream:
This allows us to reference outputs from that remote state:
Note
In terraform 0.7 (unreleased), the syntax for using values from remote state is changing slightly. Instead of terraform_remote_state.vpc.output.some_output
, the extra output
is being dropped:terraform_remote_state.vpc.some_output
Additionally, 0.7 is adding a new construct called data sources which
are immutable, read-only sources of data than can be used in terraform
runs - i.e. a json file with a list of current amis or versions of
software.
One
area that is painful in terraform is passing variables from modules and
other things. This leads to a bit of code duplication however the
tradeoffs for safety are worth it.
Use consistent variable input names across modules to help minimize confusion. A good example is our
orgname
variable.
This is used in EVERY module. This provides a programatic way to build
tags and names for AWS resources as well as ensures a factor of
uniqueness during provisioning. When you have multiple “stacks” in a
single AWS account, it’s very easy to see _WHICH_ environment a resource
goes with as all names are prefixed with the
orgname
(i.e.
nat-a.orgname.local
)
Also,
modules to not natively export all outputs from a given resource. For
this reason and for future unknowns, you should export ALL outputs from a
resource inside a module. Consider the following resource and output in
a module:
In this module, we’re ONLY exporting the vpc id. However,
what we should be doing is exporting everything that terraform returns
as an output in case we need that information in future reuses of the
module (originally this resource did not support the additional outputs.
They were added to terraform later. This code only exports older
outputs):
You can find the exported attributes of a given resource at the bottom of any resource’s page on the
terraform website
Note
Doing a terraform apply
will
update any remote state with new outputs from modules used assuming a
new version of the module was pulled in. Let’s say we needed to get
those new attributes for a new terraform use elsewhere. Following our
best practices we would:
- Make changes to the git repo of the module and note the new hash
- Update the rev in our downstream use of the module
- call
terraform get --update=true
to pull in the new version of the module
- run
terraform plan
to ensure that we aren’t actually CHANGING anything (or maybe your module allows that - it’s situational)
- run
terraform apply
which will grab the new exported outputs from the module and shove them into the current terraform run’s state.
Now we have a remote state file with additional information that we require when using remote state.
Using
the count trick is really handy in terraform when you need multiple
copies of something. Be aware, however, that counts “break” the
dependency graph. For instance let’s assume you have an aws_ec2_instance
resource and you use a count to provision 3
of
them. Now let’s assume that you ALSO use that count inside a dependent
resource (like a broker id in kafka) and that is a dependency on that
instance resource.
When you increase the count, the planner recomputes EVERYTHING using that count variable.
If
you must use a count, you should ensure that the count is not used in
ANY other resources unless you can safely update those resources as well
- this includes passing the same _count_ to a template that you use in
the instance resource.
Terraform
brings a much needed true infrastructure-as-code approach to things. It
is important, however, to not try and do too much in terraform. This is
the reason we wrap terraform in Rundeck. Rundeck allows us to wrap
terraform with steps that don’t FIT in terraform to create a final
deliverable.
Terraform is a really poor fit for application deployment and volatile lifecycle components.
My general rule is if something feels hard in terraform it probably is.
taint will destroy and re-create resource
Terraforming is a discovery utility to build terraform tf and state files from a running AWS infrastructure.
https://github.com/dtan4/terraforming
"terraform plan" should show you no changes after that.
Example of merging terraform state with existing infrastructure
Disable remote state
Pull routing data
Update current state file
Upload state to remote location