Recently at work I needed to migrate some resources from one Terraform state file to another. I found useful information on this Stack Overflow answer but decided to document the procedure I used.

We use an S3 bucket and a DynamoDB table to manage Terraform state changes. This is used to compare with the tf source files in a repository to determine changes in AWS that are necessary. I needed to split some Terraform source files into separate repositories, but just moving the source will not work the way you might want. When you run terraform plan on the old repo, the files that you moved will still be in the state file, thus, Terraform thinks it needs to delete these and will attempt to do so. This means those resources will be removed if you apply the plan. You then need to run plan and apply in the new repository to create them in AWS again. This works if you don’t mind the AWS resources getting removed temporarily.

If you need to move Terraform source files but do not want the AWS resources removed, you have to manipulate the state files directly with Terraform. In this scenario, I’m moving Terraform to a new S3 bucket and want to move some of the resources therein. Note that the version of Terraform used must be consistent between the two repositories, at least while migrating.

1. Backup old state file

Go into S3 and download the file that contains your Terraform state. This is a fail safe if anything goes horribly wrong.

2. Pull the old state file into a local file for modification

1
2

terraform init '-input=false' -reconfigure
terraform state pull > ../../old.tfstate 

3. Get the list of modules from the old state

1
2

terraform init '-input=false' -reconfigure
terraform state list > ../../old.module_list

4. Create the new state file

You might need to create and delete an AWS resource to generate an empty state file. The empty state file should look something like this in S3:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

{
    "version": 3,
    "terraform_version": "0.11.11",
    "serial": 3,
    "lineage": "b63aef7c-1042-48eb-a162-ed8f1137a9c3",
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

5. Pull the new state file into a local file

1
2

terraform init '-input=false' -reconfigure
terraform state pull > ../../new.tfstate 

6. Edit the old.module_list file

This file contains all of the resources that are referenced in the old module state. You will have to edit this file and find the resources you want moved to the new state. Remove all of the lines that you want to remain in the old state file.

7. Migrate the modules from the old state file to the new one

Terraform will not move state from one remote file to another, which is why you have to retrieve the state files locally. Once you have them locally, you are able to move state from one to another using Terraform commands. This can be time consuming if you have to move several resources, so you can use a script like the following:

1
2
3
4
5
6
7
8

#!/bin/bash

filename="../../old.module_list"
cat $filename | while read line
do
    echo $line
    terraform state mv -state=../../old.tfstate -state-out=../../new.tfstate $line $line
done

Run this script and it will move all of the modules you left in the old.module_list file from the old state to the new state.

8. Push the local Terraform state to the S3 backend location

Run this from the new location

1
2

terraform init '-input=false' -reconfigure
terraform state push ../../new.tfstate 

Run this from the old location

1
2

terraform init '-input=false' -reconfigure
terraform state push ../../old.tfstate 

9. Check your Terraform plan

Run terraform plan from each location and verify there are no state changes present. This should happen if you’ve correctly moved over the modules to the new source location, and removed the terraform source from the old location.