Terraform: State Fundamentals

Table of Contents

1. Module 1 — Overview: What Is State?
   • The Role of State in Terraform
   • Internal Structure of State Data
   • Lifecycle Scenarios
2. Module 2 — State Management Commands
   • terraform state — Subcommands
   • State Refresh and Drift Detection
   • Moving Resources — moved Block
   • Removing Resources — removed Block
3. Module 3 — Secure State Management
   • Remote Backends
   • Configuring an S3 Backend
   • State Locking
   • Terraform Workspaces
4. Module 4 — Importing Existing Resources
   • terraform import vs import Block
   • Generating Configuration with import
5. Module 5 — Reference Tables
   • Backend Comparison
   • terraform state Commands
   • terraform workspace Commands
6. Module 6 — Running Scenario: Starship Depot

Module 1 — Overview: What Is State?

The Role of State in Terraform

Terraform relies on 4 key components:

The state data is Terraform’s memory. It is used to:

• Map resources from the configuration to real objects in the target environment (e.g., aws_instance.server ↔ EC2 ID i-0xyz789)
• Detect diffs between the desired state (configuration) and the actual state (infrastructure)
• Decide on deletions: if a resource disappears from the config, Terraform destroys it using the state
• Store outputs defined in output blocks

Important: State data is stored in JSON. Never edit it by hand — use Terraform commands exclusively.

Diagram — Desired State vs Actual State

flowchart LR
    subgraph CONFIG["Configuration (.tf)"]
        direction TB
        C1["resource aws_instance.server\ninstance_type = t3.micro"]
        C2["resource aws_security_group.primary\nport 443 open"]
    end

    subgraph STATE["State Data (terraform.tfstate)"]
        direction TB
        S1["aws_instance.server\nid = i-0xyz789\ninstance_type = t3.nano ⚠️"]
        S2["aws_security_group.primary\nid = sg-0abc123"]
    end

    subgraph REAL["Actual Infrastructure (AWS)"]
        direction TB
        R1["EC2 i-0xyz789\ninstance_type = t3.nano"]
        R2["Security Group sg-0abc123"]
    end

    CONFIG -->|"terraform plan\ncompare"| STATE
    STATE -->|"refresh\nquery"| REAL
    CONFIG -->|"terraform apply\ncreate/modify/destroy"| REAL
    REAL -->|"actual state read"| STATE

    style S1 fill:#ffcccc,stroke:#cc0000

During a terraform plan:

1. Terraform loads the configuration and state into memory
2. It refreshes the state by querying the actual environment
3. It compares state vs configuration to generate a change plan

Internal Structure of State Data

{
  "version": 4,
  "terraform_version": "1.11.0",
  "serial": 12,
  "lineage": "b2c3d4e5-f6a7-8901-bcde-fg2345678901",
  "outputs": {
    "public_dns": {
      "value": "ec2-10-20-30-40.compute.amazonaws.com",
      "type": "string"
    }
  },
  "resources": [
    {
      "mode": "managed",
      "type": "aws_instance",
      "name": "server",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "id": "i-0xyz789abc123",
            "instance_type": "t3.micro",
            "ami": "ami-0fedcba9876543210"
          }
        }
      ]
    }
  ]
}

Key state fields:

Note: If a saved plan’s serial is lower than the current state serial, Terraform will refuse the apply (stale plan).

Lifecycle Scenarios

stateDiagram-v2
    [*] --> ConfigurationOnly : terraform init (empty state)
    ConfigurationOnly --> PlanGenerated : terraform plan -out=m1.tfplan
    PlanGenerated --> InfraProvisioned : terraform apply m1.tfplan
    InfraProvisioned --> ConfigModified : Modification in .tf
    ConfigModified --> PlanGenerated : terraform plan
    InfraProvisioned --> DriftDetected : Change outside Terraform
    DriftDetected --> StateRefreshed : terraform apply -refresh-only
    StateRefreshed --> ConfigModified : Update the config
    InfraProvisioned --> [*] : terraform destroy

Module 2 — State Management Commands

terraform state — Subcommands

# List all resource addresses in the state
terraform state list

# Show all attributes of a specific resource
terraform state show aws_instance.server

# Extract raw state content (JSON)
terraform state pull > state_backup.json

# Push a state manually (use with caution)
terraform state push state_backup.json

# Move a resource to a new address
terraform state mv aws_instance.server aws_instance.app

# Remove a resource from state (without destroying infra)
terraform state rm aws_instance.server

Supplementary commands:

# Display state in readable HCL format
terraform show

# Display state in standardized JSON (stable over time)
terraform show -json

# Display state outputs
terraform output

# Display outputs in JSON (includes data type)
terraform output -json

# Display a specific output without quotes
terraform output -raw public_dns

State Refresh and Drift Detection

Drift occurs when a resource is modified outside of Terraform (via the AWS console, CLI, etc.).

flowchart TD
    A["Terraform State\n(t3.micro)"] -->|"Drift detected"| B{"Drift type?"}
    B -->|"Approved drift\n(intended change)"| C["terraform apply -refresh-only\nthen update config"]
    B -->|"Unauthorized drift\n(unintended change)"| D["terraform plan\nthen terraform apply\nto revert to config"]
    B -->|"Ignored drift\n(temporarily tolerated)"| E["Ignore until\nnext cycle"]

    style C fill:#ccffcc,stroke:#009900
    style D fill:#ffcccc,stroke:#cc0000
    style E fill:#ffffcc,stroke:#999900

Commands for managing drift:

# Detect out-of-band changes (without modifying infra)
terraform plan -refresh-only

# Apply only the state update (without modifying infra)
terraform apply -refresh-only

# Example: changing t3.micro → t3.nano via AWS CLI
aws ec2 stop-instances --instance-ids $instance_id
aws ec2 wait instance-stopped --instance-ids $instance_id
aws ec2 modify-instance-attribute --instance-id $instance_id \
  --instance-type '{"Value": "t3.nano"}'
aws ec2 start-instances --instance-ids $instance_id

Moving Resources — moved Block

When refactoring code (renaming a resource label, moving into a module), you need to inform Terraform to avoid unnecessary destroy/recreate cycles.

Problem: Renaming aws_instance.server to aws_instance.app without a moved block → Terraform plans to destroy server and create app.

Solution — moved block (declarative, recommended):

# In main.tf or a dedicated moves.tf file
moved {
  from = aws_instance.server
  to   = aws_instance.app
}

Advantages of moved block vs terraform state mv:

Removing Resources — removed Block

To remove a resource from Terraform management without destroying it in the infrastructure:

# Remove the instance from state without destroying the EC2
removed {
  from = aws_instance.server

  lifecycle {
    destroy = false
  }
}

# Remove the security group from state without destroying it
removed {
  from = aws_security_group.primary

  lifecycle {
    destroy = false
  }
}

Imperative alternative (discouraged):

# Removes a resource from state — without a prior plan
terraform state rm aws_instance.server

Module 3 — Secure State Management

Remote Backends

The local backend (default) stores state in terraform.tfstate on the local filesystem. Critical limitations:

• State loss if the machine is lost or disk is corrupted
• Cannot be shared with a team or CI/CD pipeline

Remote backends solve these problems. Comparison of major backends:

flowchart LR
    TF["Terraform\nProcess"] --> B{Backend Type}
    B -->|"Remote\nStorage + Locking"| S3["AWS S3\n+ use_lockfile"]
    B -->|"Remote\nStorage + Locking"| AZ["Azure Blob\nStorage"]
    B -->|"Remote\nStorage + Locking"| GCS["Google Cloud\nStorage"]
    B -->|"Full Features\n+ Collaboration"| HCP["HCP Terraform\n(Terraform Cloud)"]
    B -->|"Local\nDev/Test only"| LOCAL["Local\nFilesystem"]

    style LOCAL fill:#ffeecc,stroke:#cc8800
    style HCP fill:#ccffcc,stroke:#009900

Comparing Backends

Configuring an S3 Backend

Step 1 — Create the S3 bucket to store state:

# s3_bucket_create/main.tf
provider "aws" {
  region = var.aws_region
}

resource "aws_s3_bucket" "rocket_depot" {
  bucket_prefix = "rocket-depot"
  force_destroy = true

  tags = {
    Environment = "terraform-demo"
    Purpose     = "State storage"
  }
}

# Enable versioning (important for state recovery)
resource "aws_s3_bucket_versioning" "rocket_depot_versioning" {
  bucket = aws_s3_bucket.rocket_depot.id
  versioning_configuration {
    status = "Enabled"
  }
}

# Server-side encryption (AES256)
resource "aws_s3_bucket_server_side_encryption_configuration" "rocket_depot_encryption" {
  bucket = aws_s3_bucket.rocket_depot.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

# Block public access
resource "aws_s3_bucket_public_access_block" "rocket_depot_pab" {
  bucket                  = aws_s3_bucket.rocket_depot.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

Step 2 — Configure the S3 backend (partial configuration):

# terraform.tf — Partial configuration (no hard-coded arguments)
terraform {
  required_version = ">= 1.11"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }

  backend "s3" {
    # Values are provided via -backend-config at init time
    # Input variables (var.*) CANNOT be used here
  }
}

Backend configuration file (s3.tfbackend):

# s3.tfbackend
bucket       = "rocket-depot-xyz999"
key          = "starship-depot/terraform.tfstate"
region       = "us-west-2"
use_lockfile = true

Initialization with partial configuration:

# Migrate to S3 backend using a config file
terraform init -backend-config="s3.tfbackend"

# Alternative — pass values directly on the command line
terraform init \
  -backend-config="bucket=rocket-depot-xyz999" \
  -backend-config="key=starship-depot/terraform.tfstate" \
  -backend-config="region=us-west-2" \
  -backend-config="use_lockfile=true"

Why don’t variables work in the backend block?
Terraform initializes the backend before evaluating variables and locals. That is why partial configurations or environment variables are used instead.

Diagram — Remote Backend S3 Architecture

flowchart TB
    subgraph DEVS["Team / CI-CD Pipeline"]
        D1["Dev 1\nterraform apply"]
        D2["Dev 2\nterraform plan"]
        CICD["CI/CD Pipeline\nterraform apply"]
    end

    subgraph AWS["AWS"]
        S3["S3 Bucket\n(State Data + Versioning\n+ Encryption)"]
        LOCK["S3 Lock File\n(.tflock)\n(use_lockfile=true)"]
        INFRA["Infrastructure\n(VPC, EC2, SG...)"]
    end

    D1 -->|"Acquires lock\nreads/writes state"| S3
    D1 -->|"Creates/modifies\nresources"| INFRA
    D2 -->|"Attempts to acquire lock\n❌ BLOCKED"| LOCK
    CICD -->|"Acquires lock\nwhen D1 finishes"| S3
    S3 --- LOCK

    style LOCK fill:#ffcccc,stroke:#cc0000
    style S3 fill:#ccffee,stroke:#009966

State Locking

Locking prevents two Terraform processes from accessing state simultaneously, avoiding corruption.

Locking behavior:

• terraform plan, terraform apply, terraform console all acquire a lock
• If a lock cannot be acquired → error with lock ID, date, and author
• If the process crashes without releasing the lock → use terraform force-unlock

# Force-release a lock (use with caution)
terraform force-unlock LOCK_ID

# Example error message during a lock conflict:
# Error: Error acquiring the state lock
# Lock Info:
#   ID:        b2c3d4e5-f6a7-8901-bcde-fg2345678901
#   Path:      starship-depot/terraform.tfstate
#   Operation: OperationTypePlan
#   Who:       operator@workstation
#   Created:   2025-03-10 14:00:00 UTC

S3 Backend — Locking evolution:

Terraform Workspaces

Workspaces allow managing multiple environments from the same root module configuration, each with its own state data.

flowchart LR
    ROOT["Root Module\n(same code)"]

    subgraph BACKEND["S3 Backend"]
        WS_DEFAULT["Workspace: default\nterraform.tfstate"]
        WS_DEV["Workspace: dev\nenv:/dev/terraform.tfstate"]
        WS_DEV2["Workspace: staging\nenv:/staging/terraform.tfstate"]
    end

    ROOT -->|"terraform workspace select default"| WS_DEFAULT
    ROOT -->|"terraform workspace select dev"| WS_DEV
    ROOT -->|"terraform workspace select staging"| WS_DEV2

    WS_DEFAULT -->|"manages"| INFRA_PROD["Infrastructure\ndefault/prod"]
    WS_DEV -->|"manages"| INFRA_DEV["Infrastructure\ndev"]
    WS_DEV2 -->|"manages"| INFRA_STG["Infrastructure\nstaging"]

    style WS_DEFAULT fill:#ccffcc,stroke:#009900
    style WS_DEV fill:#cceeff,stroke:#0066cc
    style WS_DEV2 fill:#eeccff,stroke:#6600cc

Using terraform.workspace in configuration:

# main.tf — Use the workspace to name the environment
locals {
  # If workspace is "default", use var.environment
  # Otherwise, use the workspace name
  env_name = terraform.workspace == "default" ? var.environment : terraform.workspace
}

resource "aws_instance" "server" {
  ami           = nonsensitive(data.aws_ssm_parameter.amzn2_linux.value)
  instance_type = var.instance_type

  tags = {
    Name = "starship-depot-server-${local.env_name}"
  }

  user_data = templatefile("${path.module}/templates/bootstrap.sh", {
    environment = local.env_name
  })
}

Limitations of Terraform Community Edition workspaces:

1. Shared security: All workspaces share the same backend → a developer can view/modify the production state
2. Code promotion: No native VCS integration → risk of applying dev-branch code to production
3. No RBAC: Non-granular access control per workspace

Recommendation: For long-lived isolated environments (dev/staging/prod), prefer separate configurations in distinct directories or repositories, or use HCP Terraform (Terraform Cloud) which provides RBAC and workspace isolation.

Module 4 — Importing Existing Resources

When infrastructure already exists (created manually or via another tool), it must be imported into Terraform state to begin managing it.

flowchart LR
    EXISTING["Existing Resource\n(outside Terraform)"]
    STATEEMPTY["Terraform State\n(empty)"]
    CONFIG["Configuration .tf\n(to create/write)"]

    EXISTING -->|"terraform import\nor import block"| STATEEMPTY
    CONFIG -->|"terraform plan\n(must match)"| STATEEMPTY

    subgraph AFTER["After Import"]
        STATEFULL["Terraform State\n(resource present)"]
        CONFIGFULL["Configuration .tf\n(corresponding)"]
    end

    STATEEMPTY --> STATEFULL
    CONFIG --> CONFIGFULL

terraform import vs import Block

Imperative method (legacy) — terraform import:

# Import an existing EC2 instance into state
terraform import aws_instance.server i-0xyz789abc123

# Import a security group
terraform import aws_security_group.primary sg-0abc123def456

# Limitations:
# - One resource at a time
# - Does not generate .tf configuration
# - Modifies state directly without a prior plan

Declarative method (recommended since Terraform 1.5) — import block:

# imports.tf
import {
  id = "i-0xyz789abc123"
  to = aws_instance.server
}

import {
  id = "sg-0abc123def456"
  to = aws_security_group.primary
}

# Execute the import via a predictable plan
terraform plan   # Shows what will be imported
terraform apply  # Imports the resources

Automatic configuration generation (Terraform 1.5+):

# Generate HCL configuration for imported resources
terraform plan -generate-config-out=generated.tf

This command inspects cloud resources and automatically generates the corresponding HCL resource blocks — eliminating the need to write configuration manually.

Advantages of import block vs terraform import:

Module 5 — Reference Tables

Backend Comparison

terraform state Commands

terraform workspace Commands

HCL Blocks Related to State

Module 6 — Running Scenario: Starship Depot

This module uses the Starship Depot scenario as a running example. Here is the project structure:

Configuration files (base_app/):

# terraform.tf — Provider and version requirements
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.11"
}

# To add when migrating to S3:
# backend "s3" {}  ← partial configuration

# main.tf — Main infrastructure
provider "aws" {
  region = var.region
}

# VPC module (terraform-aws-modules/vpc/aws)
module "networking" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "6.0.1"

  name           = var.network_info.vpc_name
  cidr           = var.network_info.vpc_cidr
  azs            = slice(data.aws_availability_zones.available.names, 0, ...)
  public_subnets = values(var.network_info.public_subnets)
}

# HTTP Security Group
resource "aws_security_group" "primary" {
  name   = "starship-depot-sg"
  vpc_id = module.networking.vpc_id

  ingress {
    from_port   = var.sg_port_https
    to_port     = var.sg_port_https
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# Data source: Amazon Linux 2 AMI
data "aws_ssm_parameter" "amzn2_linux" {
  name = "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2"
}

# EC2 instance
resource "aws_instance" "server" {
  ami                         = nonsensitive(data.aws_ssm_parameter.amzn2_linux.value)
  instance_type               = var.instance_type
  subnet_id                   = module.networking.public_subnets[0]
  vpc_security_group_ids      = [aws_security_group.primary.id]
  user_data_replace_on_change = true

  user_data = templatefile("${path.module}/templates/bootstrap.sh", {
    environment = var.environment
  })

  tags = {
    Name = "starship-depot-server-${var.environment}"
  }
}

Exercise workflow (complete):

# Module 1 — Initial deployment
cp -r base_app starship_depot
cd starship_depot
terraform init
terraform plan -out="m1.tfplan"
terraform apply "m1.tfplan"

# Module 2 — Drift management
terraform state list
terraform state show aws_instance.server
terraform plan -refresh-only         # Detect drift
terraform apply -refresh-only        # Sync state

# Module 2 — Moved block (after renaming in main.tf)
# Add to main.tf:
#   moved { from = aws_instance.server; to = aws_instance.app }
terraform plan   # Should show 0 infra changes

# Module 3 — Migration to S3
cd ../s3_bucket_create
terraform init && terraform apply -out="bucket.tfplan" && terraform apply "bucket.tfplan"
cd ../starship_depot
# Create s3.tfbackend with bucket/key/region/use_lockfile
terraform init -backend-config="s3.tfbackend"

# Module 3 — Workspaces
terraform workspace show           # default
terraform workspace new staging    # Creates and selects staging
terraform state list               # Empty (new state)
terraform apply                    # Deploy in staging
terraform workspace list           # default, * staging
terraform destroy -auto-approve    # Clean up staging
terraform workspace select default
terraform workspace delete staging
terraform destroy -auto-approve    # Clean up default

Key Takeaways

1. State data is Terraform’s memory — it maps configuration resources to real objects in the target environment.

2. Never edit the state JSON manually — use Terraform commands exclusively.

3. The local backend is only suitable for development — in production, use a remote backend (S3, Azure Blob, GCS, HCP Terraform).

4. Locking protects against concurrent corruption — use_lockfile = true for S3 (Terraform ≥ 1.11).

5. Prefer declarative blocks over imperative commands:

   • moved {} rather than terraform state mv
   • removed {} rather than terraform state rm
   • import {} rather than terraform import

6. Community Edition workspaces have significant security limitations — do not use them for critical long-lived environments without appropriate access controls.

7. terraform plan -refresh-only is a powerful drift detection tool in automated pipelines.

8. Partial configurations (backend "s3" {} without arguments) allow reusing Terraform configuration across multiple environments by injecting backend values at init time.

Search Terms

terraform · state · fundamentals · infrastructure · ci/cd · devops · backend · block · commands · resources · backends · diagram · management · remote