Terraform: Providers

Level: Intermediate
Target platform: AWS (main examples), Azure, Google Cloud

Table of Contents

1. Provider Fundamentals
   • Provider Architecture
   • Provider Elements
   • Defining Provider Requirements (required_providers)
   • Installing a Provider (terraform init)
   • Useful Commands
2. Adding a Provider
   • Terraform Public Registry
   • Provider Tiers
   • Adding the random Provider
   • Client Options: Mirror and Local Cache
   • Semantic Versioning and Version Constraints
   • Updating a Provider
   • Dependency Lock File
3. Authenticating to Providers
   • Providers for the Three Major Clouds
   • Authentication Methods
   • Troubleshooting Providers
4. Using Multiple Providers
   • Provider Instances and alias
   • Creating a Provider Alias (Multi-Region)
   • Multi-Account AWS: IAM Profiles
   • Multi-Account AWS: assume_role
   • configuration_aliases in Modules
   • Providers and Workspaces
5. Quick Reference
   • Popular Providers
   • Version Constraints
   • Logging Environment Variables

1. Provider Fundamentals

Provider Architecture

A Terraform provider plays the role of a translation layer between the Terraform language (resources, data sources, execution plan) and the REST APIs of a target platform (AWS, Azure, GCP, etc.).

flowchart LR
    subgraph Terraform["Terraform Core (binary)"]
        CFG[Configuration .tf]
        PLAN[Execution Plan]
        GRAPH[Dependency Graph]
    end

    subgraph PluginProtocol["Plugin Protocol (gRPC)"]
        direction TB
        GPRC[gRPC / go-plugin]
    end

    subgraph Provider["Provider Plug-in (Go binary)"]
        direction TB
        SDK[HashiCorp Provider SDK]
        RES[Resources / Data Sources]
        FUNC[Functions / Ephemeral Resources]
    end

    subgraph Platform["Target Platform"]
        API[REST APIs\nCRUD Operations]
    end

    Terraform -- "RPC calls" --> PluginProtocol
    PluginProtocol --> Provider
    Provider --> Platform

Communication: Terraform Core launches the provider binary as a child process and communicates via gRPC (HashiCorp’s go-plugin protocol). Each Terraform operation (plan, apply, destroy) triggers RPC calls to the plugin.

Provider Elements

Defining Provider Requirements (required_providers)

Two discovery methods exist:

required_providers block syntax:

terraform {
  required_version = ">= 1.11"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }

    random = {
      source  = "hashicorp/random"
      version = "~> 3.7.0"
    }
  }
}

Source address format:

[hostname/]namespace/type

Diagram — Provider Registry Flow:

flowchart TD
    INIT[terraform init]
    LOCK{.terraform.lock.hcl\nexists?}
    REQ[Reads required_providers]
    REG[Terraform Public Registry\nregistry.terraform.io]
    PRIV[Private Registry\nor File Mirror]
    CACHE[.terraform/providers/\nlocal cache]
    LOCKW[Writes/Updates\n.terraform.lock.hcl]

    INIT --> REQ
    REQ --> LOCK
    LOCK -- No --> REG
    LOCK -- Yes\n(version OK) --> CACHE
    REG -->|source = hashicorp/xxx| CACHE
    PREG[private registry configured] -->|hostname present| PRIV
    PRIV --> CACHE
    CACHE --> LOCKW

Installing a Provider (terraform init)

# Standard initialization
terraform init

# Force update of providers
terraform init -upgrade

During initialization, Terraform:

1. Reads the required_providers block
2. Checks the .terraform.lock.hcl file
3. Downloads binaries to .terraform/providers/

Useful Commands

# Terraform version + installed providers
terraform version

# List providers used and their constraints
terraform providers

# Full JSON schema of all providers (useful with jq or LLM)
terraform providers schema -json

# Create a local mirror of providers
terraform providers mirror <target_directory>

# Lock providers (updates the lock file)
terraform providers lock

2. Adding a Provider

Terraform Public Registry

The public registry (registry.terraform.io) hosts providers, modules, Sentinel policies, and run tasks. All providers are free and their source code is available on GitHub.

Provider Tiers

flowchart LR
    OFF["Official\n(HashiCorp namespace)\nEx: hashicorp/aws\nhashicorp/azurerm\nhashicorp/google"]
    PART["Partner\n(GitHub org namespace)\nEx: oracle/oci\ndatabricks/databricks"]
    COMM["Community\n(GitHub account namespace)\nEx: DeviaVir/gsuite\njakefhyde/vault-namespaces"]
    OFF --> |"Maintained by HashiCorp"| PROD[Production Ready]
    PART --> |"Maintained by partner"| PROD
    COMM --> |"Variable support"| CAUTION[Evaluate before using]

Adding the random Provider

Example: adding the random v3.7.x provider to generate a unique suffix for an S3 bucket.

1. Declare in terraform.tf:

terraform {
  required_version = ">= 1.11"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }

    random = {
      source  = "hashicorp/random"
      version = "~> 3.7.0"
    }
  }
}

2. Use a provider resource:

resource "random_string" "bucket_suffix" {
  length  = 12
  special = false
  upper   = false
}

resource "aws_s3_bucket" "flow_logs" {
  bucket = "my-app-flow-logs-${random_string.bucket_suffix.result}"
}

Client Options: Mirror and Local Cache

For environments without internet access or with limited bandwidth:

Create a filesystem mirror:

terraform providers mirror /path/to/mirror

Configure the Terraform client (~/.terraformrc on Linux/macOS, %APPDATA%/terraform.rc on Windows):

provider_installation {
  filesystem_mirror {
    path    = "/path/to/mirror"
    include = ["registry.terraform.io/hashicorp/*"]
  }

  network_mirror {
    url     = "https://my-mirror.example.com/providers/"
    include = ["registry.terraform.io/hashicorp/*"]
  }

  # Fallback to public registry for everything else
  direct {
    exclude = ["registry.terraform.io/hashicorp/*"]
  }
}

Semantic Versioning and Version Constraints

Providers follow Semantic Versioning (MAJOR.MINOR.PATCH):

Version constraint operators table:

flowchart LR
    C1["~> 3.7.0\n≥ 3.7.0 and < 3.8.0\n(patch updates)"]
    C2["~> 3.7\n≥ 3.7 and < 4.0\n(minor updates)"]
    C3[">= 3.0, < 5.0\nExplicit range"]
    style C1 fill:#d4edda
    style C2 fill:#fff3cd
    style C3 fill:#d1ecf1

Updating a Provider

# 1. Modify the version constraint in terraform.tf
# 2. Run the upgrade
terraform init -upgrade

# 3. Test the plan
terraform plan

# 4. Review release notes on GitHub before apply
# 5. Commit the updated .terraform.lock.hcl
git add .terraform.lock.hcl
git commit -m "upgrade: random provider to ~> 3.7.0"

Note: -upgrade updates all providers at once. For major updates, changing one provider at a time makes debugging easier.

Dependency Lock File

The .terraform.lock.hcl file guarantees build reproducibility by fixing exact provider versions:

# .terraform.lock.hcl (example automatically generated)
provider "registry.terraform.io/hashicorp/aws" {
  version     = "6.13.0"
  constraints = "~> 6.0"
  hashes = [
    "h1:abc123...",
    "zh:def456...",
  ]
}

Best practice: Commit .terraform.lock.hcl to source control. Do not commit the .terraform/ folder.

3. Authenticating to Providers

Providers for the Three Major Clouds

flowchart TD
    subgraph AWS["Amazon Web Services"]
        AWSP["hashicorp/aws\n(main provider)"]
        AWSCC["hashicorp/awscc\n(Cloud Control API)\nv1.0+ in 2024"]
    end
    subgraph Azure["Microsoft Azure"]
        AZR["hashicorp/azurerm\n(Resource Manager)"]
        AZAD["hashicorp/azuread\n(Azure AD)"]
        AZST["hashicorp/azurestack\n(Azure Stack)"]
        AZAPI["azure/azapi\n(Direct ARM API)"]
    end
    subgraph GCP["Google Cloud Platform"]
        GCPP["hashicorp/google\n(main provider)"]
        GCPB["hashicorp/google-beta\n(beta features)"]
    end

Authentication Methods

flowchart TD
    AUTH[Provider\nAuthentication]
    CLI[1. CLI Tool\nAWS CLI / az CLI / gcloud]
    ENV[2. Environment Variables\nAWS_ACCESS_KEY_ID\nARM_CLIENT_ID\nGOOGLE_CREDENTIALS]
    TFVAR[3. Terraform Variables\narguments in provider block]
    MACHINE[4. Machine Identity\nIAM Role / Managed Identity\nService Account]
    OIDC[5. OIDC\nOpenID Connect\nCI/CD pipelines]

    AUTH --> CLI
    AUTH --> ENV
    AUTH --> TFVAR
    AUTH --> MACHINE
    AUTH --> OIDC

1. CLI (recommended for local development):

# AWS
aws configure
aws sso login

# Azure
az login

# Google Cloud (specific command for applications)
gcloud auth application-default login

2. Environment variables:

3. Arguments in the provider block (with Terraform variables):

provider "aws" {
  region     = var.region
  access_key = var.aws_access_key
  secret_key = var.aws_secret_key
}

Never hardcode credentials in .tf files. Use environment variables or a secrets manager (Vault, AWS Secrets Manager).

4. Machine Identity (recommended in production):

# AWS — use CLI profile
provider "aws" {
  region  = var.region
  profile = "production"
}

Troubleshooting Providers

Logging environment variables:

# Example: debugging a provider issue
export TF_LOG=DEBUG
export TF_LOG_PATH=/tmp/terraform-debug.log
terraform plan

Common issues:

4. Using Multiple Providers

Provider Instances and alias

A provider can have multiple instances to manage:

• Multiple regions (before AWS provider v6)
• Multiple accounts / subscriptions
• Multiple clusters (Kubernetes)
• Multiple sets of credentials

flowchart LR
    subgraph root["Root Module"]
        PDEF["provider aws { }\n(default instance\nus-west-2)"]
        PDR["provider aws {\n  alias = 'dr'\n  region = us-east-2\n}"]
        PSEC["provider aws {\n  alias = 'security'\n  assume_role { ... }\n}"]
    end

    subgraph resources["Resources"]
        R1["aws_vpc.prod\n→ default provider"]
        R2["module.dr_vpc\n→ provider = aws.dr"]
        R3["module.prod_s3_bucket\n→ providers = {\n    aws = aws.security\n    aws.vpc_account = aws\n  }"]
    end

    PDEF --> R1
    PDR --> R2
    PSEC --> R3

Rule: Only one provider block without alias (default instance). Each additional instance must have a unique alias.

Creating a Provider Alias (Multi-Region)

# providers.tf
provider "aws" {
  region = var.region          # us-west-2 — default instance (prod)
}

provider "aws" {
  alias  = "dr"
  region = var.dr_region       # us-east-2 — DR instance
}

Usage in a module (dr.tf):

module "dr_vpc" {
  source       = "./modules/vpc"
  environment  = var.environment
  region       = var.dr_region
  network_info = var.network_info

  providers = {
    aws = aws.dr
  }
}

Usage in a resource:

resource "aws_vpc" "secondary" {
  cidr_block = "10.1.0.0/16"

  provider = aws.dr   # provider meta-argument
}

Multi-Account AWS: IAM Profiles

provider "aws" {
  region  = var.region
  profile = "prod-account"      # AWS CLI profile
}

provider "aws" {
  alias   = "dev"
  region  = var.region
  profile = "dev-account"
}

Multi-Account AWS: assume_role

provider "aws" {
  region = var.region             # main account
}

provider "aws" {
  alias  = "security"
  region = var.region

  assume_role {
    role_arn     = var.security_role_arn
    session_name = "terraform-session"
    # external_id = var.external_id   # optional
  }
}

Usage with passing providers to a module:

module "prod_s3_bucket" {
  source = "../vpc_flow_logs"

  vpc_id           = module.prod_vpc.vpc_id
  naming_prefix    = "my-app"
  iam_role_arn     = var.security_role_arn
  bucket_id_suffix = random_string.bucket_suffix.result

  providers = {
    aws             = aws.security   # security account (creates the bucket)
    aws.vpc_account = aws            # main account (configures flow logs)
  }
}

configuration_aliases in Modules

A child module can declare that it expects to receive multiple instances of the same provider via configuration_aliases:

# vpc_flow_logs/versions.tf
terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
      configuration_aliases = [aws.vpc_account]
    }
  }
}

The module receives the default aws instance (security account) AND the aws.vpc_account instance (VPC account). Both are mapped by the parent via the providers = { ... } block.

sequenceDiagram
    participant Root as Root Module
    participant Mod as vpc_flow_logs Module
    participant SecAcc as AWS Security Account
    participant VpcAcc as AWS VPC Account

    Root->>Mod: providers = { aws = aws.security, aws.vpc_account = aws }
    Mod->>SecAcc: Creates S3 bucket (via aws default)
    Mod->>VpcAcc: Configures VPC Flow Logs (via aws.vpc_account)
    SecAcc-->>Mod: bucket ARN
    VpcAcc-->>Mod: flow log ID
    Mod-->>Root: outputs

Providers and Workspaces

# Create a workspace
terraform workspace new testing

# List workspaces
terraform workspace list

# Switch workspace
terraform workspace select production

Key points:

Warning: Switching workspaces does not change credentials or provider versions. A terraform apply in the testing workspace will still deploy to the configured AWS accounts (production!). Always verify your context before applying.

5. Quick Reference

Popular Providers

Version Constraints

# Exact version (very restrictive, avoid in production)
version = "= 3.7.1"

# Minimum required
version = ">= 3.7.0"

# Recommended range — patch updates only (~> pessimistic constraint)
version = "~> 3.7.0"   # >= 3.7.0, < 3.8.0

# Recommended range — minor updates allowed
version = "~> 3.7"     # >= 3.7, < 4.0

# Explicit combination (avoids a problematic version)
version = ">= 3.7.0, < 4.0.0, != 3.8.1"

Logging Environment Variables

# Available levels: TRACE > DEBUG > INFO > WARNING > ERROR > JSON
export TF_LOG=DEBUG
export TF_LOG_PATH=/tmp/tf.log

# Separate core and provider
export TF_LOG_CORE=ERROR
export TF_LOG_PROVIDER=DEBUG
export TF_LOG_PATH=/tmp/tf-provider.log

terraform plan

Recommended File Structure

.
├── terraform.tf          # required_providers, required_version
├── providers.tf          # provider blocks (default + aliased)
├── main.tf               # main resources
├── dr.tf                 # disaster recovery resources
├── variables.tf          # variable declarations
├── outputs.tf            # outputs
├── terraform.tfvars      # variable values (don't commit if sensitive)
├── .terraform.lock.hcl   # lock file — ALWAYS commit
├── .terraform/           # downloaded providers — DO NOT commit
│   └── providers/
│       └── registry.terraform.io/hashicorp/aws/6.13.0/...
└── modules/
    └── vpc/
        ├── versions.tf   # module's required_providers
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

Search Terms

terraform · providers · infrastructure · ci/cd · devops · provider · alias · aws · constraints · multi-account · version