undo.is

It’s been a bit since I’ve made a new post here, but recently at work I ran into an interesting situation that I thought was worth documenting. At work, we recentlyreceived a directive to ensure that all of our Azure resources were compliant with the Azure CIS 1.3 controls, and I was handed a good portion of that work. As I was working through section 5.3.x requiring Diagnostic Logs to be enabled for a handful of different resource types, I noticed that we had a fair amount that were noncompliant - way too many to go through and fix by hand, at least. As our solution required these logs to be forwarded to an event hub rather than a storage account, we were unable to use the “quick fix” button in the portal, and I had to come up with another way to make all of these resources compliant.

Enter Terraform.

Our standard practice for deploying any cloud resource involves creating them through Terraform rather than through the portal or any other means, as Terraform is repeatable and self-documenting. I might go over bits of my Terraform learning journey in another post eventually, but I’ve fallen in love with this tool so much that I’ve even started using it for resource deployments in my homelab!

My end goal for this was to have my Terraform scripts do the following:

• Utilize an exported list of affected resources from the Microsoft Defender for Cloud portal.
• Determine which diagnostic logging settings were appropriate for that resource.
• Not interfere with any existing logging setting.
• Set these settings on only the resources in the targeted subscription.

The second bullet was the tricky bit in this project. Every type of resource in Azure has the capability of forwarding diagnostic settings specific to itself, and those are obviously not going to be the same for each resource type. Rather than setting specifications for each resource type manually, I needed to have this script determine which logging settings were available for any given resource and reference them later when creating the logging setting. Using a data block, I ended up doing the following:

#Import list of resources to act upon
locals {
    resources = csvdecode(file(var.path_to_resource_file))
}
#Define available diagnostic categories for specified resource
data "azurerm_monitor_diagnostic_categories" "this" {
    provider = azurerm.subscription2
    for_each = {for key, val in local.resources: key => val if val.SUBSCRIPTION_ID == var.sub_id}
    resource_id = each.value.RESOURCE_ID
}

Using the csvdecode functionality in Terraform, I imported the list of resources I needed to act upon and created an object map of available diagnostic categories for all resources in the subscription I was currently working in with a for_each argument. From here, I could then reference values in this map in future blocks to connect a given resource to the diagnostic categories available to it. With this data available, I then moved to use another for_each argument on the azurerm_monitor_diagnostic_setting resource in combination with a dynamic block for each of the settings available to that resource to ensure that logging for all available categories was enabled.

#Set up diagnostic forwarding to event hub
resource "azurerm_monitor_diagnostic_setting" "this" {
    provider = azurerm.subscription2
    for_each = {for key, val in local.resources: key => val if val.SUBSCRIPTION_ID == var.sub_id}
    name = format("%s%s",each.value.NAME,"-diagsetting")
    target_resource_id = each.value.RESOURCE_ID
    eventhub_name = var.event_hub_name
    eventhub_authorization_rule_id = data.azurerm_eventhub_namespace_authorization_rule.this.id

    dynamic "log" {
      for_each = data.azurerm_monitor_diagnostic_categories.this[each.key].logs
      content {
        category = log.value
        enabled = var.log_enabled
        retention_policy {
          days = var.log_retention_policy_days
          enabled = var.log_retention_policy_enabled
        }
        }
    }

    dynamic "metric" {
      for_each = data.azurerm_monitor_diagnostic_categories.this[each.key].metrics
      content {
        category = metric.value
        enabled = var.metric_enabled
        retention_policy {
          days = var.metric_retention_policy_days
          enabled = var.metric_retention_policy_enabled
        }
      }
    }
}

The key part here is this bit:

data.azurerm_monitor_diagnostic_categories.this[each.key].logs

As we are using the same source data for both the diagnostic category data block and the diagnostic setting resource block, they use the same key/value mapping scheme. We can then use that to join two unrelated object maps together to create the desired result; in this case, enabling all available logging categories for the resource currently being acted upon. After all is said and done, I ended up with this to execute on each environment:

#Import list of resources to act upon
locals {
    resources = csvdecode(file(var.path_to_resource_file))
}

#Define eventhub RG
data "azurerm_resource_group" "this" {
    provider = azurerm.subscription1
    name = var.event_hub_resource_group_name
}

#Define eventhub 
data "azurerm_eventhub_namespace_authorization_rule" "this" {
    provider = azurerm.subscription1
    name = var.event_hub_authorization_rule_name
    resource_group_name = data.azurerm_resource_group.this.name
    namespace_name = var.event_hub_namespace_name
}

#Define available diagnostic categories for specified resource
data "azurerm_monitor_diagnostic_categories" "this" {
    provider = azurerm.subscription2
    for_each = {for key, val in local.resources: key => val if val.SUBSCRIPTION_ID == var.sub_id}
    resource_id = each.value.RESOURCE_ID
}

#Set up diagnostic forwarding to event hub
resource "azurerm_monitor_diagnostic_setting" "this" {
    provider = azurerm.subscription2
    for_each = {for key, val in local.resources: key => val if val.SUBSCRIPTION_ID == var.sub_id}
    name = format("%s%s",each.value.NAME,"-diagsetting")
    target_resource_id = each.value.RESOURCE_ID
    eventhub_name = var.event_hub_name
    eventhub_authorization_rule_id = data.azurerm_eventhub_namespace_authorization_rule.this.id

    dynamic "log" {
      for_each = data.azurerm_monitor_diagnostic_categories.this[each.key].logs
      content {
        category = log.value
        enabled = var.log_enabled
        retention_policy {
          days = var.log_retention_policy_days
          enabled = var.log_retention_policy_enabled
        }
      }
    }

    dynamic "metric" {
      for_each = data.azurerm_monitor_diagnostic_categories.this[each.key].metrics
      content {
        category = metric.value
        enabled = var.metric_enabled
        retention_policy {
          days = var.metric_retention_policy_days
          enabled = var.metric_retention_policy_enabled
        }
      }
    }
}

This was a fun exercise in learning more about how to use Terraform in our environment, and it certainly will help if I need to come up with any other similarly creative solutions in the future!