Azure Data Services deployment using Terraform

Description

Terraform-based deployment of the following Azure resources (default deployment settings are in the parenthesis):

• Azure Service Bus (Standard, namespace,topic,subscription, auth. rules)
• Azure Data Lake Storage (ZRS, Hot, Secured, StandardV2)
• Azure Data Factory (w/Git or without)
• Azure Data Factory linked with Data Lake Storage
• Azure Data Factory Pipeline
• Azure DataBricks WorkSpace (Standard)
• Azure EventHub (Standard, namespace)
• Azure Functions (Dynamic, LRS storage, Python, w/App.Insights or without)
• Azure Data Explorer (Kusto, Standard_D11_v2, 2 nodes)
• Azure Analysis Server (backup-enabled,S0, LRS, Standard)
• Azure Event Grid (domain, EventGridSchema)
• Azure SQL Server (version 12.0)
• Azure SQL Database (ElasticPool, 5 GB max data size)
• Azure SQL Elastic Pool (StandartPool, LicenseIncluded, 50 eDTU, 50GB max data size)

Content

• auth.tf - Azure ARM provider authentication and version settings
• main.tf - Main configuration file that describes desired Azure infrastructure
• terraform.tfvars - Variables values to control/change the deployment settings
• variables.tf - List of variables used in the configuration
• outputs.tf - Outputs some useful info at the end of deployment (URLs and etc.)

Usage

Control the deployment size:

• Open the terraform.tfvars file
• Identify the "What Should Be Deployed?" section
• Use true/false to setup your configuration
• Check or change Azure service settings in the appropriate sections
• Run terraform init to get required Terraform providers
• Run terraform plan to initiate pre-deployment check
• Run terraform apply to start deployment
• (optional) terraform destroy to delete Azure resources

Tested with the latest terraform 0.15.5 and AzureRM provider 2.62.0:

Download the latest terraform: https://www.terraform.io/downloads.html

Count blocks are used frequently in this repo. If you need example of for_each block, check my gist: https://gist.github.com/rlevchenko/358ad95cc5641bc26d2a5705c2c8dbfd

Requirements

Azure Service Principal with Contributor rights or higher:

• subscription id
• client id
• tenant id
• principal secret

If Analysis Server is going to be deployed:

• valid Azure AD user(s) UPN(s) (will be AS admin(s))

Result