Azure
Please check the general requirements for basic requirements. On this page, we include Azure-specific recommendations.
Setting up Domain Name
For a seamless integration experience, we recommend using Azure DNS zones.
Configuring Network
Check the network and firewall requirements section from the general requirements.
Please follow Azure Virtual Network concepts and best practices guide to set up your virtual network, subnets, security groups, etc.
Please contact AlphaSense if you need extra annotations to the nginx ingress service kind specified in specified in cloud-provider-azure
Kubernetes Cluster
CNI
We recommend using AKS default CNI or Cilium.
Node Groups
Select one of the reference architectures and configure the node group autoscaling accordingly.
For GPU node groups, please select Ubuntu 22.04 OS family, with tags SkipGPUDriverInstall:true. We
included NVIDIA GPU operator to configure the NVIDIA
driver.
Issue with PremiumV2_LRS Azure Disks
There is a known issue in AKS when the node groups are without Availability Zone (AZ). The issue can be described as below:
- PremiumV2_LRS disk can NOT be attached to VM without AZ.
- PremiumV2_LRS can still be provisioned and attached to node by the CSI driver ( csi-azuredisk-node daemonset).
- Once attached this way, they cannot be detached by CSI driver or from the UI/CLI and deleting the PVC will NOT delete the disk.
- Deleting the VM that the disk is attached to, will NOT delete the disk nor detach it.
- New PVC can be created but if it has the same name, it will incorrectly refer to the old PV even though the id and volume is new.
To circumvent this issue, there are two ways:
- Create AKS node groups with AZ. (preferred)
- Forbid using PremiumV2_LRS.
Unfortunately, in case you have run into the above issue, the below steps might help:
- Delete the node where the stuck pod was scheduled on.
kubectl delete nodecan be used and it will delete that node in AKS and create a new node. - Delete both the PVC and the statefulset pods and make sure new pods are scheduled on new node. If
the pod deletion gets stuck due to the
PremiumV2_LRSdisk then it can be force deleted. - Rinse and repeat until the pod gets scheduled onto a new node where there have not been
PremiumV2_LRSdisks attached. However, in the Azure UI thePremiumV2_LRSdisks would still be seen and cannot be deleted.
If a node group has VMs without AZ, then it has to be recreated i.e. destroyed and created again
with similar configurations to have VMs and disks with AZ. However, the pods that will get scheduled
on new nodes with AZ, will run into Volume Node Affinity Conflict error, for all the existing
disks (any SKU type). This is due to misalignment between the zones where your PVs are provisioned
and where your pods are scheduled, and therefore the pods cannot be scheduled unless data is
manually migrated to newer disks or switching back to node groups with VMs without AZ.
References:
Storage Account
A storage account with a designated name is required. The Bash scripts provided below can assist in querying the storage account name.
Please ensure that placeholders in the commands are replaced with the relevant information before execution.
export RESOURCE_GROUP_NAME=<your resource group name, i.e: alphasense>
export LOCATION=<your location, i.e: eastus>
PREFIX=$(echo "${RESOURCE_GROUP_NAME}" | tr -cd '[:alnum:]_' | tr '[:upper:]' '[:lower:]')
export STORAGE_ACCOUNT_NAME="${PREFIX:0:22}"sa
az storage account create \
--resource-group $RESOURCE_GROUP_NAME \
--name $STORAGE_ACCOUNT_NAME \
--location $LOCATION \
--sku Standard_RAGRS \
--kind StorageV2 \
--access-tier Hot \
--allow-blob-public-access false
MySQL Database Cluster
Please check the engine and required parameters in the MySQL
database cluster section of the general
requirements.
We recommend using Azure Database for MySQL servers.
If you use Azure Database for MySQL servers, please follow the server parameters guide and CLI docs to configure the server parameters.
Crossplane Managed Identity
To enable Crossplane to create all the necessary resources and permissions, a role must be created. The following examples and lists are provided for audit purposes. You can decide how to create the role and identity, whether through Terraform or another method of your choice.
Please ensure that placeholders in the commands are replaced with the relevant information before execution.
export RESOURCE_GROUP_NAME=<your resource group name, i.e: alphasense>
export RESOURGE_GROUP_ID="$(az group show --name ${RESOURCE_GROUP_NAME} --query id -o tsv)"
export CLUSTER_NAME=<your cluster name, e.g. thunderstorm>
export CROSSPLANE_ASSIGNED_IDENTITY_NAME="${CLUSTER_NAME}-crossplane-controller"
export AKS_OIDC_ISSUER="$(az aks show --name ${CLUSTER_NAME} --resource-group "${RESOURCE_GROUP_NAME}" --query "oidcIssuerProfile.issuerUrl" -o tsv)"
cat << 'EOF' > crossplane_role_profivider.json
{
"Name": "${CLUSTER_NAME}-crossplane-provider",
"IsCustom": true,
"Description": "This Role is used by Crossplane provider",
"Actions": [
"Microsoft.Authorization/policies/auditIfNotExists/action",
"Microsoft.Authorization/policies/audit/action",
"Microsoft.Authorization/roleAssignments/read",
"Microsoft.Authorization/roleAssignments/write",
"Microsoft.Authorization/roleAssignments/delete",
"Microsoft.Authorization/roleDefinitions/write",
"Microsoft.Authorization/roleDefinitions/delete",
"Microsoft.Authorization/roleDefinitions/read",
"Microsoft.ManagedIdentity/userAssignedIdentities/write",
"Microsoft.ManagedIdentity/userAssignedIdentities/read",
"Microsoft.ManagedIdentity/userAssignedIdentities/delete",
"Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/write",
"Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/read",
"Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/delete",
"Microsoft.Network/virtualNetworks/read",
"Microsoft.Network/virtualNetworks/subnets/read",
"Microsoft.Network/virtualNetworks/subnets/join/action",
"Microsoft.Network/virtualNetworks/subnets/joinViaServiceEndpoint/action",
"Microsoft.ServiceBus/namespaces/write",
"Microsoft.ServiceBus/namespaces/read",
"Microsoft.ServiceBus/namespaces/Delete",
"Microsoft.ServiceBus/namespaces/queues/read",
"Microsoft.ServiceBus/namespaces/queues/write",
"Microsoft.ServiceBus/namespaces/queues/Delete",
"Microsoft.ServiceBus/namespaces/topics/read",
"Microsoft.ServiceBus/namespaces/topics/write",
"Microsoft.ServiceBus/namespaces/topics/Delete",
"Microsoft.ServiceBus/namespaces/topics/subscriptions/write",
"Microsoft.ServiceBus/namespaces/topics/subscriptions/Delete",
"Microsoft.ServiceBus/namespaces/topics/subscriptions/read",
"Microsoft.ServiceBus/namespaces/topics/subscriptions/rules/write",
"Microsoft.ServiceBus/namespaces/topics/subscriptions/rules/read",
"Microsoft.ServiceBus/namespaces/topics/subscriptions/rules/Delete",
"Microsoft.Storage/skus/read",
"Microsoft.Storage/storageAccounts/read",
"Microsoft.Storage/storageAccounts/write",
"Microsoft.Storage/storageAccounts/delete",
"Microsoft.Storage/storageAccounts/blobServices/containers/delete",
"Microsoft.Storage/storageAccounts/blobServices/containers/write",
"Microsoft.Storage/storageAccounts/blobServices/containers/read",
"Microsoft.Storage/storageAccounts/blobServices/read",
"Microsoft.Storage/storageAccounts/blobServices/write",
"Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action",
"Microsoft.Storage/storageAccounts/fileServices/read",
"Microsoft.Storage/storageAccounts/listkeys/action",
"Microsoft.Storage/storageAccounts/managementPolicies/delete",
"Microsoft.Storage/storageAccounts/managementPolicies/read",
"Microsoft.Storage/storageAccounts/managementPolicies/write",
"Microsoft.Storage/storageAccounts/regeneratekey/action"
],
"NotActions": [],
"AssignableScopes": ["${RESOURGE_GROUP_ID}"]
}
EOF
az role definition create --name ${CLUSTER_NAME}-crossplane-provider --role-definition @crossplane_role_profivider.json --location ${LOCATION}
az identity create --resource-group ${RESOURCE_GROUP_NAME} --name ${CROSSPLANE_ASSIGNED_IDENTITY_NAME} --location ${LOCATION}
export ROLE_ID=$(az role definition list --location ${LOCATION} --name ${CLUSTER_NAME}-crossplane-provider --query "[].{id:id}" -o tsv)
export PRINCIPAL_ID=$(az identity show --location ${LOCATION} --name ${CROSSPLANE_ASSIGNED_IDENTITY_NAME} --resource-group ${RESOURCE_GROUP_NAME} --query principalId -o tsv)
az role assignment create --location ${LOCATION} --assignee ${PRINCIPAL_ID} --role ${ROLE_ID} --scope ${RESOURCE_GROUP_ID}
az identity federated-credential create --name crossplane-provider --identity-name "${CROSSPLANE_ASSIGNED_IDENTITY_NAME}" --resource-group "${RESOURCE_GROUP_NAME}" --issuer "${AKS_OIDC_ISSUER}" --subject system:serviceaccount:crossplane:azure-provider-sa --audience api://AzureADTokenExchange
Kubernetes Secrets
Check the Kubernetes secrets section from the general requirements.
Telemetry
For auditing and troubleshooting purposes, you can use the built-in Grafana central hub to access the logs and metrics. Details can be found in Observability.