|
| 1 | +--- |
| 2 | +title: Deploy availability groups with DH2i DxEnterprise sidecar on Kubernetes |
| 3 | +description: Set up an availability group in SQL Server on Kubernetes using DH2i DxEnterprise in a sidecar configuration. |
| 4 | +author: aravindmahadevan-ms |
| 5 | +ms.author: armaha |
| 6 | +ms.reviewer: amitkh, randolphwest |
| 7 | +ms.date: 04/06/2023 |
| 8 | +ms.service: sql |
| 9 | +ms.subservice: linux |
| 10 | +ms.topic: tutorial |
| 11 | +ms.custom: intro-deployment |
| 12 | +--- |
| 13 | +# Deploy availability groups with DH2i DxEnterprise in a sidecar configuration |
| 14 | + |
| 15 | +This tutorial explains how to configure [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] availability groups on a Kubernetes cluster with DH2i DxEnterprise. Using the steps mentioned in this article, learn how to deploy a StatefulSet and use the DH2i DxEnterprise solution to create and configure an availability group (AG). This tutorial consists of the following steps: |
| 16 | + |
| 17 | +> [!div class="checklist"] |
| 18 | +> - Create a headless service configuration |
| 19 | +> - Create a StatefulSet configuration |
| 20 | +> - Create a [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] AG, then configure a primary pod and add it to the AG |
| 21 | +> - Join two other pods to cluster and add them to the AG |
| 22 | +> - Create a database in the AG, and test failover |
| 23 | +
|
| 24 | +## Prerequisites |
| 25 | + |
| 26 | +This tutorial shows an example of an AG with three replicas. You need: |
| 27 | + |
| 28 | +- An Azure Kubernetes Service (AKS) or Kubernetes cluster. |
| 29 | +- A valid DxEnterprise license with AG features and tunnels enabled. For more information, see the [developer edition](https://dh2i.com/trial/) for nonproduction usage, or [DxEnterprise software](https://dh2i.com/dxenterprise-high-availability/) for production workloads. |
| 30 | + |
| 31 | +## Create the headless service |
| 32 | + |
| 33 | +1. In a Kubernetes cluster, headless services allow your pods to connect to one another using hostnames. |
| 34 | + |
| 35 | + To create the headless service, Create a YAML file called `headless_services.yml`, with the following sample content: |
| 36 | + |
| 37 | + ```yaml |
| 38 | + #Headless services for local connections/resolution |
| 39 | + apiVersion: v1 |
| 40 | + kind: Service |
| 41 | + metadata: |
| 42 | + name: dxemssql-0 |
| 43 | + spec: |
| 44 | + clusterIP: None |
| 45 | + selector: |
| 46 | + statefulset.kubernetes.io/pod-name: dxemssql-0 |
| 47 | + ports: |
| 48 | + - name: dxl |
| 49 | + protocol: TCP |
| 50 | + port: 7979 |
| 51 | + - name: dxc-tcp |
| 52 | + protocol: TCP |
| 53 | + port: 7980 |
| 54 | + - name: dxc-udp |
| 55 | + protocol: UDP |
| 56 | + port: 7981 |
| 57 | + - name: sql |
| 58 | + protocol: TCP |
| 59 | + port: 1433 |
| 60 | + - name: listener |
| 61 | + protocol: TCP |
| 62 | + port: 14033 |
| 63 | + --- |
| 64 | + apiVersion: v1 |
| 65 | + kind: Service |
| 66 | + metadata: |
| 67 | + name: dxemssql-1 |
| 68 | + spec: |
| 69 | + clusterIP: None |
| 70 | + selector: |
| 71 | + statefulset.kubernetes.io/pod-name: dxemssql-1 |
| 72 | + ports: |
| 73 | + - name: dxl |
| 74 | + protocol: TCP |
| 75 | + port: 7979 |
| 76 | + - name: dxc-tcp |
| 77 | + protocol: TCP |
| 78 | + port: 7980 |
| 79 | + - name: dxc-udp |
| 80 | + protocol: UDP |
| 81 | + port: 7981 |
| 82 | + - name: sql |
| 83 | + protocol: TCP |
| 84 | + port: 1433 |
| 85 | + - name: listener |
| 86 | + protocol: TCP |
| 87 | + port: 14033 |
| 88 | + --- |
| 89 | + apiVersion: v1 |
| 90 | + kind: Service |
| 91 | + metadata: |
| 92 | + name: dxemssql-2 |
| 93 | + spec: |
| 94 | + clusterIP: None |
| 95 | + selector: |
| 96 | + statefulset.kubernetes.io/pod-name: dxemssql-2 |
| 97 | + ports: |
| 98 | + - name: dxl |
| 99 | + protocol: TCP |
| 100 | + port: 7979 |
| 101 | + - name: dxc-tcp |
| 102 | + protocol: TCP |
| 103 | + port: 7980 |
| 104 | + - name: dxc-udp |
| 105 | + protocol: UDP |
| 106 | + port: 7981 |
| 107 | + - name: sql |
| 108 | + protocol: TCP |
| 109 | + port: 1433 |
| 110 | + - name: listener |
| 111 | + protocol: TCP |
| 112 | + port: 14033 |
| 113 | + ``` |
| 114 | +
|
| 115 | +1. Run the following command to apply the configuration: |
| 116 | +
|
| 117 | + ```bash |
| 118 | + kubectl apply -f headless_services.yml |
| 119 | + ``` |
| 120 | + |
| 121 | +## Create the StatefulSet |
| 122 | + |
| 123 | +1. Create a StatefulSet YAML file with following sample content, and name it `dxemmsql.yml`. |
| 124 | + |
| 125 | + This StatefulSet configuration creates three DxEMSSQL replicas that utilize persistent volume claims to store their data. Each pod in this StatefulSet comprises two containers: a [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] container and a DxEnterprise container. These containers are started separately from one another in a "sidecar" configuration, but DxEnterprise manages the AG replica in the [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] container. |
| 126 | + |
| 127 | + ```yaml |
| 128 | + #DxEnterprise + MSSQL StatefulSet |
| 129 | + apiVersion: apps/v1 |
| 130 | + kind: StatefulSet |
| 131 | + metadata: |
| 132 | + name: dxemssql |
| 133 | + spec: |
| 134 | + serviceName: "dxemssql" |
| 135 | + replicas: 3 |
| 136 | + selector: |
| 137 | + matchLabels: |
| 138 | + app: dxemssql |
| 139 | + template: |
| 140 | + metadata: |
| 141 | + labels: |
| 142 | + app: dxemssql |
| 143 | + spec: |
| 144 | + securityContext: |
| 145 | + fsGroup: 10001 |
| 146 | + containers: |
| 147 | + - name: sql |
| 148 | + image: mcr.microsoft.com/mssql/server:2022-latest |
| 149 | + env: |
| 150 | + - name: ACCEPT_EULA |
| 151 | + value: "Y" |
| 152 | + - name: MSSQL_ENABLE_HADR |
| 153 | + value: "1" |
| 154 | + - name: MSSQL_SA_PASSWORD |
| 155 | + valueFrom: |
| 156 | + secretKeyRef: |
| 157 | + name: mssql |
| 158 | + key: MSSQL_SA_PASSWORD |
| 159 | + volumeMounts: |
| 160 | + - name: mssql |
| 161 | + mountPath: "/var/opt/mssql" |
| 162 | + - name: dxe |
| 163 | + image: dh2i/dxe |
| 164 | + env: |
| 165 | + - name: MSSQL_SA_PASSWORD |
| 166 | + valueFrom: |
| 167 | + secretKeyRef: |
| 168 | + name: mssql |
| 169 | + key: MSSQL_SA_PASSWORD |
| 170 | + volumeMounts: |
| 171 | + - name: dxe |
| 172 | + mountPath: "/etc/dh2i" |
| 173 | + volumeClaimTemplates: |
| 174 | + - metadata: |
| 175 | + name: dxe |
| 176 | + spec: |
| 177 | + accessModes: |
| 178 | + - ReadWriteOnce |
| 179 | + resources: |
| 180 | + requests: |
| 181 | + storage: 1Gi |
| 182 | + - metadata: |
| 183 | + name: mssql |
| 184 | + spec: |
| 185 | + accessModes: |
| 186 | + - ReadWriteOnce |
| 187 | + resources: |
| 188 | + requests: |
| 189 | + storage: 1Gi |
| 190 | + ``` |
| 191 | +
|
| 192 | +1. Create a credential for the [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] instance: |
| 193 | +
|
| 194 | + ```bash |
| 195 | + kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD="<password>" |
| 196 | + ``` |
| 197 | + |
| 198 | +1. Apply the StatefulSet configuration: |
| 199 | + |
| 200 | + ```bash |
| 201 | + kubectl apply -f dxemssql.yaml |
| 202 | + ``` |
| 203 | + |
| 204 | +1. Verify the status of the pods, and proceed to the next step when the pod's status becomes `running`: |
| 205 | + |
| 206 | + ```bash |
| 207 | + kubectl get pods |
| 208 | + kubectl describe pods |
| 209 | + ``` |
| 210 | + |
| 211 | +## Create the availability group |
| 212 | + |
| 213 | +In this section, you create the AG, configure the primary pod, and add it to the AG. |
| 214 | + |
| 215 | +1. First, you must activate your DxEnterprise license using the following command. You need a valid activation code for this step: |
| 216 | + |
| 217 | + ```bash |
| 218 | + kubectl exec -c dxe dxemssql-0 -- dxcli activate-server AAAA-BBBB-CCCC-DDDD --accept-eula |
| 219 | + ``` |
| 220 | + |
| 221 | +1. Add a virtual host to the cluster: |
| 222 | + |
| 223 | + ```bash |
| 224 | + kubectl exec -c dxe dxemssql-0 -- dxcli cluster-add-vhost vhost1 "" dxemssql-0 |
| 225 | + ``` |
| 226 | + |
| 227 | +1. Encrypt the [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] sysadmin password for DxEnterprise. The encrypted password is used to create the AG in the later steps: |
| 228 | + |
| 229 | + ```bash |
| 230 | + kubectl exec -c dxe dxemssql-0 -- dxcli encrypt-text p@ssw0rd |
| 231 | + ``` |
| 232 | + |
| 233 | +1. Add an AG to the virtual host. The [!INCLUDE [ssnoversion-md](../includes/ssnoversion-md.md)] sysadmin password must be encrypted using the output from the previous step, and used in the following example: |
| 234 | + |
| 235 | + ```bash |
| 236 | + kubectl exec -c dxe dxemssql-0 -- dxcli add-ags vhost1 ags1 "dxemssql-0|mssqlserver|sa|entergeneratedpasswordinpreviousstephere=|5022|synchronous_commit|0" |
| 237 | + ``` |
| 238 | + |
| 239 | +1. Set a cluster passkey and enable hostname lookup in DxEnterprise's global settings: |
| 240 | + |
| 241 | + ```bash |
| 242 | + kubectl exec -c dxe dxemssql-0 -- dxcli cluster-set-secret-ex clusp@ssw0rd |
| 243 | + kubectl exec -c dxe dxemssql-0 -- dxcli set-globalsetting membername.lookup true |
| 244 | + ``` |
| 245 | + |
| 246 | +## Join two pods to cluster and add them to the availability group |
| 247 | + |
| 248 | +1. To join the pods to DxEnterprise cluster, you need to activate the DxEnterprise license and run the command to join second pod and add it to existing AG using the `dxcli add ags-node` command. You need a valid activation code for this step: |
| 249 | + |
| 250 | + ```bash |
| 251 | + kubectl exec -c dxe dxemssql-1 -- dxcli activate-server AAAA-BBBB-CCCC-DDDD --accept-eula |
| 252 | + kubectl exec -c dxe dxemssql-1 -- dxcli join-cluster-ex dxemssql-0 p@ssw0rd |
| 253 | + ``` |
| 254 | + |
| 255 | +1. Use the encrypted password output you get on the previous step on the following command: |
| 256 | + |
| 257 | + ```bash |
| 258 | + kubectl exec -c dxe dxemssql-1 -- dxcli add-ags-node vhost1 ags1 "dxemssql-1|mssqlserver|sa|entergeneratedpasswordinpreviousstephere=|5022|synchronous_commit|0" |
| 259 | + ``` |
| 260 | + |
| 261 | +1. Activate the DxEnterprise license for the third pod: |
| 262 | + |
| 263 | + ```bash |
| 264 | + kubectl exec -c dxe dxemssql-2 -- dxcli activate-server AAAA-BBBB-CCCC-DDDD --accept-eula |
| 265 | + ``` |
| 266 | + |
| 267 | +1. Join the third pod to the DxEnterprise cluster, and add the pod to the existing AG: |
| 268 | + |
| 269 | + ```bash |
| 270 | + kubectl exec -c dxe dxemssql-2 -- dxcli join-cluster-ex dxemssql-0 p@ssw0rd |
| 271 | + kubectl exec -c dxe dxemssql-2 -- dxcli add-ags-node vhost1 ags1 "dxemssql-2|mssqlserver|sa|6pnFaDrRS+W/F+dkRuPKAA==|5022|synchronous_commit|0" |
| 272 | + ``` |
| 273 | + |
| 274 | +1. Create database in the AG: |
| 275 | + |
| 276 | + ```bash |
| 277 | + kubectl exec -c sql dxemssql-0 -- /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P p@ssw0rd -Q "create database db1" |
| 278 | + ``` |
| 279 | + |
| 280 | +1. Add the database to the AG and verify that it was added successfully: |
| 281 | + |
| 282 | + ```bash |
| 283 | + kubectl exec -c dxe dxemssql-0 -- dxcli add-ags-databases vhost1 ags1 db1 |
| 284 | + kubectl exec -c dxe dxemssql-0 -- dxcli get-ags-detail vhost1 ags1 | kubectl exec -c dxe dxemssql-0 -- dxcli format-xml |
| 285 | + ``` |
| 286 | + |
| 287 | +## Test failover |
| 288 | + |
| 289 | +1. To verify the state of the AG before failover, check the status of the database of each AG member, and find out which pod is the primary: |
| 290 | + |
| 291 | + ```bash |
| 292 | + kubectl exec -c dxe dxemssql-0 -- dxcli get-ags-detail vhost1 ags1 | kubectl exec -c dxe dxemssql-0 -- dxcli format-xml |
| 293 | + ``` |
| 294 | + |
| 295 | +1. Delete the primary pod: |
| 296 | + |
| 297 | + ```bash |
| 298 | + kubectl delete pod <PRIMARY_POD> |
| 299 | + ``` |
| 300 | + |
| 301 | +1. Now check the status of AG. The database should be active and synchronized on the other pods. After few minutes, the deleted pod automatically rejoins the cluster and synchronizes with the AG: |
| 302 | + |
| 303 | + ```bash |
| 304 | + kubectl exec -c dxe dxemssql-0 -- dxcli get-ags-detail |
| 305 | + ``` |
| 306 | + |
| 307 | +1. To set the rejoined pod role back to primary, run the following command: |
| 308 | + |
| 309 | + ```bash |
| 310 | + kubectl exec -c dxe dxemssql-0 -- dxcli vhost-start-node vhost1 dxemssql-0 |
| 311 | + ``` |
| 312 | + |
| 313 | +## Next steps |
| 314 | + |
| 315 | +- [Deploy SQL Server Linux containers on Kubernetes with StatefulSets](sql-server-linux-kubernetes-best-practices-statefulsets.md) |
| 316 | +- [Tutorial: Configure Active Directory authentication with SQL Server on Linux containers](sql-server-linux-containers-ad-auth-adutil-tutorial.md) |
| 317 | +- [Deploy availability group with DH2i for SQL Server containers on AKS](tutorial-sql-server-containers-kubernetes-dh2i.md) |
0 commit comments