|
| 1 | +--- |
| 2 | +# required metadata |
| 3 | + |
| 4 | +title: Fail over HA availability group SQL Server on Linux | Microsoft Docs |
| 5 | +description: |
| 6 | +author: MikeRayMSFT |
| 7 | +ms.author: mikeray |
| 8 | +manager: jhubbard |
| 9 | +ms.date: 04/12/2017 |
| 10 | +ms.topic: article |
| 11 | +ms.prod: sql-linux |
| 12 | +ms.technology: database-engine |
| 13 | +ms.assetid: |
| 14 | + |
| 15 | +# optional metadata |
| 16 | +# keywords: "" |
| 17 | +# ROBOTS: "" |
| 18 | +# audience: "" |
| 19 | +# ms.devlang: "" |
| 20 | +# ms.reviewer: "" |
| 21 | +# ms.suite: "" |
| 22 | +# ms.tgt_pltfrm: "" |
| 23 | +# ms.custom: "" |
| 24 | + |
| 25 | +--- |
| 26 | + |
| 27 | +# Fail over HA availability group for SQL Server on Linux |
| 28 | + |
| 29 | +Use the cluster management tools to failover an availability group managed by an external cluster manager. For example, if a solution uses Pacemaker to manage a Linux cluster, use `pcs` to perform manual failovers. |
| 30 | + |
| 31 | +> [!IMPORTANT] |
| 32 | +> Under normal conditions, do not fail over with Transact-SQL or SQL Server management tools like SSMS or PowerShell. When `CLUSTER_TYPE = EXTERNAL`, the only acceptable value for `FAILOVER_MODE` is `EXTERNAL`. With these settings, all manual or automatic failover actions are executed by the external cluster manager - for example Pacemaker. |
| 33 | +
|
| 34 | +In extreme cases, you might have to failover with SQL Server tools to bypass the external cluster manager. For example, if the cluster is unresponsive, or the cluster management tools cannot interact with the cluster. This is not recommended for regular operations, and should be used only when the cluster does not fail over with the cluster management tools. |
| 35 | + |
| 36 | +If you cannot failover the availability group with the cluster management tools, follow these steps to failover from SQL Server tools: |
| 37 | + |
| 38 | +1. Verify that the availability group resource is not managed by the cluster any more. |
| 39 | + |
| 40 | + - Attempt to set the resource to unmanaged mode. This signals the resource agent to stop resource monitoring and management. For example: |
| 41 | + |
| 42 | + ```bash |
| 43 | + sudo pcs resource unmanage <**resourceName**> |
| 44 | + ``` |
| 45 | + |
| 46 | + - If the attempt to set the resource mode to unmanaged mode fails, delete the resource. For example: |
| 47 | + |
| 48 | + ```bash |
| 49 | + sudo pcs resource delete <**resourceName**> |
| 50 | + ``` |
| 51 | + |
| 52 | + >[!NOTE] |
| 53 | + >When you delete a resource it also deletes all of the associated constraints. |
| 54 | + |
| 55 | +1. Manually set the session context to `external_cluster`. |
| 56 | + |
| 57 | + ```Transact-SQL |
| 58 | + EXEC sp_set_session_context @key = N'external_cluster', @value = N'yes'; |
| 59 | + ``` |
| 60 | + |
| 61 | +1. Fail over the availability group with Transact-SQL. In the example below replace `<**MyAg**>` with the name of your availability group. Connect to the instance of SQL Server that hosts the target secondary replica and run the following command: |
| 62 | + |
| 63 | + ```Transact-SQL |
| 64 | + ALTER AVAILABILITY GROUP <**MyAg**> FAILOVER; |
| 65 | + ``` |
| 66 | + |
| 67 | +1. Restart cluster resource monitoring and management. Run the following command: |
| 68 | + |
| 69 | + ```bash |
| 70 | + sudo pcs resource manage <**resourceName**> |
| 71 | + ``` |
| 72 | + |
| 73 | +## Pacemaker notification for availability group resource promotion |
| 74 | + |
| 75 | +Before the CTP 1.4 release, the Pacemaker resource agent for availability groups could not know if a replica marked as `SYNCHRONOUS_COMMIT` was really up-to-date or not. It was possible that the replica had stopped synchronizing with the primary but was not aware. Thus the agent could promote an out-of-date replica to primary - which, if successful, would cause data loss. |
| 76 | + |
| 77 | +SQL Server vNext CTP 1.4 added `sequence_number` to `sys.availability_groups` to solve this issue. `sequence_number` is a monotonically increasing BIGINT that represents how up-to-date the local availability group replica is with respect to the rest of the replicas in the availability group. Performing failovers, adding or removing replicas, and other availability group operations update this number. The number is updated on the primary, then pushed to secondaries. Thus a secondary replica that is up-to-date will have the same sequence_number as the primary. |
| 78 | + |
| 79 | +When Pacemaker decides to promote a replica to primary, it first sends a notification to all replicas to extract the sequence number and store it (we call this the pre-promote notification). Next, when Pacemaker actually tries to promote a replica to primary, the replica only promotes itself if its sequence number is the highest of all the sequence numbers from all replicas and rejects the promote operation otherwise. In this way only the replica with the highest sequence number can be promoted to primary, ensuring no data loss. |
| 80 | + |
| 81 | +Note that this is only guaranteed to work as long as at least one replica available for promotion has the same sequence number as the previous primary. To ensure this, the default behavior is for the Pacemaker resource agent to automatically set `REQUIRED_COPIES_TO_COMMIT` such that at least one synchronous commit secondary replica is up to date and available to be the target of an automatic failover. With each monitoring action, the value of `REQUIRED_COPIES_TO_COMMIT` is computed (and updated if necessary) as ('number of synchronous commit replicas' / 2). Then, at failover time, the resource agent will require (`total number of replicas` - `required_copies_to_commit` replicas) to respond to the pre-promote notification to be able to promote one of them to primary. The replica with the highest `sequence_number` will be promoted to primary. |
| 82 | + |
| 83 | +For example, let's consider the case of an availability group with three synchronous replicas - one primary replica and two synchronous commit secondary replicas. |
| 84 | +
|
| 85 | +- `REQUIRED_COPIES_TO_COMMIT` is 3 / 2 = 1 |
| 86 | +
|
| 87 | +- The required number of replicas to respond to pre-promote action is 3 - 1 = 2. So 2 replicas have to be up for the failover to be triggered. This means that if one of the secondary replicas is unresponsive and only one of the secondaries responds to the pre-promote action, the resource agent cannot guarantee that the secondary that responded has the highest sequence_number, and a failover is not triggered. |
| 88 | +
|
| 89 | +A user can choose to override the default behavior, and configure the availability group resource to not set `REQUIRED_COPIES_TO_COMMIT` automatically as above. |
| 90 | +
|
| 91 | +>[!IMPORTANT] |
| 92 | +>When `REQUIRED_COPIES_TO_COMMIT` is 0 there is risk of data loss. In the case of an outage of the primary, the resource agent will not automatically trigger a failover. The user has to decide if they want to wait for primary to recover or manually fail over. |
| 93 | +
|
| 94 | +To set `REQUIRED_COPIES_TO_COMMIT` to 0, run: |
| 95 | +
|
| 96 | +```bash |
| 97 | +sudo pcs resource update <**ag1**> required_copies_to_commit=0 |
| 98 | +``` |
| 99 | +
|
| 100 | +To revert to default computed value, run: |
| 101 | +
|
| 102 | +```bash |
| 103 | +sudo pcs resource update <**ag1**> required_copies_to_commit= |
| 104 | +``` |
| 105 | +
|
| 106 | +Because the resource agent requires Pacemaker to send notifications to all replicas, the availability resource needs to be configured with `notify=true`. The following commmand sets `notify=true`. |
| 107 | +
|
| 108 | +```bash |
| 109 | +sudo pcs resource create ag_cluster ocf:mssql:ag ag_name=<**ag1**> --master meta notify=true |
| 110 | +``` |
| 111 | +
|
| 112 | +## Manage availability group with two synchronous replicas |
| 113 | +
|
| 114 | +The above default behavior applies to the case of 2 synchronous replicas (primary + secondary) as well. Pacemaker will default `REQUIRED_COPIES_TO_COMMIT` = 1 to ensure the secondary replica is always up to date for maximum data protection. |
| 115 | +
|
| 116 | +>[!WARNING] |
| 117 | +>This comes with higher risk of unavailability of the primary replica due to planned or unplanned outages on the secondary. The user can choose to change the default behavior of the resource agent and override the `REQUIRED_COPIES_TO_COMMIT` to 0: |
| 118 | +
|
| 119 | +```bash |
| 120 | +sudo pcs resource update <**ag1**> required_copies_to_commit=0 |
| 121 | +``` |
| 122 | +
|
| 123 | +Once overridden, the resource agent will use the new setting for `REQUIRED_COPIES_TO_COMMIT` and stop computing it. This means that users have to manually update it accordingly (for example, if they increase the number of replicas). |
| 124 | +
|
| 125 | +The table below describes the outcome of an outage for primary or secondary replicas in different availability group resource configurations: |
| 126 | +
|
| 127 | +| | Availability group with 2 sync replicas | |Availability Group with 3 sync replicas | | | |
| 128 | +|---|---|---|---|---|---| |
| 129 | +| |`REQUIRED_COPIES_TO_COMMIT=0`|`REQUIRED_COPIES_TO_COMMIT=1`* |`REQUIRED_COPIES_TO_COMMIT=0` |`REQUIRED_COPIES_TO_COMMIT=1`* |`REQUIRED_COPIES_TO_COMMIT=2` |
| 130 | +|**Primary outage** |User has to issue a manual FAILOVER (might have data loss) -> New primary is R/W |Cluster will automatically issue FAILOVER (no data loss) -> New primary is RO until former primary recovers and joins availability group as secondary | User has to issue a manual FAILOVER (might have data loss) -> New primary is R/W | Cluster will automatically issue FAILOVER (no data loss) -> New primary is R/W |Cluster will automatically issue FAILOVER (no data loss) -> New primary is R/O until former primary recovers and joins availability group as secondary |
| 131 | +|**One secondary replica outage** |Primary is R/W, running exposed to data loss |Primary is RO until secondary recovers |Primary is R/W | Primary is R/W Primary is RO |
| 132 | +
|
| 133 | +* SQL Server resource agent for Pacemaker default behavior |
| 134 | +
|
| 135 | +## Notes |
| 136 | +
|
| 137 | +### Database level monitoring and failover trigger |
| 138 | +
|
| 139 | +For `CLUSTER_TYPE=EXTERNAL`, the failover trigger semantics are different compared to WSFC. When the availability group is on an instance of SQL Server in a WSFC, transitioning out of `ONLINE` state for the database causes the availability group health to report a fault. This signals the cluster manager to trigger a failover. In Linux, the SQL Server instance cannot communicate with the cluster. Monitoring for database health is done "outside-in". If you opted in for database level failover monitoring and failover (by setting the DDL option `DB_FAILOVER=ON`), the cluster will check if the database state is `ONLINE` every time when it runs a monitoring action. The cluster queries the state in `sys.databases`. For any state different than `ONLINE`, it triggers a failover automatically (if automatic failover conditions are met). The actual time of the failover depends on the frequency of the monitoring action as well as the database state being updated in sys.databases. |
| 140 | +
|
| 141 | +## Next steps |
| 142 | +
|
| 143 | +[Configure Red Hat Enterprise Linux Cluster for SQL Server Availability Group Cluster Resources](sql-server-linux-availability-group-cluster-rhel.md) |
| 144 | +
|
| 145 | +[Configure SUSE Linux Enterprise Server Cluster for SQL Server Availability Group Cluster Resources](sql-server-linux-availability-group-cluster-sles.md) |
| 146 | +
|
| 147 | +[Configure Ubuntu Cluster for SQL Server Availability Group Cluster Resources](sql-server-linux-availability-group-cluster-ubuntu.md) |
0 commit comments