November 2023: This publish was reviewed and up to date with the final availability of Multi-AZ deployments for provisioned RA3 clusters.
Initially revealed on December ninth, 2022.

Amazon Redshift is a totally managed, petabyte scale cloud information warehouse that lets you analyze giant datasets utilizing customary SQL. Information warehouse workloads are more and more getting used with mission-critical analytics functions that require the best ranges of resilience and availability. Amazon Redshift is a cloud-based information warehouse that helps many restoration capabilities to deal with unexpected outages and decrease downtime. Amazon Redshift RA3 occasion sorts retailer their information in Redshift Managed Storage (RMS), which is backed by Amazon Easy Storage Service (Amazon S3) making it extremely accessible and sturdy by default. Amazon Redshift additionally helps automated backups that may get better an information warehouse, routinely remediate failures and relocates clusters to totally different AZs with out adjustments to functions. Though many purchasers profit from these options, enterprise information warehouse prospects require a low Restoration Time Goal (RTO) and better availability to assist their enterprise continuity with minimal affect to functions.

Amazon Redshift simply introduced the final availability of Multi-AZ deployments for provisioned RA3 clusters that assist working your information warehouse in two Availability Zones concurrently and might proceed working in unexpected failure situations. A Multi-AZ deployment is meant for patrons with mission-critical analytics functions that require the best ranges of  resilience and availability.

A Redshift Multi-AZ deployment leverages compute assets in two AZs to scale information warehouse workload processing.  In conditions the place there’s a excessive stage or concurrency Redshift will routinely leverage the assets in each AZs to scale the workload for each learn and write requests.

Our pre-launch checks discovered that Amazon Redshift Multi-AZ deployments cut back restoration time to below 60 seconds or much less within the unlikely case of an AZ failure.

Single-AZ vs. Multi-AZ deployment

Amazon Redshift requires a cluster subnet group to create a cluster in your VPC. The cluster subnet group contains details about the VPC ID and an inventory of subnets in your VPC. Whenever you launch a cluster, Amazon Redshift both creates a default cluster subnet group routinely otherwise you select a cluster subnet group of your alternative in order that Amazon Redshift can provision your cluster in one of many subnets within the VPC. You may configure your cluster subnet group so as to add subnets from totally different Availability Zones that you really want Amazon Redshift to make use of for cluster deployment.

All Amazon Redshift clusters right now are created and located in a specific Availability Zone inside an AWS Area and thus known as Single-AZ deployments. For a Single-AZ deployment, Amazon Redshift selects the subnet from one of many Availability Zones inside a Area and deploys the cluster there. You may select an Availability Zone for deployment, and Amazon Redshift will deploy your cluster within the chosen Availability Zone based mostly on the subnets supplied.

However, a multi-AZ deployment is provisioned in two Availability Zones concurrently. For a Multi-AZ deployment, Amazon Redshift routinely selects two subnets from two totally different Availability Zones and deploys an equal variety of compute nodes in every Availability Zone. All these compute nodes are utilized through a single endpoint as compute nodes from each Availability Zones are used for workload processing.

As proven within the following diagrams, Amazon Redshift deploys a cluster in a single Availability Zone for Single-AZ deployment, and two Availability Zones for Multi-AZ deployment.

Auto restoration of multi-AZ deployment

Within the unlikely occasion of an Availability Zone failure, Amazon Redshift Multi-AZ deployments proceed to serve your workloads by routinely utilizing assets within the different Availability Zone. You aren’t required to make any utility adjustments to take care of enterprise continuity throughout unexpected outages since a multi-AZ deployment is accessed as a single information warehouse with one endpoint. Amazon Redshift Multi-AZ deployments are designed to make sure there isn’t any information loss, and you may question all information dedicated up till the purpose of failure.

As proven within the beneath diagram, if there’s an unlikely occasion that causes compute nodes in AZ1 to fail, then a multi-AZ deployment routinely recovers to make use of compute assets in AZ2. Amazon Redshift can even routinely provision equivalent compute nodes in one other availability zone (AZ3) to proceed working concurrently in two Availability zones (AZ2 and AZ3).

Amazon Redshift Multi-AZ deployment will not be solely used for cover towards the potential of Availability Zone failures, however it might probably additionally maximize your information warehouse efficiency by routinely distributing workload processing throughout two Availability Zones. A Multi-AZ deployment will at all times course of a person question utilizing compute assets solely from one Availability Zone, however it might probably routinely distribute processing of a number of simultaneous queries to each Availability Zones to extend total efficiency for top concurrency workloads.

It’s follow to arrange automated retries in your extract, remodel, and cargo (ETL) processes and dashboards in order that they are often reissued and served by the cluster within the secondary Availability Zone when an unlikely failure occurs within the main Availability Zone. If a connection is dropped, it might probably then be retried or reestablished instantly. As well as, queries and masses that have been working within the failed Availability Zone might be aborted. New queries issued at or after a failure happens could expertise run delays whereas the multi-AZ information warehouse is being recovered to a two AZ setup.

Overview of resolution

On this publish, we offer a walkthrough of the best way to create and handle a Multi-AZ deployment for Amazon Redshift utilizing the AWS Administration Console. We additionally check the fault tolerance of an Amazon Redshift Multi-AZ information warehouse and monitor queries in your Multi-AZ deployment.

Create a brand new Multi-AZ deployment from the console

You may simply create a brand new multi-AZ deployments by way of Amazon Redshift console. Amazon Redshift will deploy the identical variety of nodes in every of the 2 Availability Zones for a Multi-AZ deployment. All nodes of a multi-AZ deployment can carry out learn and write workload processing throughout regular operation. A Multi-AZ deployment is supported just for provisioned RA3 clusters.

Comply with these steps to create an Amazon Redshift provisioned cluster in two Availability Zones:

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. Click on on Create cluster.

For normal details about creating clusters, see Making a cluster.

3. Select one of many RA3 node sorts on the Node kind drop-down menu. The Multi-AZ deployment possibility solely turns into accessible whenever you select an RA3 node kind.
4. For Multi-AZ deployment, choose Multi-AZ possibility.
5. For Variety of nodes per AZ, enter the variety of nodes that you just want in your cluster.

1. Beneath the Database configurations, select Admin person identify and Admin person password.
2. Flip Use defaults on subsequent to Extra configurations to switch the default settings.
3. Beneath Community and safety, specify the next:
   1. For Digital non-public cloud (VPC), select the VPC you wish to deploy the cluster in.
   2. For VPC safety teams, both go away as default or add the safety teams of your alternative.
   3. For Cluster subnet group, both go away as default or add a cluster subnet group of your alternative. For a Multi-AZ deployment, a cluster subnet group should embody one subnet every from no less than three or extra totally different Availability Zones.

For normal details about managing cluster subnet teams, see Cluster subnet teams

9. Beneath Database configuration, for Database port, you both use the default worth 5439 or select a worth from the vary of 5431–5455 and 8191–8215.
10. Beneath Database configuration, within the Database encryption part, to make use of a customized AWS Key Administration Service (AWS KMS) key apart from the default KMS key, select Customise encryption settings. This feature is deselected by default.
11. Beneath Select an AWS KMS key, you’ll be able to both select an present KMS key, or select Create an AWS KMS key to create a brand new KMS key.

For extra data to create key utilizing KMS, discuss with Creating keys.

12. Select Create cluster.

When the cluster creation succeeds, you’ll be able to view the small print on the cluster particulars web page.

Beneath Common data, you’ll be able to see Multi-AZ as Sure.

On the Properties tab, below Community and safety settings, yow will discover the small print on the first and secondary Availability Zone.

Create a brand new Multi-AZ deployment from the CLI

The next create-cluster AWS CLI command reveals the best way to create a Multi-AZ cluster

aws redshift create-cluster 
--port 5439 
    --master-username grasp
    --master-user-password ######
    --node-type ra3.4xlarge
    --number-of-nodes 2
    --profile maz-test
    --endpoint-url https://redshift.us-east-1.amazonaws.com
    --region eu-west-1
    --cluster-identifier redshift-cluster-1
    --multi-az 
    --maintenance-track-name CURRENT
    --encrypted

Convert a Single-AZ deployment to Multi-AZ deployment

To transform an present Single-AZ deployment to a Multi-AZ deployment, you’ll be able to go to the Redshift console and choose your Redshift cluster that at the moment is Single-AZ setup and navigate to Actions and choose Activate Multi-AZ. Your Single-AZ cluster have to be encrypted for a profitable conversion to Multi-AZ. Throughout conversion to Multi-AZ, Redshift will double the entire variety of nodes distributing them equally in every AZ. Redshift is not going to help you break up present variety of nodes whereas changing to Multi-AZ to take care of constant question efficiency.

Full the next steps to create a Multi-AZ deployment restored from a snapshot:

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. Choose your cluster and navigate to the cluster particulars web page.
3. On the Actions menu, select Activate Multi-AZ.

4. Evaluation the modification abstract and make sure by selecting Activate Multi-AZ.

Utilizing the beneath AWS CLI command you’ll be able to convert a single AZ Redshift information warehouse to Multi-AZ.

aws redshift modify-cluster 
    --profile maz-test
    --endpoint-url https://redshift.eu-west-1.amazonaws.com
    --region eu-west-1
    --cluster-identifier redshift-cluster-1
    --multi-az

Convert a Multi-AZ deployment to Single-AZ deployment

Redshift additionally helps conversion of a Multi-AZ deployment into Single-AZ.  This feature gives prospects with the flexibleness to modify between totally different deployments with few straightforward steps as follows:

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. Choose your cluster and navigate to the cluster particulars web page.
3. On the Actions menu, select Deactivate Multi-AZ.
   
   
4. Evaluation the modification abstract and make sure by selecting Deactivate Multi-AZ.
   

Making a Multi-AZ information warehouse restored from a snapshot

Current prospects also can create a Multi-AZ deployment by restoring a snapshot from an present Single-AZ deployment. See the required steps as beneath.

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. Choose the cluster and navigate to the cluster particulars web page.
3. Select the Upkeep
4. Choose a snapshot and select Restore snapshot, Restore to provisioned cluster.
   
5. Evaluation the Cluster configuration and Cluster particulars values of the brand new cluster to be created utilizing the snapshot data.
6. Choose Multi-AZ possibility and replace the properties of the brand new cluster, then select Restore cluster from snapshot on the backside of the web page.
   

Resizing a Multi-AZ information warehouse

Redshift Multi-AZ function additionally helps resizing Multi-AZ Redshift cluster deployments to vary the cluster configuration based mostly on scaling wants. You may change each quantity and kind of nodes as per wants.

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. Choose your cluster and navigate to the cluster particulars web page.
3. On the Actions menu, select
4. As soon as chosen it should convey into one other display screen to point out cluster resize display screen the place you’ll be able to select kind and variety of nodes and click on on Resize cluster.

Failing over Multi-AZ deployment

Along with the automated restoration course of, you can even set off this course of manually in your information warehouse utilizing the Failover main compute possibility. This method can be utilized to handle operational upkeep and different deliberate operational procedures as per the wants of the respective atmosphere. When the cluster efficiently recovers, Multi-AZ deployment turns into accessible. Your Multi-AZ deployment additionally routinely provisions new compute nodes in one other Availability Zone as quickly as it’s accessible.

Let’s manually set off the Failover of your Redshift Multi-AZ deployment.

1. On the Amazon Redshift console, select Clusters within the navigation pane.
2. Navigate to the cluster element web page
3. From Actions, select Failover main compute.
4. When prompted, select Affirm.

After the cluster is again to Accessible standing, you’ll be able to observe that the first and secondary Availability Zones have modified.

The next screenshot reveals the standing earlier than injecting failure.

The next screenshot reveals the standing after injecting failure.

Restore a desk from snapshot

You may restore a single desk from a snapshot out of your Multi-AZ cluster. Whenever you restore a single desk from a snapshot, you specify the supply snapshot, database, schema, and desk identify, and the goal database, schema, and a brand new desk identify for the restored desk.

To revive a desk from a snapshot:

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. Choose your cluster and navigate to the cluster particulars web page.
3. On the Actions menu, select Restore desk.
4. Enter the details about which snapshot, supply desk, and goal desk to make use of, after which select Restore desk.

Allow public connections in your Multi-AZ information warehouse

1. From the navigation menu, select CLUSTERS.
2. Select the Multi-AZ cluster that you just wish to modify.
3. Select Actions.
4. Select Activate Publicly accessible.
5. Select Elastic IP tackle, if you don’t select one, an tackle might be randomly assigned to you.
6. Select Save adjustments.

Monitor queries for Multi-AZ deployments

A Multi-AZ deployment makes use of compute assets which are deployed in each Availability Zones and might proceed working within the occasion that the assets in a given Availability Zone will not be accessible. All of the compute assets are used always, which permits full operation throughout two Availability Zones in each learn and write operations.

You may question SYS_views within the pg_catalog schema to observe Multi-AZ question runs. The SYS_views cowl question run actions and stats from main and secondary clusters.

The next are the system tables within the SYS_view record:

Comply with these steps to observe the question run on Multi-AZ deployment from the Amazon Redshift Console:

1. On the Amazon Redshift console, hook up with the database in your Multi-AZ deployment and run queries by way of the question editor.
2. Run any pattern question on the Multi-AZ Redshift deployment.
3. For a Multi-AZ deployment, you’ll be able to establish a question and the Availability Zone the place it’s being run (working on the first or secondary availability zone) through the use of the compute_type column within the SYS_QUERY_HISTORY desk. The legitimate values for the compute kind column are as follows:
   1. main – When run on main availability zone within the Multi-AZ deployment.
   2. secondary – When run on secondary availability zone within the Multi-AZ deployment.

The next is a pattern question utilizing the compute_type column to observe a question:

dev=# choose (compute_type) as compute_type, left(query_text, 50) query_text from sys_query_history order by start_time desc;

 compute_type | query_text
--------------+----------------------------------------------------
 secondary    | choose rely(*) from t1;
 main 	   choose rely(*) from t2;

You can even entry the question historical past from the console to research your question diagnostics.

4. On the Question monitoring tab, select Hook up with database.

5. For Connection, select Create a brand new connection
6. For Authentication, select Short-term credentials
7. For Database identify, enter the database identify (for instance, dev).
8. For Database person, enter the database person identify (for instance, awsuser).
9. Select Join.

After you’re linked, below Question Monitoring, on the Question historical past tab, you’ll be able to view all of the queries and masses, as proven within the following screenshot.

Beneath Metric filters, you should utilize the assorted filters within the Extra filtering choices part to view question historical past based mostly on Time interval, Customers, Databases, or SQL instructions.

There are a couple of limitations when working with Amazon Redshift Multi-AZ in preview mode, refer right here for the constraints.

Buyer suggestions

Janssen Prescribed drugs, a subsidiary of Johnson & Johnson, researches and manufactures medicines with a concentrate on the altering wants of sufferers and the healthcare business. “Janssen Pharmaceutical makes use of Amazon Redshift to allow vital insights that drive essential enterprise choices for our information scientists, information stewards, enterprise customers, and exterior stakeholders. With Amazon Redshift Multi-AZ, we could be assured that our information warehouse will at all times be accessible with none disruptions which may delay affect our potential to make vital enterprise choices.” – Shyam Mohapatra, Director of Info Expertise – Janssen Pharmaceutical Corporations of Johnson & Johnson

Stripe is a know-how firm that builds financial infrastructure for the web. Stripe’s merchandise energy funds for on-line and in-person retailers, subscriptions companies, software program platforms and marketplaces, and every thing in between. “Hundreds of thousands of firms use Stripe’s software program and APIs to just accept funds, ship payouts, and handle their companies on-line.  Entry to their Stripe information through main information warehouses like Amazon Redshift has been a high request from our prospects. Our prospects wanted extremely accessible, safe, quick, and built-in analytics at scale with out constructing advanced information pipelines or shifting and copying information round. With Stripe Information Pipeline for Amazon Redshift, we’re serving to our prospects arrange a direct and dependable information pipeline in a couple of clicks.  Stripe Information Pipeline permits our prospects to routinely share their full, up-to-date Stripe information with their Amazon Redshift information warehouse, and take their enterprise analytics and reporting to the subsequent stage.” – Brian Brunner, Senior Supervisor, Engineering at Stripe

Conclusion

This publish demonstrated the best way to configure an Amazon Redshift Multi-AZ deployment in two Availability Zones and check the fault tolerance of your workloads throughout an unlikely failure of an Availability Zone. Amazon Redshift Multi-AZ deployment additionally helps enhance total efficiency of your information warehouse as a result of compute nodes in each Availability Zones are used for learn and write operations. Amazon Redshift Multi-AZ information warehouse helps meet the calls for of consumers with mission vital analytics functions that require the best ranges of availability and resiliency. For extra particulars, refer Configuring Multi-AZ deployment.

In regards to the Authors

Ranjan Burman is an Analytics Specialist Options Architect at AWS. He makes a speciality of Amazon Redshift and helps prospects construct scalable analytical options. He has greater than 16 years of expertise in numerous database and information warehousing applied sciences. He’s enthusiastic about automating and fixing buyer issues with cloud options.

Saurav Das is a part of the Amazon Redshift Product Administration group. He has greater than 16 years of expertise in working with relational databases applied sciences and information safety. He has a deep curiosity in fixing buyer challenges centered round excessive availability and catastrophe restoration.

Anusha Challa is a Senior Analytics Specialist Options Architect targeted on Amazon Redshift. She has helped many purchasers construct large-scale information warehouse options within the cloud and on premises. She is enthusiastic about information analytics and information science.

Nita Shah is an Analytics Specialist Options Architect at AWS based mostly out of New York. She has been constructing information warehouse options for over 20 years and makes a speciality of Amazon Redshift. She is targeted on serving to prospects design and construct enterprise-scale well-architected analytics and resolution assist platforms.

Suresh Patnam is a Principal BDM – GTM AI/ML Chief at AWS. He works with prospects to construct IT technique, making digital transformation by way of the cloud extra accessible through the use of information and AI/ML. In his spare time, Suresh enjoys taking part in tennis and spending time along with his household.