Zerto Automation: Failback Automation using Ansible DR-as-code

[Mihir10]

What is Zerto? 

Zerto is an enterprise-grade disaster recovery (DR) and data protection platform designed for virtualized and cloud environments. It provides continuous data replication, near-zero recovery point objectives (RPO), and automated failover/failback capabilities across sites. 

Unlike traditional backup solutions, Zerto uses continuous block-level replication, ensuring minimal data loss and rapid recovery during outages. 

What is a Failback? 

A Failback is the controlled return of workloads from the disaster recovery (DR) site back to their original production site after a failover event. 

In Zerto terms, a failback is a Move operation: 

• The workload is migrated from its current (recovery) site back to the original protected site 

• Reverse protection is established so replication continues in the homeward direction 

• DNS/IP records are repointed back to the home site 

• The environment returns to its normal operating state 

In simple terms, failback is the mirror image of failover — same safety gates, opposite direction. 

What Happens During a Failback? 

When a failback is executed: 

• Zerto initiates a Move with manual commit (commitPolicy 0) 

• The workload migrates back to the original site 

• Zerto pauses at "MovingBeforeCommit" — the safety window 

• Infoblox swaps DNS/IP records back to the home-site IPs 

• The move is committed (or rolled back if the swap fails) 

• Reverse protection is re-established 

Why Automated Failback Matters 

• Ensures consistent return of workloads after DR events 

• Provides the same all-or-nothing safety gate as failover 

• Eliminates manual DNS repointing errors during return migration 

• Enables parallel failback of multiple VPGs simultaneously 

• Prevents half-migrated states where VMs have moved but DNS has not 

Challenges 

Manual failback is risky due to: 

• Reversed topology — ownership and direction checks differ from failover 

• Two-system coordination — Zerto and Infoblox must act in lockstep (again) 

• Timing sensitivity — DNS must be swapped inside the MovingBeforeCommit window 

• Multi-VPG scale — entire EPRIDs need to come home together 

• Risk of leaving workloads stranded at the DR site if something fails partially 

Architecture 

The automation uses a fan-out model. An orchestrator parses the Excel workload definition, groups VPGs by EPRID, and launches parallel worker jobs, one per VPG. Each worker initiates a Zerto Move, pauses at the commit gate, performs Infoblox IP reassignment, and commits or rolls back based on validation. The orchestrator then merges all worker results and sends a consolidated report.

Figure – Failback flow 
 
Step 1: Input Processing (Targets from the Workbook) 

• Excel input sheet acts as the single source of truth 

• Parsed using Python (custom module) + Ansible 

• Filtered to the selected EPRID, grouped by VPG 

Key fields per workload: 

• vpg_name — VPG to move back to the original site 

• zvm_fqdn — ZVM currently hosting the recovered workload 

• eprid — Application grouping selected at launch 

• expected_ip_address / expected_dr_ip_address — Source & DR IPs used to drive the swap 

• hostname / fqdn — DNS records repointed during failback 

Step 2: Credential Validation 

• Orchestrator collects every unique ZVM that will be contacted 

• Verifies a matching Zerto_API credential exists in AAP 

• If any credential is missing, the run stops immediately 

• Builds a ZVM-FQDN -> credential-ID map 

• Injects the correct credential into each worker job at launch 

Step 3: Zerto Move (With Before-Commit Safety Gate) 

The failback flow authenticates and runs pre-flight checks that reflect the reversed topology: 

Health — VPG must be actively replicating (MeetingSLA) 

Ownership — This site must now be the protected site for the VPG (only true if the earlier failover established reverse protection) 

Topology — Protected and recovery sites must still differ 

The Move is issued with commitPolicy 0 (manual commit), creating the before-commit pause: 

GET  /v1/vpgs/{vpg_id}     # MeetingSLA? protected vs recovery site? 
GET  /v1/localsite          # confirm THIS site is now the protected site 
assert status == 'MeetingSLA' and local_site == protected_site 
 
POST /v1/vpgs/{vpg_id}/Move 
     body: { commitPolicy: 0, 
             reverseProtection: <bool>, 
             keepSourceVms: <bool>, 
             forceShutdown: <bool> } 
# poll until SubStatus == MovingBeforeCommit 

commitPolicy 0 is what creates the before-commit pause. Without it, Zerto would finalise the failback before DNS could be repointed. 

Step 4: Infoblox IP/DNS Swap 

With the VPG paused at MovingBeforeCommit, Infoblox performs the same defensive cross-over used in failover — only now repointing records back toward the home site: 

• Fetch current Host/A records for source and DR hostnames via WAPI 

• Pre-validate: confirm records currently hold expected IPs 

• Reassign: WAPI PUT calls cross-over so production hostname resolves to home-site IP 

• Post-validate: re-read records, produce PASS/FAIL verdict 

GET  /wapi/<ver>/record:host  (source & DR hostnames)   # fetch 
# pre-validate: source_old_ip == expected, dr_old_ip == expected 
 
PUT  /wapi/<ver>/<source_ref>  body: { ipv4addr: <home_IP> } 
PUT  /wapi/<ver>/<dr_ref>      body: { ipv4addr: <dr_IP> } 
# post-validate -> infoblox_final_status = PASS | FAIL 

Step 5: Commit or Rollback (Data-Driven Decision) 

The Infoblox validation result drives the Zerto decision: 

Swap PASS → MoveCommit → Poll VPG to MeetingSLA → Workload live at home site, reverse protection active 

Swap FAIL → Move rollback (Zerto) → Infoblox rollback (revert IPs) → Workload stays at DR site, environment unchanged 

The before-commit gate guarantees the failback is atomic: either the workload is fully home with DNS pointing at it, or it is fully back at DR with the original records intact. 

Failover vs Failback at a Glance 

• Zerto operation: Failover vs Move (commitPolicy 0) 

• Direction: Protected -> Recovery vs Recovery -> Protected (home) 

• Triggered from: Recovery site ZVM vs Current site (now protected) ZVM 

• Pause point: BeforeCommit vs MovingBeforeCommit 

• Infoblox: Cross-over to DR IPs vs Cross-over back to home IPs 

• Commit gate: Infoblox PASS -> commit (both) 

• On failure: FailoverRollback + IP revert vs Move rollback + IP revert 

Reporting & Observability 

• Each worker exports a structured artifact: 

  - VPG name, stage reached, commit/rollback flags 

  - Final Zerto status, full Infoblox status 

• Orchestrator merges all artifacts into a consolidated view 

• Sends Infoblox summary email + overall failback summary email 

• If any VPG failed, the job fails at the end after reporting 

• If a worker dies before producing artifacts, a stub result is synthesised so no VPG silently disappears 

Key Takeaways 

By combining Ansible automation, structured inputs, and Zerto + Infoblox APIs, failback evolves from a manual, error-prone return migration into a repeatable, scalable, and enterprise-grade automation workflow, with the same all-or-nothing safety gate that ensures environments are never left in an inconsistent state.

[Thaufique_Mod]

Thanks for sharing this informative post. It is ivery helpful.