Decide Whether a Failed Automation Can Be Retried Safely
Prepared with AI assistance by Alactic Editorial. Examples identified as hypothetical are illustrative, not customer results.
A workflow reports failure after creating a follow-up task but before recording that task’s reference. Rerunning the workflow might create a second task. The error is real, but it does not tell you whether the business action happened.
Retry only when you can show that the retry will not repeat an unwanted business action. Classify the failed run as safe to retry when nothing happened and nothing remains in progress, or when a verified replay safeguard will preserve the completed result. Require reconciliation when completion is uncertain. Require manual recovery when an action already happened and the available retry cannot reliably skip or preserve it.
The useful question is not simply whether the workflow failed. It is: What changed, what might still change, and what would this particular retry repeat? The method below tests that question at the business-action level. It does not cover retry timing, queue management, or whether a platform offers automatic retries.
1. Define the business action before testing the retry
A workflow can contain several actions with different recovery needs. Reading a record is not equivalent to creating a task, changing an owner, or sending a customer communication. A single run-level status hides these differences.
Write down each action that leaves a lasting effect. For each one, identify:
- The intended result: what should exist or change after success.
- The destination: where that result can be checked independently of the failed run.
- The duplicate consequence: what happens if the action occurs twice.
- The completion evidence: a destination record, event, reference, or other observable result.
- The replay behavior: whether repeating the action creates another result, preserves the existing one, or causes a different effect.
Do not assume that writing the same value twice is harmless. A repeated field update might leave the field unchanged while triggering another notification or downstream workflow. Test the business effects, not just the final field value.
Also define what the retry actually restarts. It might repeat the whole workflow, resume from a checkpoint, or rerun only one action. A safe decision for one restart boundary does not establish safety for another.
2. Build a completion record from observable evidence
For every action in the failed run, assign one of three observed states: confirmed absent, confirmed completed, or uncertain. Record the evidence and the time it was checked.
Confirmed absent needs more than an empty search result. Establish that the action was never submitted, or that the destination definitively rejected it without creating the result. If an earlier request could still complete, absence is not yet confirmed.
Confirmed completed means the intended business result can be identified. A workflow log saying that an action started is not enough. Neither is a request reference that only proves receipt. Match the destination result to the intended record, action, and run where possible.
Uncertain covers conflicting evidence, a lost response, an incomplete history, or a result that cannot be uniquely matched. Keep this state explicit. Renaming it “probably failed” encourages unsafe reruns.
Capture the original attempt and any retries together. Otherwise, an operator might find no result for the latest attempt while overlooking a result created by the first one. Protect the evidence from accidental loss while investigating, and use only authorised access to the affected records.
3. Use this filled hypothetical retry decision table
Fictional worked example: A test workflow reads a CRM lead, creates a follow-up task, and then updates the lead’s follow-up status. These are hypothetical operations, not a description of a specific product capability. The test uses synthetic records and an isolated test destination.
The intended outcome is one follow-up task associated with the test lead and a status showing that follow-up has been arranged. The cases below are separate test runs.
Case | Observed completion | Decision | Required checks A: Failure before action | No task request submitted; no write started | Safe to retry | Confirm stopped execution, no pending action, unchanged eligibility B: Failure after action | Task confirmed; status update failed | Manual recovery | Match task to lead, verify status, establish whether task creation can be skipped C: Uncertain completion | Task request sent; response lost | Reconciliation required | Inspect destination history, match result, resolve any in-progress request D: Failure during retry | Retry created a second task, then failed | Manual recovery | Stop further attempts, inventory both tasks and both runs, choose retained result
Case A: failure before any business action
In this hypothetical run, input validation stops execution before task creation. The execution history confirms that no write request was submitted, and no action remains active. The test lead is still eligible for follow-up.
Decision: safe to retry after correcting the input. Check that the correction has not changed the intended recipient, record, or action. Then authorise one new attempt and inspect the destination afterward. If the history cannot establish that execution stopped before submission, move the case to reconciliation instead.
Case B: failure after a confirmed action
In this hypothetical run, task T-104 exists and is linked to the intended test lead. The later status update failed. A full rerun would create another task because this test design has no verified replay safeguard.
Decision: manual recovery for the available full-run retry. Keep the existing task and complete only the missing status change through an authorised recovery path. If a supported action-level resume can skip task creation, verify that behavior separately before using it. A partial failure is not a reason to repeat the successful part.
Case C: uncertain completion
In this hypothetical run, the task request was submitted, but its response was lost. The first destination search finds nothing. That does not establish whether creation failed, remains in progress, or has not yet become visible.
Decision: reconciliation required. Inspect the destination’s available history and correlate any task with the original attempt. Do not create a new task merely because the workflow lacks a success response. If completion remains unknowable, keep the retry blocked and assign an owner to decide the recovery path.
Case D: failure during retry
In this hypothetical run, an operator reran the workflow after a partial failure. The retry created task T-105 alongside the original T-104, then failed before updating the lead.
Decision: manual recovery. Stop further attempts. Inspect both tasks for activity or dependent work before deciding which to retain. Resolve the duplicate through an authorised process, finish the missing status update, and record the disposition of both attempts. Another rerun would compound the problem rather than repair it.
4. Reconcile uncertainty before choosing a recovery action
Reconciliation is an evidence-gathering step, not a softer name for retrying. Its purpose is to turn an uncertain action into a confirmed result or a documented exception that needs human handling.
- Hold additional attempts. Avoid adding new effects while investigating earlier ones.
- Collect the attempt history. Include the original run, operator reruns, submitted requests, and known destination references.
- Inspect the destination. Search using stable business identifiers and action details, not only a broad name match.
- Check pending work. Determine whether an earlier action could still complete. If that cannot be established, retain the uncertainty.
- Record the finding. State which action completed, which did not, and what evidence supports each conclusion.
- Choose a recovery boundary. Retry only the confirmed missing work if the available mechanism supports that safely; otherwise use manual recovery.
Matching requires care. Two tasks with the same title might represent different legitimate actions. Conversely, a duplicate may have a different timestamp or owner. Use enough context to distinguish the intended business event from unrelated activity.
When evidence conflicts, prefer a documented hold over an unsupported declaration of failure. Give the hold a named owner and a review point so that uncertainty does not become an abandoned operational backlog.
5. Test replay safeguards rather than trusting their names
A replay safeguard aims to make repeated attempts preserve one intended business outcome. It might use a stable action identifier, a destination uniqueness rule, or a supported resume mechanism. These are recommended design options, not assumptions about available platform controls.
A useful test repeats the same intended action after interrupting the workflow immediately after destination completion but before local success is recorded. Inspect whether the next attempt preserves the original result or creates another one.
Check the safeguard’s boundaries:
- Does the same intended action retain the same identity across retries?
- Can separate legitimate actions still produce separate results?
- Does a retry preserve the original destination reference?
- Can two overlapping attempts both pass an existence check and create duplicates?
- Does repeating an update trigger additional downstream work?
- Does changed input cause a visible exception rather than silently reusing an unrelated result?
A “check whether it exists, then create it” design can still fail if another attempt creates the result between those steps. Sequential testing alone will not expose that failure mode.
There is a trade-off: stopping on ambiguous evidence slows recovery, while aggressive replay risks duplicate work. Verified replay protection can reduce that tension, but a passing test supports only the conditions actually tested. It does not establish that every action in the workflow is safe to repeat.
6. Make recovery narrower than the original workflow
Once an action is confirmed complete, preserve it unless there is a specific reason and authorisation to undo it. Recovery should target the missing or incorrect state, not recreate every preceding step.
Manual recovery may mean completing a missing status update, resolving a duplicate task, or escalating an unresolved action to the destination owner. It should not mean making undocumented edits until the run looks successful.
Before changing anything, record the intended final state and the actions required to reach it. Consider whether the original result has already been used. A task may have been assigned or acted on; deleting it can hide work or disrupt ownership.
Keep the distinction between repairing business state and repairing workflow records. Marking a run resolved does not complete a missing business action. Equally, completing the business action does not automatically correct a misleading execution history. Record both outcomes so that a later operator does not rerun the same work.
Eligibility also needs a fresh check. A retry may be duplicate-safe yet no longer appropriate because ownership, approval, recipient permission, or the underlying request has changed since the first attempt.
7. Turn the decision table into a repeatable test
Use isolated test records and authorised destinations. Where a test environment cannot reproduce an interruption safely, document the untested condition rather than claiming coverage.
For each business action, create a test case for failure before submission, after confirmed completion, after submission with an unknown response, and during a subsequent attempt. Record the available restart boundary in every case.
Judge the result using observable outcomes:
- Decision correctness: did the operator classify the run using completion evidence rather than the overall failure label?
- Duplicate effects: did the retry produce an additional unintended task, update-triggered action, or other lasting result?
- Recovery completeness: does the final business state match the intended outcome?
- Evidence quality: can another operator identify what happened without guessing?
- Control of uncertainty: were unresolved actions held for reconciliation rather than replayed?
Next action: select one workflow with a lasting business effect and fill the four-case decision table before authorising its next test rerun. Record the classification, supporting evidence, recovery owner, and observed final state for each case. Any case that cannot distinguish “not completed” from “completion unknown” remains a reconciliation case—not a safe retry.
Put the next conversation to work.
Explore Alactic Communications for WhatsApp conversations, campaigns and AI-assisted workflows. Choose the product and plan that fit your team.
Explore Communications