Handling Parallel Data ("Sandbox & Merge")

SignStack's PARALLEL container steps allow multiple tasks or sub-workflows to run simultaneously, significantly speeding up processes like group approvals. However, this introduces a challenge: how do you safely manage updates to the EntityContext when multiple branches might be changing data at the same time?

SignStack solves this using a robust "Sandbox & Merge" pattern. Each parallel branch works in isolation (its "sandbox"), and a dedicated mapper on the parent container step intelligently merges the results.

Think of it like Git branching and merging 🌳. Each parallel step is a separate branch making its own commits. The final step is merging those branches back into the main line.

The Challenge: Parallel Updates

Imagine a PARALLEL step with two child tasks:

financeReviewStep: Updates entities.dealInfo.financeNotes.
legalReviewStep: Updates entities.dealInfo.legalNotes.

If both steps tried to update the same central dealInfo entity directly, you'd have a race condition – the final state would depend entirely on which step finished last, potentially overwriting the other's changes.

The Solution: Sandboxed Evolution

SignStack prevents race conditions by giving each parallel branch its own isolated view or "sandbox" of the EntityContext.

Branching: When the parent PARALLEL container step starts, each child step (financeReviewStep, legalReviewStep) receives a copy of the EntityContext as it existed when the parent started.
Isolated Updates: When financeReviewStep completes, its onCompleteMapper updates only its own sandboxed copy of the dealInfo entity. legalReviewStep does the same in its separate sandbox. The changes are isolated.

The Merge: Using the Parent's `onCompleteMapper`

The crucial step happens after the PARALLEL container finishes (based on its completionRule). The onCompleteMapper defined on the parent PARALLEL container itself is executed. Its job is to merge the results from the child branches.

Input Context for the Parent Mapper:

This mapper receives a special context object containing:

entities: The state of the EntityContext as it existed before the parallel step started (the "base" branch).
stepOutputs: An object containing the final EntityContext state produced by each completed child step. The keys are the keys of the child steps.

Example stepOutputs Context:

{
  "financeReviewStep": { // Key of the first child step
    "dealInfo": { "financeNotes": "Approved", /* ...other dealInfo properties... */ },
    "clientInfo": { /* ... clientInfo state from this branch ... */ }
  },
  "legalReviewStep": { // Key of the second child step
    "dealInfo": { "legalNotes": "Minor revisions needed", /* ...other dealInfo properties... */ },
    "clientInfo": { /* ... clientInfo state from this branch ... */ }
  }
}

Writing Merge Expressions

The mapperExpression on the parent container uses this context to define the merge logic.

Example: Merging Notes into dealInfo

// onCompleteMapper on the PARENT PARALLEL container step
"onCompleteMappers": [
  {
    "targetEntityKey": "dealInfo",
    "mapperExpression": "$merge([\
        entities.dealInfo, // Start with the original state\
        { 'financeNotes': stepOutputs.financeReviewStep.dealInfo.financeNotes }, // Add finance notes\
        { 'legalNotes': stepOutputs.legalReviewStep.dealInfo.legalNotes } // Add legal notes\
      ])"
  }
  // Potentially another mapper to merge changes to clientInfo if needed
]

Explanation: This expression starts with the original dealInfo entity (entities.dealInfo) and uses $merge to layer on the specific financeNotes from the finance branch's output and the legalNotes from the legal branch's output, creating a single, consolidated dealInfo entity for the subsequent steps.

Handling Conflicts & Advanced Strategies

Last Write Wins (Implicit): If multiple branches update the same property (e.g., both try to set dealInfo.status), the simple $merge shown above will effectively result in "last write wins" based on the order in the $merge array.
Explicit Logic: You can write more complex JSONata expressions to handle conflicts explicitly (e.g., combine comments, prioritize one branch's update based on a condition, flag conflicts for manual review by setting a specific status).
completionRule Impact: If using ANY or QUORUM, your merge expression needs to be aware that stepOutputs might not contain entries for all child steps. Use $exists() or other checks as needed.

The "Sandbox & Merge" pattern, powered by the parent container's onCompleteMapper and the stepOutputs context, provides a robust and explicit way to manage data consistency in complex parallel workflows.