Skip to main content

Technical Overview

Wix Pilot integrates seamlessly with your testing environment by combining natural language processing with robust testing capabilities.

Building Blocks of Wix Pilot​

To enable Wix Pilot to work harmoniously with your testing framework and app, it relies on several building blocks:

  • Dynamic Code Generation: Pilot generates testing code on-the-fly to perform actions or assertions based on your instructions.
  • Visual Analysis: Pilot can analyze the app's screen to verify the presence of specific elements or text, enabling assertions beyond standard UI checks.
  • App View Hierarchy: The testing framework generates an XML representation of the app's view hierarchy, helping Pilot interact with all UI elements, even those not directly visible.
  • Snapshot Images: Optional snapshot images provide Pilot with visual context for more precise understanding and analysis.
  • API Catalog: Pilot uses an extensible API catalog that can be enriched with custom testing commands and context.
  • Caching Mechanism: Pilot caches execution results to optimize performance and reduce unnecessary LLM calls.
  • Test Context Awareness: Pilot maintains awareness of previously executed steps, ensuring continuity and readability in the test flow.

Pilot's Execution Flow​

StartGather Context(App's UI State)Check Cache(Based on intent)Result in Cache?Retrieve Result(From Cache)Execute CachedDetox CodeProvide Feedback(Error / Result)Create LLM Prompt(Based on Context and Intent)Generate Detox Code(Using LLM)Execute GeneratedDetox CodeCache Generated CodeProvide Feedback(Error / Result)Next Step?EndYesNoNoYes

The execution flow of Wix Pilot can be broken down into these main steps:

  1. Initialize: Set up Pilot with your framework configuration using Pilot.init().
  2. Start Session: Begin a test flow with pilot.start().
  3. Execute Tests: Choose between:
    • Single steps using pilot.perform() for granular control
    • Automated flows using pilot.autopilot() for goal-driven testing
  4. End Session: Conclude the test flow with pilot.end().

Performance Optimization​

Pilot implements several optimization strategies:

  • Smart Caching: Temporary and permanent caching of execution results
  • Extensible API Catalog: Add custom commands to reduce LLM calls
  • Snapshot Management: Efficient handling of screen captures and comparisons

You can configure these optimizations through the Config options when initializing Pilot:

pilot.init({
  frameworkDriver: yourDriver,
  promptHandler: yourHandler,
  options: {
    cacheMode: 'full' | 'disabled',
    analysisMode: 'basic' | 'detailed'
  }
});