Getting Started with wpipe

Welcome to the future of Python orchestration. This guide will help you set up wpipe v2.4.0-LTS and execute your first industrial-grade pipeline in minutes.

⚡ Fast-Track to Power

WPipe is designed for engineers who value performance, zero-boilerplate, and resilience. Let's get your environment ready.

1. Installation

Install the core engine via PyPI. We recommend using a virtual environment for your projects.

Standard

pip install wpipe

Full (Dev/Docs)

pip install "wpipe[dev,docs]"

Source

git clone https://github.com/wisrovi/wpipe
cd wpipe
pip install -e .

2. Verification

Ensure the engine is correctly synchronized with your system.

import wpipe
from wpipe import Pipeline

print(f"🚀 WPipe v{wpipe.__version__} - Engine Synchronized.")
# Expected: 🚀 WPipe v2.4.0 - Engine Synchronized.

3. Build Your First Pipeline

Follow these three steps to master the basic mental model.

Create pure Python functions that return a dictionary (the context update).

🏗️ 1. Define Steps

def fetch(data):
    return {"user": "wisrovi"}

Instantiate the Pipeline and register your steps with version control.

🧩 2. Orchestrate

pipe = Pipeline(verbose=True)
pipe.set_steps([
    (fetch, "FetchUser", "v1.0")
])

Invoke the .run() method with initial data. Watch the magic happen.

⚡ 3. Execute

result = pipe.run({})
# {'user': 'wisrovi'}

4. The “Hello World” of Orchestration

Here is a complete, production-ready example demonstrating data accumulation.

from wpipe import Pipeline, step

# Each step automatically receives all data from previous steps
@step(name="Step_Alpha", version="1.0.0")
def step_alpha(data):
    return {"a": 10}

@step(name="Step_Beta", version="1.0.0")
def step_beta(data):
    # Accessing data['a'] from previous step
    return {"b": data["a"] * 2}

if __name__ == "__main__":
    # Initialize engine
    engine = Pipeline(pipeline_name="MyFirstLTS", verbose=True)

    # Build the chain
    engine.set_steps([
        step_alpha,
        step_beta
    ])

    # Execute
    final_warehouse = engine.run({"input": "raw_signal"})

    print(f"Result: {final_warehouse}")
    # Result: {'input': 'raw_signal', 'a': 10, 'b': 20}

5. Enterprise Capabilities at a Glance

WPipe v2.4.0-LTS is not just about sequences. It’s about industrial reliability.

Capability	Why it matters
Parallel Execution	Run independent tasks in Threads or Processes with zero complexity.
Smart Checkpoints	Auto-resume from milestones if the system crashes or fails.
SQLite WAL Mode	Ultra-fast persistence with zero-lock concurrency.
Dashboard	Real-time visual monitoring of your entire pipeline fleet.