wpipe
📚 Documentation
Getting Started with
wpipe
1. Installation
2. Verification
3. Build Your First Pipeline
4. The “Hello World” of Orchestration
5. Enterprise Capabilities at a Glance
Next Steps
Installation Guide
1. Requirements
2. Standard Installation
3. Advanced Installation
4. Containerized (Docker)
5. Verification
6. Troubleshooting
Mastering
wpipe
: Usage Guide
1. Fundamental Step Types
2. Advanced Control Flow
3. Enterprise Resiliency
4. High Performance
5. Operational Integration
User Guide
1. Getting Started
Pipeline Basics
2. Core Features
Conditions
Retry Logic
API Integration
SQLite Storage
YAML Configuration
Nested Pipelines
Error Handling
3. Advanced Topics
Best Practices
Troubleshooting
Pipeline Basics
Conditional Branching
Retry Logic
API Integration
SQLite Storage
YAML Configuration
Nested Pipelines
Error Handling
Best Practices
Troubleshooting
WPipe Academy: Master Your Pipelines
La Senda del Maestro: 140 Niveles de Poder
Listado Detallado de Niveles
Nivel 1: Conceptos Básicos de Pipeline
Nivel 2: Metadatos y Decorador @step
Nivel 3: demo_level3.py
Nivel 4: demo_level4.py
Nivel 5: demo_level5.py
Nivel 6: demo_level6.py
Nivel 7: demo_level7.py
Nivel 8: demo_level8.py
Nivel 9: demo_level9.py
Nivel 10: demo_level10.py
Nivel 11: demo_level11.py
Nivel 12: demo_level12.py
Nivel 13: demo_level13.py
Nivel 14: demo_level14.py
Nivel 15: demo_level15.py
Nivel 16: demo_level16.py
Nivel 17: demo_level17.py
Nivel 18: demo_level18.py
Nivel 19: demo_level19.py
Nivel 20: demo_level20.py
Nivel 21: demo_level21.py
Nivel 22: demo_level22.py
Nivel 23: demo_level23.py
Nivel 24: demo_level24.py
Nivel 25: Super-Computación (Threads vs Processes)
Nivel 26: demo_level26.py
Nivel 27: demo_level27.py
Nivel 28: demo_level28.py
Nivel 29: demo_level29.py
Nivel 30: demo_level30.py
Nivel 31: demo_level31.py
Nivel 32: demo_level32.py
Nivel 33: demo_level33.py
Nivel 34: demo_level34.py
Nivel 35: demo_level35.py
Nivel 36: demo_level36.py
Nivel 37: demo_level37.py
Nivel 38: demo_level38.py
Nivel 39: demo_level39.py
Nivel 40: demo_level40.py
Nivel 41: demo_level41.py
Nivel 42: demo_level42.py
Nivel 43: demo_level43.py
Nivel 44: demo_level44.py
Nivel 45: demo_level45.py
Nivel 46: demo_level46.py
Nivel 47: demo_level47.py
Nivel 48: demo_level48.py
Nivel 49: demo_level49.py
Nivel 50: Integración Final - El Viaje Completo
Nivel 51: demo_level51.py
Nivel 52: demo_level52.py
Nivel 53: demo_level53.py
Nivel 54: demo_level54.py
Nivel 55: demo_level55.py
Nivel 56: demo_level56.py
Nivel 57: demo_level57.py
Nivel 58: demo_level58.py
Nivel 59: demo_level59.py
Nivel 60: demo_level60.py
Nivel 61: demo_level61.py
Nivel 62: demo_level62.py
Nivel 63: demo_level63.py
Nivel 64: demo_level64.py
Nivel 65: demo_level65.py
Nivel 66: demo_level66.py
Nivel 67: demo_level67.py
Nivel 68: demo_level68.py
Nivel 69: demo_level69.py
Nivel 70: demo_level70.py
Nivel 71: demo_level71.py
Nivel 72: demo_level72.py
Nivel 73: demo_level73.py
Nivel 74: demo_level74.py
Nivel 75: Orquestación Híbrida (For + Parallel)
Nivel 76: demo_level76.py
Nivel 77: demo_level77.py
Nivel 78: demo_level78.py
Nivel 79: demo_level79.py
Nivel 80: demo_level80.py
Nivel 81: demo_level81.py
Nivel 82: demo_level82.py
Nivel 83: demo_level83.py
Nivel 84: demo_level84.py
Nivel 85: demo_level85.py
Nivel 86: demo_level86.py
Nivel 87: demo_level87.py
Nivel 88: demo_level88.py
Nivel 89: demo_level89.py
Nivel 90: demo_level90.py
Nivel 91: demo_level91.py
Nivel 92: demo_level92.py
Nivel 93: demo_level93.py
Nivel 94: demo_level94.py
Nivel 95: demo_level95.py
Nivel 96: demo_level96.py
Nivel 97: demo_level97.py
Nivel 98: demo_level98.py
Nivel 99: demo_level99.py
Nivel 100: Auditoría de Seguridad (Alerts History)
Nivel 101: demo_level101.py
Nivel 102: demo_level102.py
Nivel 103: demo_level103.py
Nivel 104: demo_level104.py
Nivel 105: demo_level105.py
Nivel 106: demo_level106.py
Nivel 107: demo_level107.py
Nivel 108: demo_level108.py
Nivel 109: demo_level109.py
Nivel 110: demo_level110.py
Nivel 111: demo_level111.py
Nivel 112: demo_level112.py
Nivel 113: demo_level113.py
Nivel 114: demo_level114.py
Nivel 115: demo_level115.py
Nivel 116: demo_level116.py
Nivel 117: demo_level117.py
Nivel 118: demo_level118.py
Nivel 119: demo_level119.py
Nivel 120: demo_level120.py
Nivel 121: demo_level121.py
Nivel 122: demo_level122.py
Nivel 123: demo_level123.py
Nivel 124: demo_level124.py
Nivel 125: demo_level125.py
Nivel 126: demo_level126.py
Nivel 127: demo_level127.py
Nivel 128: demo_level128.py
Nivel 129: demo_level129.py
Nivel 130: Integración de API y Métricas de Negocio
Nivel 131: Background Tasks - Ejecución No Bloqueante
Nivel 132: Background con Captura de Errores
Nivel 133: Múltiples Background Tasks
Nivel 134: Background con Pipeline Anidado
Nivel 135: Background para Logging/Telemetría
Nivel 136: Background en PipelineAsync
Nivel 137: Background con Tuple Steps
Nivel 138: Background dentro de Condition
Nivel 139: Background sin capture_error (Default)
Nivel 140: El Zen de WPipe - Demo Final de Background Tasks
📜 Próximos Pasos
Basic Pipeline Tutorial
1. Introduction and Overview
1.1 What is a Pipeline?
1.2 What You’ll Build
2. Environment Setup
2.1 Prerequisites
2.2 Installation
3. Your First Pipeline
3.1 Basic Structure
3.2 Step-by-Step Implementation
4. Understanding Data Flow
4.1 How Data Accumulates
4.2 Practical Example of Data Evolution
5. Passing Initial Data
5.1 Providing Input Data
5.2 Use Case: Parameterized Pipelines
6. Step Function Patterns
6.1 Lambda Functions
6.2 Class-Based Steps
7. Error Handling
7.1 Basic Error Handling
7.2 Validation Errors
8. Advanced Features
8.1 Conditional Execution
8.2 Retry Logic
9. Testing Your Pipeline
9.1 Unit Testing Steps
9.2 Integration Testing
10. Best Practices Summary
11. Exercises
11.1 Exercise 1: String Processing Pipeline
11.2 Exercise 2: Temperature Converter
11.3 Exercise 3: Shopping Cart
12. Next Steps
Class-Based Steps Tutorial
1. Introduction
2. The __call__ Protocol
3. Creating Step Classes
3.1 Basic Step Class
3.2 Configurable Step Class
4. Stateful Steps
4.1 Running Total
4.2 Accumulator Pattern
5. Composable Steps
5.1 Pipeline Building Blocks
6. Class Methods as Steps
7. Best Practices
7.1 Keep Steps Focused
7.2 Use Type Hints
7.3 Document Your Classes
8. Advanced Patterns
8.1 Step Factory
8.2 Decorator Pattern
9. Complete Example
10. Next Steps
API Integration Tutorial
1. Introduction
2. Understanding the API Client
3. Worker Registration
3.1 Basic Registration
3.2 Setting Worker ID
4. Process Tracking
4.1 Starting a Process
4.2 Process Flow
5. API Methods Reference
5.1 Worker Methods
5.2 Process Methods
5.3 Task Methods
6. Custom API Implementation
6.1 Extending APIClient
7. Error Handling with API
7.1 Graceful Degradation
8. Complete Example
9. Testing API Integration
9.1 Mock Server
10. Security Best Practices
10.1 Token Management
10.2 HTTPS Only
11. Troubleshooting
11.1 Common Issues
11.2 Debug Mode
12. Next Steps
Error Handling Tutorial
1. Introduction
2. Understanding Exceptions
2.1 TaskError
2.2 Error Codes
3. Raising TaskError
3.1 Basic Usage
3.2 With Step Information
4. Catching Errors
4.1 Basic Catch
4.2 Specific Error Codes
5. Error Recovery
5.1 Graceful Degradation
5.2 Fallback Values
6. Validation Patterns
6.1 Early Validation
6.2 Chain Validation
7. API Error Handling
7.1 Handle API Failures
7.2 Timeout Handling
8. Custom Error Classes
8.1 Define Custom Errors
8.2 Use Custom Errors
9. Error Logging
9.1 Log Errors
9.2 Error Context
10. Complete Example
11. Best Practices
12. Next Steps
Retry Logic Tutorial
1. Introduction
2. Built-in Retry Support
2.1 Configuring Retries
2.2 How It Works
3. Custom Retry Decorators
3.1 Basic Retry Decorator
3.2 Using the Decorator
4. Advanced Retry Patterns
4.1 Retry with Custom Logic
4.2 Retry with Logging
5. Step-Level Retries
5.1 Independent Retry Config
6. Handling Different Failure Types
6.1 Network Failures
6.2 Database Failures
7. Timeout Handling
7.1 Combined Retry and Timeout
8. Complete Example
9. Best Practices
10. Next Steps
Nested Pipelines Tutorial
1. Introduction
2. Basic Nested Pipeline
2.1 Creating Nested Pipelines
3. Pipeline Factory Pattern
3.1 Creating Reusable Pipelines
4. Complete Example
5. Best Practices
6. Next Steps
SQLite Integration Tutorial
1. Introduction
2. Basic SQLite Usage
2.1 Using Wsqlite
3. Querying Results
3.1 Reading Records
4. Advanced Patterns
4.1 Multiple Executions
5. Complete Example
6. Best Practices
7. Next Steps
YAML Configuration Tutorial
1. Introduction
2. Basic YAML Usage
2.1 Creating Configuration File
2.2 Loading Configuration
3. Environment Variables
3.1 Using env vars in YAML
4. Complete Example
5. Best Practices
6. Next Steps
Conditional Branches Tutorial
1. Introduction
1.1 When to Use Conditions
1.2 Basic Concept
2. Condition Class Reference
3. Basic Examples
3.1 Simple Boolean Check
3.2 String Matching
3.3 Without Else Branch
4. Complex Conditions
4.1 Multiple Comparisons
4.2 Numeric Ranges
4.3 In Operator
4.4 String Contains
5. Nested Conditions
5.1 Chained Conditions
5.2 Multiple Independent Conditions
6. Real-World Examples
6.1 User Registration Flow
6.2 Order Processing
6.3 Data Validation Pipeline
7. Best Practices
7.1 Keep Expressions Simple
7.2 Use Meaningful Variable Names
7.3 Order Conditions Strategically
7.4 Test Both Branches
8. Troubleshooting
8.1 Condition Not Evaluating
8.2 Type Mismatch
9. Complete Example
10. Next Steps
Advanced Patterns Tutorial
1. DAG Scheduling (Directed Acyclic Graphs)
2. Native Parallelism (IO vs CPU)
3. Forensic Error Capture
4. High-Performance Monitoring
5. Dynamic Pipeline Composition
Production Deployment Tutorial
1. Environment Setup
1.1 Production Requirements
2. Configuration Management
2.1 Environment-Based Config
3. Logging and Monitoring
3.1 Structured Logging
4. Error Handling in Production
4.1 Graceful Degradation
5. Health Checks
5.1 Health Check Endpoint
6. CI/CD Integration
6.1 GitHub Actions Example
7. Docker Deployment
7.1 Dockerfile Example
8. Complete Example
9. Best Practices
10. Next Steps
The Learning Tour
Core Curriculum
Recommended Learning Plan
Troubleshooting During Tutorials
Next Steps
API Technical Specification
1. Core Orchestration
2. Logical Control Blocks
3. Decorators & Intelligence
4. Persistence (WSQLite)
5. Industrial Observability
6. Error Architecture
7. Technical Utilities
Examples Gallery
01 Basic Pipeline (15 Examples)
02 API Pipeline (20 Examples)
03 Error Handling (15 Examples)
04 Conditional Branching (12 Examples)
05 Retry Logic (12 Examples)
06 SQLite Integration (14 Examples)
07 Nested Pipelines (14 Examples)
08 YAML Configuration (14 Examples)
09 Microservice (11 Examples)
Running Examples
Example Directory Structure
System Architecture: The WPipe Engine
1. The “Warehouse” Model
2. Core Internal Layers
3. Persistence Strategy: WSQLite
4. Resiliency & Reliability
4.1 Smart Checkpointing
4.2 Forensic Error Capture
5. Concurrency Model
6. Module Hierarchy
Next Steps
Best Practices for Industrial Pipelines
1. Step Architecture: Atomic & Pure
2. Data Contract Management
3. Resiliency & Failure Strategy
4. Performance & Scalability
5. Monitoring & Governance
FAQ
1. General Questions
1.1 What is wpipe?
1.2 What are the main features?
1.3 What is the license?
1.4 Is wpipe production-ready?
2. Installation Questions
2.1 What Python version is required?
2.2 How do I install wpipe?
2.3 How do I install from source?
2.4 What are the dependencies?
3. Getting Started Questions
3.1 How do I create a basic pipeline?
3.2 What is the data flow between steps?
3.3 Can I use classes as steps?
3.4 How do I run a pipeline?
4. API Integration Questions
4.1 How do I connect to an API?
4.2 How do I register a worker?
4.3 How do I perform health checks?
5. SQLite Questions
5.1 How do I persist results?
5.2 What data is stored?
5.3 How do I query results?
6. Error Handling Questions
6.1 How do I handle errors?
6.2 What error codes are available?
6.3 How do I create custom error codes?
7. Configuration Questions
7.1 How do I use YAML configuration?
7.2 Can I use environment variables?
8. Troubleshooting Questions
8.1 My pipeline is not connecting to the API
8.2 The pipeline is stuck on a step
8.3 I’m getting TaskError exceptions
8.4 How do I enable verbose output?
9. Performance Questions
9.1 How fast is wpipe?
9.2 Can I process large datasets?
9.3 How do I monitor memory usage?
10. Advanced Questions
10.1 Can I create conditional branches?
10.2 Can I nest pipelines?
10.3 Can I add callbacks?
11. Contributing Questions
11.1 How do I report bugs?
11.2 How do I contribute?
11.3 Where can I get help?
Glossary
1. Core Concepts
1.1 Pipeline
1.2 Step
1.3 Data Flow
1.4 Step Name
1.5 Step Version
2. API Integration
2.1 API Client
2.2 Worker
2.3 Worker ID
2.4 Health Check
2.5 Task Status
3. Data Persistence
3.1 SQLite
3.2 Execution Record
3.3 Wsqlite
4. Error Handling
4.1 TaskError
4.2 Error Codes
4.3 Validation Error
4.4 Retryable Error
5. Configuration
5.1 YAML Configuration
5.2 Environment Variable
5.3 Default Value
6. Progress and Logging
6.1 Verbose Mode
6.2 Progress Manager
6.3 Logger
7. Advanced Concepts
7.1 Nested Pipeline
7.2 Condition
7.3 Decorator
7.4 Callback
7.5 Idempotent Step
8. Testing Terms
8.1 Unit Test
8.2 Integration Test
8.3 Mock
9. Acronyms
10. Related Libraries
Contributing
1. Getting Started
1.1 Fork the Repository
1.2 Set Up Development Environment
1.3 Run Tests
2. Development Workflow
2.1 Create a Branch
2.2 Make Your Changes
2.3 Run Quality Checks
3. Testing Guidelines
3.1 Write Tests
3.2 Test File Naming
3.3 Run Specific Tests
4. Documentation
4.1 Update Documentation
4.2 Build Documentation
4.3 Documentation Style
5. Commit Guidelines
5.1 Commit Messages
5.2 Commit Message Format
6. Pull Request Process
6.1 Create Pull Request
6.2 Pull Request Template
6.3 What Happens Next
7. Reporting Issues
7.1 Bug Reports
7.2 Feature Requests
8. Code of Conduct
9. Questions?
Changelog
[1.5.3] - 2026-04-13
Added
[1.0.0] - 2026-03-22
Added
[0.1.7] - 2024-12-15
Added
Fixed
[0.1.6] - 2024-11-20
Added
[0.1.5] - 2024-10-10
Added
[0.1.0] - 2024-01-01
Added
wpipe
Index
Edit on GitHub
Index