Data Pipeline Orchestrator
Pipelines
| ID |
Name |
Status |
Last Run |
Next Run |
Schedule |
Task Count |
| pipe-001 |
ETL - Sales Data |
active |
2026-07-11 03:55 |
2026-07-12 03:55 |
0 0 * * * |
4 |
| pipe-002 |
Stream - Events |
active |
2026-07-11 05:50 |
2026-07-11 05:55 |
*/5 * * * * |
2 |
| pipe-003 |
Batch - ML Features |
paused |
2026-07-10 05:55 |
2026-07-12 05:55 |
0 2 * * 0 |
3 |
Pipeline Tasks
| Pipeline ID |
Task ID |
Type |
Source |
Destination |
Status |
| pipe-001 |
extract |
jdbc |
sales_db |
s3_raw |
success |
| pipe-001 |
transform |
spark |
s3_raw |
s3_curated |
success |
| pipe-001 |
load |
jdbc |
s3_curated |
warehouse |
running |
Pipeline Runs
| Pipeline ID |
Run ID |
Started |
Ended |
Status |
Records Processed |
| pipe-001 |
run-001 |
2026-07-11 03:55 |
2026-07-11 04:55 |
success |
50,000 |