In a session hosted by The Top Voices, the reasons were examined why early data workflows — often built on scattered SQL scripts and cron jobs — become brittle and risky as companies grow. Drawing on hands-on experience at fintech unicorn Tabby, a roadmap was outlined for evolving from ad-hoc automations to scalable SQL systems, with clarity, observability, and reliability at their core.
Speaker
Ivan Timonov is an MLOps and platform engineer at Tabby, where he works on infrastructure, automation, and systems design. His work focuses on helping teams move away from short-term fixes toward sustainable, cost-efficient platforms.
From Quick Scripts to Fragile Systems
Most startups begin with “just a few SQL jobs.” They live in the data warehouse UI, run on schedules defined in dropdown menus, or are wrapped in simple cron scripts. At first, this feels acceptable — until usage scales, other teams start depending on the results, and late-night failures begin to pile up. What was once “a few queries” turns into a fragile, production-critical system with no clear contracts or visibility.
A Maturity Model for SQL Automation
To assess and evolve such systems, a three-level maturity model can be used:
- Level 1 – UI-Driven Jobs
Jobs live in console tabs, configured manually. There’s no source of truth, no history, and limited reproducibility.
- Level 2 – Script-Based Automation
Bash, Python, and cron enter the picture. But environments are inconsistent, ownership is unclear, and debugging is painful.
- Level 3 – System-Driven Automation
Jobs become structured objects, with declarative specs, defined inputs/outputs, validation, and built-in observability. This is the target state.
Six Principles of Scalable SQL Systems
The transition to reliable SQL automation is guided by the following principles:
- Single Source of Truth
Each job should have a clear, versioned spec describing purpose, schedule, inputs, outputs, owner, and limits.
- Explicit Contracts
Jobs should declare their behavior and guarantees — what they do, what they depend on, and how they handle failures or re-runs.
- Standardization Over Snowflakes
Use consistent naming, scheduling, and metadata conventions to reduce overhead and reasoning cost.
- Validation Before Execution
Jobs must be checked — not just executed — to catch issues early and enforce policies like cost or resource limits.
- Observability by Default
Metrics, logs, and run histories should be available and accessible to both platform and data stakeholders.
- Idempotency and Portability
Rerunning a job should never corrupt data. Jobs should also migrate cleanly across environments.
A Lightweight Architecture That Scales
Instead of relying on a heavy orchestrator, the system can be broken down into five minimal components:
- Job Definition Store (e.g. Git, YAML): the spec lives here.
- Policy & Validation Layer: enforces consistency and safety.
- Scheduling Layer: separates time logic from job logic.
- Execution Layer: translates specs into warehouse queries.
- Observability Layer: logs, metrics, and dashboards.
Each layer can be implemented incrementally using lightweight tools, starting with as little as one sprint of work.
How to Start — Even with a Small Team
A practical starting point might include:
- Create a simple job registry with clear owners and schedules.
- Require metadata like domain or cost limits for new jobs.
- Enable basic run logging to track success/failure.
From there, the system can evolve gradually by adding field validation, enforcing naming conventions, and introducing dashboards or alerts. Existing jobs can be migrated to the new model only when they are touched—no full rewrite is required.
Conclusion
What starts as “just a few queries” often grows into invisible infrastructure. Without a system-level approach, hidden costs accumulate in the form of incidents, slowdowns, and unclear ownership. Regaining control does not require heavy orchestration. By following a few core principles — declarative specifications, validation, and observability — teams can build SQL automation systems that are transparent, scalable, and ready for growth.