Progressive path
From visible output to implementation.
Each level adds detail without forcing the reader to restart: enter at the most useful point and continue progressively.
In practice
Six technical concepts translated into mental pictures.
Adapter
An interpreter that transforms different sources into a common form.
State machine
A traffic light distinguishing completed, skipped, blocked or failed work.
Idempotency
Pressing the button twice must not create two identical publications.
Log
A diary recording what happened and when.
Offline test
Testing the system with controlled copies of data without relying on the external source each time.
Secrets and credentials
Passwords and keys must not appear in code or reports.
Operational level
The pipeline is split into components with verifiable responsibilities.
From component to behaviour.
A reliable workflow grows from separated responsibilities, explicit states and repeatable controls. Architecture makes visible where a run may proceed, degrade or stop.
sourcesmethodcontrolsoutputsmemory
Implemented toolchain
Frameworks and tools mapped to a clear role.
Core and data
Tabular processing, object validation and auditable persistence.
Machine learning
Supervised models, anomaly detection, clustering, graphs and bootstrap.
Acquisition
HTTP clients, web and PDF parsing, public feeds and APIs.
Rendering and web
Static cards, navigable charts and an indexable bilingual website.
Automation
Schedules, protected environments, artifacts, tests and idempotent publishing.
Delivery
Standalone social records, static site delivery and language routing.
Run lifecycle
Each execution moves through explicit states.
Trigger
Schedule, event, manual dispatch or a new source.
Acquire
Retries, timeout, source date and metadata.
Validate
Schema, freshness, missing data, ranges and gates.
Compute
Decide
Published, dry run, skipped, blocked or failed.
Persist
Artifacts, state, provenance, hashes and URIs.
Software quality
Different tests cover different failure classes.
Unit tests
Parsers, transformations, captions, record keys, metrics and guards.
Contract tests
Versions, file structure, output schemas and release requirements.
Offline integration
Fixtures and mocks reproduce external sources without network dependency.
Dry runs
Generate cards, payloads and provenance without sending live records.
CI
Ruff, pytest, artifact builds and checks before scheduled execution.
Runbooks
Outcomes, failure modes, retries and state recovery are documented.
Maturity boundary
Implemented components and possible evolution remain distinct.
Current repositories and workflows
GitHub Actions, artifacts, JSON state, pytest, Ruff, static rendering, AT Protocol and Cloudflare Pages.
More structured production
OCI/Docker containers, registries, FastAPI APIs, DVC or MLflow, OpenTelemetry, Prometheus/Grafana and object storage. These are not presented as active components.