Forecasts
Error, bias, coverage, directional accuracy and baseline comparison, only on matured outcomes.
General principle
Limitations of rules, indicators and models
Outputs produced by rules, indicators and models are experimental or informational results, not decisions.
They may be incomplete, unstable or incorrect, especially when data are missing or delayed, exceptional events occur, regimes change, or conditions differ from those observed during design and validation.
Where relevant and available, each result should be interpreted together with its baseline, observation period, sample size, metrics, measure of uncertainty and known failure conditions.
Results and evidence status
What is measured, under evaluation or operational.
The table does not invent a common score: it separates predictive evidence, operational reliability and activities still maturing.
| Workflow | Evidence status | Maturity | Detail |
|---|---|---|---|
| WF-1 · Weather automation | Operational | Active · educational | Open the project page for available metrics, period and limitations. |
| WF-2 · The New Margin | Under evaluation | Active · experimental | Open the project page for available metrics, period and limitations. |
| WF-3 · Market Overview | Under evaluation | Active · experimental | Open the project page for available metrics, period and limitations. |
| WF-4 · AI Supply Chain | Under evaluation | Active · experimental | Open the project page for available metrics, period and limitations. |
| WF-5 · Satellite | Operational | Active · event-conditioned | Open the project page for available metrics, period and limitations. |
| WF-6 · Energy Crisis Thermometer | Under evaluation | Active · experimental | Open the project page for available metrics, period and limitations. |
| WF-7 · Etna Forecast | Measured in workflow | Active · educational | Open the project page for available metrics, period and limitations. |
| WF-8 · Bluesky multi-source | Operational | Active · experimental | Open the project page for available metrics, period and limitations. |
| WF-9 · World Economy Engine | Under evaluation | Active · experimental · evidence-aware | Open the project page for available metrics, period and limitations. |
| WF-10 · Etna Sentinel | Operational | Active · educational · fail-closed | Open the project page for available metrics, period and limitations. |
| WF-11 · Hydrogen Route Observatory | Under evaluation | Experimental · calculation · evidence-aware | Open the project page for available metrics, period and limitations. |
| WF-12 · Ammonia & Fertilizer Chain | Under evaluation | Experimental · calculation + forecast | Open the project page for available metrics, period and limitations. |
| WF-13 · Italy Variable Renewable Forecast | Under evaluation | Experimental · forecast · point-in-time | Open the project page for available metrics, period and limitations. |
| WF-14 · Process Sentinel | Controlled benchmark | Experimental · synthetic data (simulated data) · anomaly detection | Open the project page for available metrics, period and limitations. |
| WF-15 · Virtual Analyzer | Controlled benchmark | Experimental · synthetic data (simulated data) · soft sensor | Open the project page for available metrics, period and limitations. |
| WF-16 · Energy & Steam Optimizer | Controlled benchmark | Experimental · hybrid public + synthetic data (simulated data) · optimisation | Open the project page for available metrics, period and limitations. |
| WF-17 · Etna Fusion Lab | Prospective maturation | Active · experimental · quality-aware | Open the project page for available metrics, period and limitations. |
| WF-18 · Climate Comfort Hours | Operational · heuristic | Active · educational · heuristic | Open the project page for available metrics, period and limitations. |
| WF-19 · Lab Intelligence Community | Operational · editorial | Active · editorial · automated | Open the project page for available metrics, period and limitations. |
| WF-20 · The Public Service Run | Under evaluation · calibration | Active · educational · experimental | Open the project page for available metrics, period and limitations. |
| WF-21 · Port Operability · NYC Ferry | Prospective maturation | Active · research shadow · prospective verification | Open the project page for available metrics, period and limitations. |
Measured in workflow means that the project page publishes specific metrics; under evaluation means relevant metrics are not yet centralised; operational concerns checks, runs and traceability rather than predictive skill.
Available general metrics
Classifications and anomalies
Precision, recall or anomaly frequency when reliable labels exist; otherwise context and human review.
Operating workflows
Successful, failed, skipped and blocked runs, missing data and prevented duplicates.
Transparency rule
Never show only the best metric; state period, sample size and preliminary status.
Workflow status
| Workflow | Status | Cadence | Relevant metrics |
|---|---|---|---|
| WF-1 · Weather automation | Active · educational | Monday, Wednesday and Friday | Run continuity, missing data caught and duplicates prevented. No clinical performance metric is claimed. |
| WF-2 · The New Margin | Active · experimental | Weekly | MAE, median error, bias and interval coverage, always with period and matured sample size. |
| WF-3 · Market Overview | Active · experimental | Weekdays | Directional accuracy, average move, errors, period stability and baseline comparison. |
| WF-4 · AI Supply Chain | Active · experimental | Weekly | Model-versus-naïve comparison, outlook error and basket stability over time. |
| WF-5 · Satellite | Active · event-conditioned | Checked every 6 hours; post on a valid new event | Run outcomes (published, skipped, blocked), source availability and duplicates prevented. No operational dispersion skill is claimed. |
| WF-6 · Energy Crisis Thermometer | Active · experimental | Several weekly runs | Driver coverage, shipping confidence, OOS forecast metrics, Dynamic versus Static, drawdown and period stability. |
| WF-7 · Etna Forecast | Active · educational | Monday, Thursday, Sunday and events | Brier score, comparison with climatology/persistence/Hawkes, calibration, matured sample and gate-selected source. |
| WF-8 · Bluesky multi-source | Active · experimental | Monday, Wednesday and Friday | Source-gate audit, publication receipts, grapheme count, multi-image output and PDS/AppView verification. |
| WF-9 · World Economy Engine | Active · experimental · evidence-aware | Weekly · Monday | Expanding backtest with purge gap, prequential selection, naive benchmark, bootstrap skill, 90% conformal intervals, graph ablation and anti-leakage/vintage audit. |
| WF-10 · Etna Sentinel | Active · educational · fail-closed | Tue/Thu/Sun + VONA monitor every 3 hours | Unit/end-to-end tests, data-quality and disclosure gates, deduplication/idempotency, fail-closed on uncertain/positive official status and immutable history. |
| WF-11 · Hydrogen Route Observatory | Experimental · calculation · evidence-aware | Mon–Fri 10:50 Europe/Rome | Physical/dimensional invariants, sourcing coherence, lineage, snapshot replay. |
| WF-12 · Ammonia & Fertilizer Chain | Experimental · calculation + forecast | Mon–Fri 09:50 Europe/Rome; monthly maturation | Walk-forward, persistence, seasonality, gas-only and physical baselines; MAE/RMSE/bias/coverage/skill. |
| WF-13 · Italy Variable Renewable Forecast | Experimental · forecast · point-in-time | Daily 08:50 Europe/Rome | Time split and persistence; proxy separated from actuals and never matured as a real forecast; prospective ENTSO-E A69 baseline when available; nMAE/RMSE/bias/coverage/skill. |
| WF-14 · Process Sentinel | Experimental · synthetic data (simulated data) · anomaly detection | Every 6 h: 00:50/06:50/12:50/18:50 UTC | Parameters frozen on a separate development simulation; same frozen 28-day synthetic data (simulated data) benchmark; event recall, missed rate, false-alarm episodes/24h, P50/P90 delay, v1.4.16 ensemble comparison and anti-leakage. |
| WF-15 · Virtual Analyzer | Experimental · synthetic data (simulated data) · soft sensor | Every 6 h: 01:50/07:50/13:50/19:50 UTC | RMSE/MAE/bias/coverage, skill versus last lab, split-conformal and truth-vs-lab; risk-coverage diagnostic only. |
| WF-16 · Energy & Steam Optimizer | Experimental · hybrid public + synthetic data (simulated data) · optimisation | Daily 11:50 Europe/Rome | Feasibility, balances, optimality, solve time, stability and reference policy. |
| WF-17 · Etna Fusion Lab | Active · experimental · quality-aware | Hourly nowcast · social Tue/Thu/Sat | Immutable issue ledger, reconciliation sidecars, prospective calibration, hysteresis/cooldown and adaptive learner kept in shadow before gates. |
| WF-18 · Climate Comfort Hours | Active · educational · heuristic | Tue/Thu 17:00 · Sun 21:00 Europe/Rome | Independent city/day validation, batch→recent cache→single retry, grey cells for missing data, publication only with 4 core cities and at least 80% of the basket. |
| WF-19 · Lab Intelligence Community | Active · editorial · automated | Mon/Wed/Fri/Sun fixed · Tue/Sat FLEX | slot and blackout validation, weekly cap, deduplication, reviewable diff and fail-closed gates for schedule, policy and credentials |
| WF-20 · The Public Service Run | Active · educational · experimental | Daily forecast 05:17 · social Tue IT / Fri EN | MAE, RMSE, MAPE, median/p90 absolute error, P10–P90 coverage, top-1 departure accuracy, departure regret and explicit warm-up stage |
| WF-21 · Port Operability · NYC Ferry | Active · research shadow · prospective verification | Daily run + 30-min event watcher · social Fri IT / Mon EN | Immutable forecasts verified at D+1 and D+7; prospective metrics include episode recall, lead time, alert burden on verified available days, horizon coverage and restricted calibration; UNKNOWN stays outside denominators. |
Status describes project maturity and does not guarantee continuous availability of external sources.
How to use this evidence
Different metrics answer different questions.
Delivery capability
Inspect testing, modularity, logging, schedules, error handling and communication of limitations.
Professional evidence →Empirical validity
Check baselines, temporal separation, sample size, calibration and negative results.
Academic checklist →Operating reliability
Assess freshness, skipped runs, blocking gates, idempotency and output traceability.
Applications →Data not yet centralised: the site does not invent an overall run-success rate. It will be shown only when a shared automated register supplies it.