Split last_upsert (persisted write) and last_refresh (run liveness) in stats
This commit is contained in:
@@ -280,13 +280,22 @@ Use `reset()` after a **structural change** in the source (columns added/removed
|
||||
stats = engine.stats # Stats snapshot
|
||||
print(stats.hits, stats.misses, stats.refetches, stats.errors)
|
||||
for name, t in stats.tables.items():
|
||||
print(name, t.rows, t.state, t.tracking, t.last_refresh)
|
||||
print(name, t.rows, t.state, t.tracking, t.last_upsert, t.last_refresh)
|
||||
if t.consecutive_failures:
|
||||
print(f" {name} failing ×{t.consecutive_failures}: {t.last_error} ({t.last_error_at})")
|
||||
```
|
||||
|
||||
`Stats.errors` is the total number of load/refresh failures since start. Each `TableStats` also carries `last_error`, `last_error_at` and `consecutive_failures` (reset to 0 on the next success) — so a delta that fails *before* streaming (which otherwise leaves `state` looking `ready`) is still visible, and the table is marked `error`.
|
||||
|
||||
Two timestamps distinguish *data freshness* from *liveness*:
|
||||
|
||||
| field | meaning |
|
||||
|---|---|
|
||||
| `last_upsert` | wall-clock (UTC) of the last actual **data write** — full load or a delta cycle that wrote rows. Persisted, survives restarts. Answers *"when did the data last change?"* |
|
||||
| `last_refresh` | wall-clock (UTC) of the last time a **refresh cycle ran** for the table — bumped **even when it wrote nothing**. In-memory per process (`None` until the first cycle runs after start). Answers *"is the refresh loop alive?"* |
|
||||
|
||||
A delta table that runs every cycle but finds no new rows keeps `last_refresh` ticking while `last_upsert` stays put — that's healthy, not stuck. (Both are UTC ISO strings; the default log timestamps are local time, so expect an offset.)
|
||||
|
||||
Each `TableStats` reports a live processing **state** and how the table is kept fresh (**tracking**):
|
||||
|
||||
| `state` | Meaning |
|
||||
|
||||
Reference in New Issue
Block a user