Add support for query parameters, JOINs, SELECT * and three-part table names

This commit is contained in:
Jan Doubravský
2026-06-04 18:25:47 +02:00
parent b044ca43f8
commit 530c2618cf
14 changed files with 511 additions and 106 deletions
+27 -5
View File
@@ -22,6 +22,8 @@ Application (SQLAlchemy)
On the first SELECT for a table, SQLmem fetches the required rows from the database and stores them in an in-memory SQLite instance. Subsequent queries for the same columns hit the in-memory cache with no database round-trip. When a query requests a column not yet in cache, SQLmem re-fetches the table with the expanded column set.
Parametrized queries, JOINs and `SELECT *` are all supported. Each table referenced in a JOIN is cached independently; the JOIN itself runs in the in-memory SQLite. Query parameters are applied during in-memory filtering, so cache loads always fetch the full table regardless of the `WHERE` values.
## Installation
```bash
@@ -45,9 +47,25 @@ engine = CachingEngine(base_engine)
results = engine.execute("SELECT id, name FROM users WHERE status = 'active'")
for row in results:
print(row["id"], row["name"])
# Positional parameters (?):
engine.execute("SELECT id, name FROM users WHERE id = ?", ("42",))
# Named parameters (:name):
engine.execute("SELECT id, name FROM users WHERE id = :id", {"id": "42"})
# JOINs — each table is cached independently:
engine.execute(
"SELECT u.name, o.total FROM users u "
"JOIN orders o ON o.user_id = u.id WHERE u.id = ?",
("42",),
)
# SELECT * — loads and caches the whole table:
engine.execute("SELECT * FROM users")
```
`execute()` returns a list of dicts. Results are compatible with standard iteration patterns.
`execute()` returns a list of dicts. Parameters are passed straight through to SQLite, so positional (`?`) and named (`:name`) styles both work. Results are compatible with standard iteration patterns.
## Cache behaviour
@@ -57,10 +75,12 @@ for row in results:
Query 1: SELECT a, b FROM orders → cache miss → fetch orders(a, b) from DB
Query 2: SELECT a, d FROM orders → new column d → re-fetch orders(a, b, d)
Query 3: SELECT b FROM orders → cache hit, no DB query
Query 4: SELECT * FROM orders → UnsupportedQueryError (wildcard not supported)
Query 5: SELECT a FROM orders JOIN … → UnsupportedQueryError (JOIN not supported)
Query 4: SELECT * FROM orders → fetches all columns, marks the table fully cached
Query 5: SELECT a FROM orders → cache hit (table already full)
```
**`SELECT *`** loads every column and marks the table as fully cached, so any later column query is a guaranteed cache hit with no re-fetch.
**Writes are blocked** — INSERT, UPDATE, and DELETE raise `ReadOnlyError`. SQLmem is a read-only cache.
## Persistence
@@ -89,13 +109,14 @@ Set via environment variables or a `.env` file:
| `SQLMEM_DEBUG` | `false` | `true` enables DEBUG-level logging |
| `SQLMEM_CACHE_DB` | `cache.db` | Path to the on-disk persistence file |
| `SQLMEM_BACKUP_INTERVAL` | `3600` | Backup interval in seconds |
| `SQLMEM_SQL_DIALECT` | `tsql` | sqlglot dialect used to parse incoming SQL (e.g. `tsql`, `postgres`, `mysql`) |
## Exceptions
| Exception | When raised |
|---|---|
| `ReadOnlyError` | INSERT, UPDATE, or DELETE statement |
| `UnsupportedQueryError` | `SELECT *` or any JOIN |
| `UnsupportedQueryError` | non-SELECT statement, `SELECT` without `FROM`, or an unqualified column in a multi-table query |
```python
from sqlmem import ReadOnlyError, UnsupportedQueryError
@@ -118,7 +139,8 @@ Set `SQLMEM_DEBUG=true` in `.env` to make the default level DEBUG when no explic
## Limitations
- `SELECT *` and JOIN queries are not supported.
- In a multi-table (JOIN) query, every column must be qualified with its table or alias; unqualified columns raise `UnsupportedQueryError`.
- Tables are keyed by their base name — two tables with the same name in different schemas share one cache entry.
- No distributed cache backend (Redis etc.).
- No transactional consistency guarantees.
- Write operations (INSERT/UPDATE/DELETE) are always blocked.