# SQLmem

Transparent in-memory cache layer between SQLAlchemy and your database. Drop it in front of any SQLAlchemy engine — SELECT queries are served from a fast in-memory SQLite cache, writes pass through unchanged.

## How it works

```
Application (SQLAlchemy)
        │
        ▼
  [ SQLmem Proxy ]
  ┌──────────────────────────────┐
  │  SQL Parser                  │  → detects SELECT vs. write
  │  Column Registry             │  → tracks which columns are cached per table
  │  Cache Manager (SQLite RAM)  │  → stores data in memory
  │  Query Executor              │  → cache hit / miss logic
  └──────────────────────────────┘
        │
        ▼
  Database (via original SQLAlchemy engine)
```

On the first SELECT for a table, SQLmem fetches the required rows from the database and stores them in an in-memory SQLite instance. Subsequent queries for the same columns hit the in-memory cache with no database round-trip. When a query requests a column not yet in cache, SQLmem re-fetches the table with the expanded column set.

## Installation

```bash
pip install sqlmem
# or with Poetry
poetry add sqlmem
```

Requires Python 3.14.

## Quick start

```python
from sqlmem import CachingEngine
from sqlalchemy import create_engine, text

base_engine = create_engine("postgresql://user:pass@host/db")
engine = CachingEngine(base_engine)

# Use exactly like a regular SQLAlchemy engine:
results = engine.execute("SELECT id, name FROM users WHERE status = 'active'")
for row in results:
    print(row["id"], row["name"])
```

`execute()` returns a list of dicts. Results are compatible with standard iteration patterns.

## Cache behaviour

**Column accumulation** — SQLmem learns which columns your app needs at runtime, no upfront configuration required:

```
Query 1: SELECT a, b FROM orders   → cache miss → fetch orders(a, b) from DB
Query 2: SELECT a, d FROM orders   → new column d → re-fetch orders(a, b, d)
Query 3: SELECT b FROM orders      → cache hit, no DB query
Query 4: SELECT * FROM orders      → UnsupportedQueryError (wildcard not supported)
Query 5: SELECT a FROM orders JOIN … → UnsupportedQueryError (JOIN not supported)
```

**Writes are blocked** — INSERT, UPDATE, and DELETE raise `ReadOnlyError`. SQLmem is a read-only cache.

## Persistence

The in-memory cache is optionally persisted to `cache.db` on disk:

- **On startup**: if `cache.db` exists, it is loaded into memory.
- **Hourly**: a background thread writes a snapshot to disk.
- **On shutdown**: a final flush via `atexit` and SIGTERM handler.

Schema version is checked on load — if it does not match, the stale file is discarded and the cache is rebuilt from the database.

## Manual cache invalidation

```python
engine.invalidate("orders")   # drops the table from cache; next query re-fetches from DB
engine.close()                # flush to disk and shut down background thread
```

## Configuration

Set via environment variables or a `.env` file:

| Variable | Default | Description |
|---|---|---|
| `SQLMEM_DEBUG` | `false` | `true` enables DEBUG-level logging |
| `SQLMEM_CACHE_DB` | `cache.db` | Path to the on-disk persistence file |
| `SQLMEM_BACKUP_INTERVAL` | `3600` | Backup interval in seconds |

## Exceptions

| Exception | When raised |
|---|---|
| `ReadOnlyError` | INSERT, UPDATE, or DELETE statement |
| `UnsupportedQueryError` | `SELECT *` or any JOIN |

```python
from sqlmem import ReadOnlyError, UnsupportedQueryError
```

## Logging

SQLmem is silent by default. Call `add_sink()` to opt in:

```python
import sys
from sqlmem import add_sink

add_sink(sys.stderr)                      # INFO by default
add_sink(sys.stderr, level="DEBUG")       # verbose: every query, cache hit/miss, backup
add_sink("sqlmem.log", rotation="10 MB") # to a file
```

Set `SQLMEM_DEBUG=true` in `.env` to make the default level DEBUG when no explicit `level` is passed to `add_sink()`.

## Limitations

- `SELECT *` and JOIN queries are not supported.
- No distributed cache backend (Redis etc.).
- No transactional consistency guarantees.
- Write operations (INSERT/UPDATE/DELETE) are always blocked.

## Dependencies

| Layer | Library |
|---|---|
| SQL parsing | `sqlglot` |
| Cache storage | `sqlite3` (stdlib) |
| Integration | SQLAlchemy 2.x |
| Logging | `loguru`, `python-dotenv` |

## License

MIT