Migrate from Poetry and pyenv to uv

2026-10-12 15:48:26 +02:00
parent c399a6aed3
commit c69f5f9166
93 changed files with 13068 additions and 2401 deletions
@@ -0,0 +1,251 @@
+# `src/core/csfd.py`
+
+ČSFD.cz scraper — a *title bridge* to TMDb.
+
+Curator identifies movies by **TMDb id**. A direct TMDb search on a Czech
+distribution title is unreliable (remakes, localized names, "weird" hits).
+ČSFD, on the other hand, indexes Czech release names well **and** lists a
+film's *original* title. So the bridge is::
+
+    filename → ČSFD search → ČSFD film page → original title + year
+             → TMDb search → tmdbid
+
+This module only scrapes ČSFD; the TMDb half lives in
+`src.core.file_manager` (`FileManager.resolve_via_csfd`). Nothing
+here feeds tags/genres — those still come from TMDb via Jellyfin/NFO.
+
+ČSFD serves an **Anubis** proof-of-work wall in front of every page: a plain
+request gets a 200 with a JS challenge instead of the movie. We detect it,
+solve the SHA-256 PoW the way the bundled worker JS does, and replay the
+request through a shared `requests.Session` (so a batch pays the PoW
+cost only once).
+
+Source: [`src/core/csfd.py`](../../src/core/csfd.py)
+
+## Classes
+
+### `CsfdError`
+
+*class · line 62*
+
+```python
+class CsfdError(RuntimeError):
+```
+
+A ČSFD lookup could not be completed (deps missing, network, anti-bot).
+
+### `CsfdFilm`
+
+*class · line 67*
+
+```python
+class CsfdFilm:
+```
+
+The slice of a ČSFD film page Curator needs to reach a TMDb id.
+
+#### `CsfdFilm.tmdb_query`
+
+*property · line 77*
+
+```python
+tmdb_query -> tuple[str, Optional[str]]
+```
+
+`(title, year)` best suited for a TMDb search — original title first.
+
+## Functions
+
+### `dependencies_available`
+
+*function · line 82*
+
+```python
+dependencies_available() -> bool
+```
+
+True when `requests` + `beautifulsoup4` are importable.
+
+### `_check_dependencies`
+
+*function · line 87*
+
+```python
+_check_dependencies() -> None
+```
+
+Raise `CsfdError` when the optional `requests` / `beautifulsoup4` packages are missing (they are needed only for ČSFD scraping).
+
+### `_extract_csfd_id`
+
+*function · line 95*
+
+```python
+_extract_csfd_id(url: str) -> Optional[int]
+```
+
+Pull the numeric film id out of a ČSFD URL, or `None`.
+
+### `_extract_json_blob`
+
+*function · line 105*
+
+```python
+_extract_json_blob(html: str, element_id: str)
+```
+
+Return the parsed JSON from an Anubis `<script id=...>` blob, or None.
+
+### `_solve_anubis_pow`
+
+*function · line 120*
+
+```python
+_solve_anubis_pow(random_data: str, difficulty: int) -> tuple[str, int, int]
+```
+
+Brute-force the Anubis proof-of-work.
+
+Mirrors the bundled `sha256-purejs` worker: find the smallest `nonce`
+such that `sha256(random_data + str(nonce))` has `difficulty` leading
+zero nibbles. Returns `(hash_hex, nonce, elapsed_ms)`.
+
+### `_solve_anubis_challenge`
+
+*function · line 143*
+
+```python
+_solve_anubis_challenge(session, html: str, url: str)
+```
+
+Solve the Anubis challenge in `html` and return the real page response.
+
+### `_get_page`
+
+*function · line 180*
+
+```python
+_get_page(session, url: str)
+```
+
+GET `url` through `session`, transparently clearing an Anubis wall.
+
+### `_parse_json_ld`
+
+*function · line 193*
+
+```python
+_parse_json_ld(soup) -> tuple[str, Optional[str]]
+```
+
+`(name, year)` from the film page's JSON-LD (name is the Czech title).
+
+### `_parse_origin`
+
+*function · line 215*
+
+```python
+_parse_origin(soup) -> tuple[Optional[str], Optional[str]]
+```
+
+`(origin_country, year)` from the `.origin` line.
+
+ČSFD separates values with inline bullet `<span>` elements (no commas),
+e.g. `USA • 1999 • 136 min`. For a co-production only the first country
+is returned — enough to match it against a name's flag.
+
+### `_parse_original_title`
+
+*function · line 245*
+
+```python
+_parse_original_title(soup, origin_country: Optional[str]) -> Optional[str]
+```
+
+The film's original title from `.film-names` — or `None` to use the CZ title.
+
+Each `<li>` carries a country/language flag (`img.flag` `title`) and a
+title. The original title is the entry whose flag matches the **country of
+origin** (USA → "The Shawshank Redemption"). ČSFD does **not** repeat the
+Czech title here when it equals the `<h1>`, so a Czech film has only
+foreign/festival alt-titles in this list (`[angličtina] Cosy Dens`,
+`[Slovinsko] Samotarji`) — picking one of those would be wrong, so we
+return `None` and let the caller fall back to the Czech `<h1>` title.
+
+### `fetch_film`
+
+*function · line 276*
+
+```python
+fetch_film(url: str, session=None) -> CsfdFilm
+```
+
+Fetch a ČSFD film page → `CsfdFilm` (Czech + original title, year).
+
+Pass a shared `requests.Session` to reuse the Anubis auth cookie across a
+batch. Raises `CsfdError` on a network / anti-bot / parse failure.
+
+### `_films_section`
+
+*function · line 311*
+
+```python
+_films_section(soup)
+```
+
+The `<section>` holding the *Filmy* results, or the whole document.
+
+ČSFD's search page groups hits into `Filmy` / `Seriály` / `Tvůrci` /
+`Uživatelé` sections; `a.film-title-name` matches in all of them, so
+without scoping a bare-title search happily returns a series episode.
+
+### `_is_film_href`
+
+*function · line 325*
+
+```python
+_is_film_href(href: str) -> bool
+```
+
+True for a film URL `/film/<id>-slug/` — a series episode has two segments
+(`/film/<series>/<episode>/`) and is rejected.
+
+### `search_films`
+
+*function · line 332*
+
+```python
+search_films(query: str, limit: int=5, session=None) -> list[CsfdFilm]
+```
+
+Search ČSFD for `query`; film hits only (no series/episodes), best first.
+
+### `find_csfd_url`
+
+*function · line 365*
+
+```python
+find_csfd_url(query: str, session=None) -> Optional[str]
+```
+
+Return the first ČSFD film URL matching `query`, or `None`.
+
+ČSFD's full-text search treats a trailing year as a search term (not a
+filter) and can miss on `"Title 2006"` while `"Title"` hits — so a
+yearful query with no result is retried without the year.
+
+### `clean_filename_to_query`
+
+*function · line 397*
+
+```python
+clean_filename_to_query(filename: str) -> str
+```
+
+Turn a (possibly release-named) filename into a ČSFD search query.
+
+Strips the path/extension, splits on common separators and keeps the words
+before the first release marker (year, resolution, codec, source, …). The
+detected year is appended back as a disambiguator::
+
+    "Matrix.1999.1080p.BluRay.x264-GROUP.mkv" -> "Matrix 1999"