Skip to content

Update to 1.4.8-beta - #25

Merged
Core447 merged 12 commits into
mainfrom
dev
Apr 9, 2024
Merged

Core447 merged 12 commits into
mainfrom
dev

Conversation

@Core447

@Core447 Core447 commented Apr 9, 2024

Copy link
Copy Markdown
Member

No description provided.

@Core447
Core447 merged commit c040f5e into main Apr 9, 2024
nazbert added a commit to nazbert/Deckard that referenced this pull request Jul 12, 2026
…ryable tabs

Three linked failure modes behind a frozen store tab / garbage store URLs:

1. get_official_store_branch returned the NoConnectionError INSTANCE when
   versions.json couldn't be fetched and the cache was too stale;
   get_stores appended it verbatim and build_url interpolated the object's
   repr into request URLs and cache keys. The branch contract is now
   str-only: any failure falls back to STORE_BRANCH, deliberately uncached
   so a later successful fetch still corrects it. get_stores enforces the
   same invariant at its boundary and sanitizes custom-store entries
   (skip url-less ones, default non-str branches to "main").

2. json.loads(versions_file) was unguarded; a truncated cached
   versions.json -- exactly what the stale-cache fallback (782a1dac) can
   serve, see StreamController#25 -- raised JSONDecodeError through the page load. Parse
   is now guarded, including non-dict payloads and non-str mappings.

3. Even with the parse guarded, ANY load() exception died in the page's
   @log.catch with the spinner still up and StorePage._loaded stuck True,
   so the tab never retried until the window was recreated. The load
   thread now runs a guarded wrapper: failures show the error page and
   reset _loaded (show_connection_error re-arms too, and marshals its
   widget change to the main thread), so revisiting the tab retries. The
   four subclass load()s drop @log.catch so failures actually reach the
   wrapper.

Covered by tests/scenario_store_branch_contract.py (network-free).

Closes StreamController#24
nazbert added a commit to nazbert/Deckard that referenced this pull request Jul 12, 2026
…-file lock

StoreCache.open_cache_file stamped entry["fetched"]=now and persisted the
index BEFORE the caller wrote a byte -- then handed back an in-place write
handle. A crash mid-write left a truncated file the index swore was
fresh, and get_remote_file's stale-fallback (782a1dac) served that poison
for up to DAYS_TO_KEEP whenever the live fetch failed. There was also no
per-file lock, so two tabs force-refetching the same key (versions.json)
could interleave writers.

Write modes now return an _AtomicCacheWriter: content goes to a sibling
temp file, fsynced and os.replace()d over the real path on close, and
"fetched" is stamped only after that commit. An exception inside the
caller's `with` block (or an explicit abort) discards the temp file --
the previous content and its honest stamp survive. A per-key lock held
for the writer's open->close window serializes concurrent writers;
readers need no lock since os.replace guarantees old-or-new-complete.
Unsupported write modes ("a"/"x"/"+", which have no callers) now fail
loud instead of silently writing in place.

Also closes the circular legacy clock: get_fetched_date's fallback for
entries predating "fetched" is now the cache file's mtime (true content
age -- reads never touch it) instead of the index "date" that every read
renews, which made legacy entries eternally "fresh" to the fallback.

Covered by tests/scenario_store_cache_atomic.py (network-free): crash
mid-write preserves prior content+stamp and leaks no temp files, fetched
stamps only on close, same-key writers serialize, get_remote_file's
fallback serves the last committed copy after a crashed rewrite, legacy
entries age by mtime.

Closes StreamController#25
nazbert added a commit to nazbert/Deckard that referenced this pull request Aug 9, 2026
…refs StreamController#180)

Every StoreCache read -- including a pure cache hit -- renewed the entry's
last-use clock and rewrote the WHOLE files.json index: json.dump of every
entry, fsync'd, synchronously on the calling thread. A warm store browse of
N catalog files therefore performed N full index dumps (measured: 60 reads
= 60 writes, 22.8 ms) purely to move a timestamp.

Reads and the first sighting of a cache string now mutate the in-memory
entry and mark the index dirty; a single daemon threading.Timer flushes the
whole index once, FLUSH_DEBOUNCE_S (2 s) later -- armed on the first dirty
mark, a no-op while one is pending. Same 60 reads: 0 synchronous writes + 1
flush.

The split is by what losing a write costs, not by convenience:

  * _stamp_committed and remove_old_cache_files stay SYNCHRONOUS. A lost
    "path"/"fetched" record orphans the blob forever, since
    remove_old_cache_files only ever walks index entries -- that is the
    crash-safety result of #73/StreamController#25 and is not deferrable.
  * A lost "date" renewal only makes an entry look <= 2 s older against a
    3-day eviction bound, and the next read renews it again.

A synchronous dump of the live index also clears the dirty flag, so a
pending timer fires as a no-op instead of writing twice.

Also closes a latent hazard the debounce would have widened: entry
assignments ran unlocked while set_files copied the dict under write_lock,
so a concurrent mutation could raise "dictionary changed size during
iteration". Mutation and write now share one write_lock acquisition.

Draining at exit needs both hooks: the module registers an atexit flush for
plain interpreter exits (CLI, harness), and App.on_quit flushes explicitly
because the app ends in os._exit(0), which skips atexit entirely.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant