Repository navigation
feat!: migrations move under il migration; Python script becomes inputlayer-migrate - #101
Merged
Merged
Conversation
This was referenced Aug 19, 2026
jsam
force-pushed
the
py/il-migrate-rename
branch
from
August 19, 2026 16:18
6288d06 to
c2cb59a
Compare
jsam
force-pushed
the
py/il-migrate-rename
branch
from
August 19, 2026 16:18
c2cb59a to
1545250
Compare
…mes inputlayer-migrate il is the single product CLI and migrations are a noun group under it: il migration generate / apply / revert / status. The delegation layer in the il binary translates these to the Python tool's subcommands, so Django-flavored names (makemigrations) never surface in the product CLI and 'il migrate migrate' never exists. The SDK's console script is renamed from 'il' to 'inputlayer-migrate' (matching its existing argparse prog) so the name 'il' belongs to the product CLI alone. Docs updated to the new vocabulary. Breaking for scripts invoking the old Python 'il' directly: use 'il migration <verb>' (or 'inputlayer-migrate <subcommand>' where only the SDK is installed).
jsam
force-pushed
the
py/il-migrate-rename
branch
from
August 19, 2026 17:13
1545250 to
55cdf29
Compare
Found by running the full lifecycle (generate/apply/status/revert)
end to end via il migration against a live server. Two bugs, both
invisible to the mocked unit tests:
1. The recorder relation was named __inputlayer_migrations__. The
engine rejects that name in schema declarations (must start with a
lowercase letter) but silently auto-creates it untyped on insert,
and queries against it fail - so applied-state was written into a
relation that could never be read. Renamed to inputlayer_migrations.
(Engine-side inconsistency filed separately: inserts bypass the
relation-name validation that schema declarations enforce.)
2. get_applied() used '?Name, At <- rel(Name, At)', which the engine
rejects ('Query cannot contain a rule definition'). Every status
read returned [], so showmigrations always showed unapplied, revert
never found anything, and apply re-applied migrations every run.
Switched to the plain query form and deduplicated.
Also: --migrations-dir is now accepted after the subcommand on every
subcommand (argparse.SUPPRESS so the global default survives), since
delegating callers always place flags after the subcommand.
Verified live: generate -> apply -> status [X] -> re-apply no-op ->
second migration -> revert to 0001 -> status shows [X]/[ ]. Parser
position tests added; full SDK suite 960 passed.
Migration artifacts were Python source (operations rendered as code,
loaded via importlib) - unusable by any other SDK language, and even
listing history required a Python runtime. Migrations are now data:
- writer emits NNNN_name.json: {format, dependencies, operations, state}
using the to_dict/from_dict serialization the operation classes
already had. No code, no imports, nothing language-specific.
- loader reads both formats: JSON (current) and legacy .py (back-compat,
read-only). JSON state normalizes column pairs back to tuples so the
autodetector sees no phantom changes.
- test_writer rewritten for the JSON document (structure, operation
roundtrip incl. forward/backward command equality, no-code guarantee);
legacy .py loading stays covered in test_loader. Suite: 956 passed.
- migrations guide: anatomy section now shows the JSON document.
This is step 1-2 of #104: the on-disk format is now consumable by a
future JS frontend and by il natively (apply/revert/status without any
SDK installed). Verified live: generate -> apply -> status -> revert
against a running engine with JSON files.
…y, loader validation, stale docs Adversarial review of this PR found (all reproduced by execution): 1. BLOCKER: a directory containing both 0001_x.py and 0001_x.json (a stale legacy twin of a regenerated migration) loaded twice and APPLIED twice in one run - destructive for Drop/Replace operations. load_migrations now rejects duplicate names and duplicate numbers with a clear error. 2. Engine errors were invisible: the WS protocol reports failures as an 'error'-column result, and nothing in the migration stack checked - a failed op mid-migration was recorded as applied with exit 0, and get_applied() could return the error text as a migration name. New errors.py check_engine_result is applied to every recorder and executor statement; a failed migration is never recorded. 3. The JSON loader had zero validation: malformed JSON, missing op fields, or unknown op types produced raw tracebacks without filenames, and the format field was write-only. All now raise MigrationError naming the file; format > supported rejects with an upgrade hint. The CLI catches MigrationError cleanly. 4. Migration names were unvalidated: --name a/b wrote into a nonexistent subdirectory and quotes broke the recorder's IQL. Writer enforces [a-z0-9_]+; recorder escapes interpolated names. 5. Stale docs taught the removed 'il' script: the PyPI-shipping SDK README, python-sdk.mdx, and the generated GUI docs bundle (regenerated). migrations.mdx output examples now match actual CLI output. New tests/test_migration_safety.py covers all of it (13 tests: dual format collisions, the legit mixed directory, malformed JSON, format version, error-frame propagation, no-record-on-failure, name safety). Full SDK suite: 969 passed.
This was referenced Aug 20, 2026
Pre-1.0 decision: no legacy formats. The importlib-based .py loader, the Migration base class it existed for (and its package exports), and the dual-format handling are gone. Stray .py files from older SDKs are inert - ignored by discovery, never imported, never double-applied. Duplicate-number detection remains for renumber collisions. test_loader rewritten around JSON; safety tests updated (stray .py twin is ignored); docs and GUI bundle updated. Suite: 969 passed.
Found by live demo verification: an unreachable server printed a raw InputLayerConnectionError traceback. The CLI now catches InputLayerError alongside MigrationError and prints a one-line error with exit 1. Test added; suite 970 passed.
jsam
force-pushed
the
py/il-migrate-rename
branch
from
August 20, 2026 10:20
5416fd3 to
862fe72
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Three related changes that make migrations part of the single il CLI and language-independent:
1. Vocabulary: il migration
il owns the vocabulary; the delegation layer (#98, with integration tests) translates. Console script renamed il -> inputlayer-migrate. --migrations-dir accepted after the subcommand on every subcommand.
2. Language-neutral migration files (step 1-2 of inputlayer/inputlayer-clients#1)
Migration artifacts were Python source loaded via importlib - unusable by any other language. They are now JSON documents: {format, dependencies, operations (typed, with enough structure to derive apply AND revert IQL), state}. The writer emits .json; the loader reads .json plus legacy .py (read-only back-compat). test_writer rewritten around the JSON document incl. a no-code guarantee and operation roundtrip; JSON state normalizes column pairs to tuples so the autodetector sees no phantom changes. This unblocks the JS frontend and native il apply (#104 steps 3-5).
3. Recorder bugs: migrations never worked against a real engine
Found by live end-to-end verification, invisible to mocked unit tests:
Verified
Known limits
Merge order
With or right after #98. No conflicts with other open PRs.