Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ covering only what changed for that package:
[PostgreSQL](src/EFCore.ComplexIndexes.PostgreSQL/CHANGELOG.md),
[SQL Server](src/EFCore.ComplexIndexes.SqlServer/CHANGELOG.md).

## 5.4.0

Fixes to what 5.x already ships, found while testing the packages against EF Core 11 and planning
the next features: indexes on JSON members that no query could use, and constraint names that only
failed — or quietly vanished — when the migration was applied.

- **Fixed:** indexes on members of a `ToJson()` complex property are written the way Npgsql's queries read the member, so PostgreSQL can use them. It uses an expression index only for a query whose expression matches, and until now every member was extracted as `"doc" -> 'A' ->> 'B'` text with no cast, which matched only a top-level string. A nested member (`#>> '{Address,City}'` in the query) or a typed one (`CAST("doc" ->> 'Rank' AS integer)`) got an index that applied cleanly, enforced uniqueness, and was never used by a single query. Members now render as Npgsql renders them — `->>` or `#>>`, cast to the member's store type unless it is a string, `decode(…, 'base64')` for `byte[]`, `jsonb` for a primitive collection or a `json`/`jsonb` member — in index parts, typed expression indexes and index filters. That also makes a filter such as `x => x.Profile.Rank > 5` valid SQL: it compared text with an integer and failed at apply time. Default index names do not change. **Upgrading:** the first `migrations add` drops and re-creates each affected index under its existing name; indexes on top-level string members are untouched. Until that migration exists, `has-pending-model-changes` reports changes and `Migrate()` raises EF Core's pending-model-changes error. The re-create blocks writes while it builds, so on a large table declare the index with `IsCreatedConcurrently()` first. Exclusion constraint filters keep their rendering. The snapshot gains one `CustomIndex:RenderingVersion` annotation, which records the rules a model was declared under and is what lets the change reach databases built by earlier versions.
- **Changed:** a non-unique index that starts with a date or time JSON member (`DateTime`, `DateTimeOffset`, `DateOnly`, `TimeOnly`) is rejected at `migrations add`. EF Core's queries cast such a member to `timestamptz`, `date` or `time`, and PostgreSQL cannot index those casts (the conversion from text is not IMMUTABLE), so no query could ever use the index. A unique one is still allowed and enforces uniqueness on the stored text; a member in a later position leaves the index usable through the parts before it.
- **Fixed:** on PostgreSQL, a name this package introduces is checked against every object it shares a namespace with, at `migrations add`. PostgreSQL keeps constraint names unique per table and index names unique per schema — including the index behind every primary key, unique, exclusion and temporal constraint — but each kind was checked against its own kind on one table at most. Two temporal constraints with one name, a temporal foreign key named like a foreign key on its table, or a temporal constraint named like an index anywhere in the schema scaffolded cleanly and failed when applied (42710, 42P07). Against an exclusion constraint it did not even fail: every exclusion constraint is added after `DROP CONSTRAINT IF EXISTS`, so a same-named temporal constraint was dropped and the migration applied clean, one declared guarantee short. Two explicitly named complex indexes on different tables of one schema are caught the same way. A collision between two of EF Core's own objects is left to EF; a snapshot holding a collision stays diffable.

## 5.3.0

One fix, found the first time a 5.2.0 converter-member path met the model snapshot
Expand Down
42 changes: 37 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,19 @@ revoked rows" collapsed to one constraint. Name collisions matter more here than
every ADD is preceded by `DROP CONSTRAINT IF EXISTS`, so a duplicate name does not fail at apply
time — the second constraint silently replaces the first.

The per-kind checks above each compare a kind with itself on one table, which is not how
PostgreSQL scopes names, so since 5.4.0 the Npgsql differ's `ValidateNamesAcrossKinds` runs last
over everything the target model introduces — complex indexes, exclusion, temporal and temporal
foreign key constraints — plus EF's own indexes, keys, foreign keys and check constraints. Two
namespaces: constraint names per **table** (42710; against an exclusion constraint, a silent
replacement instead, confirmed on PostgreSQL 18: the temporal `UNIQUE` vanished and the migration
applied clean), and index names per **schema**, where every primary key, unique, exclusion and
temporal constraint also owns an index (42P07). A collision is reported only when one party is
this package's; two of EF's own objects are EF's business. It collects into a list, not a set: the
two same-named declarations it exists to find produce identical entries, and a set merged them.
Core's `ValidateUniqueIndexNames` stays per table on purpose — SQL Server scopes index names per
table.

### The read model is the differ's reader

`ComplexIndexModelExtensions.GetDeclaredComplexIndexes` (core) and
Expand Down Expand Up @@ -360,13 +373,32 @@ differ let satellites resolve what the core cannot:

- `IsForwardedIndexAnnotation` — the annotation whitelist (see below).
- `ResolveUnmappedPart` — a path with no table column; the Npgsql differ builds a JSON extraction
(`"col" -> 'A' ->> 'B'`) when the path traverses a `ToJson()` complex property, honoring
`HasJsonPropertyName`. Members extract as text — no automatic casts (text→timestamptz casts are
not IMMUTABLE and would blow up `CREATE INDEX`). A path that *ends* at the JSON-mapped complex
when the path traverses a `ToJson()` complex property, honoring `HasJsonPropertyName`, rendered
**exactly as Npgsql's query translation renders the member** (`NpgsqlQuerySqlGenerator.VisitJsonScalar`
and `GenerateJsonPath`, identical in Npgsql 10 and 11): `->>` for one step, `#>> '{A,B}'` for
more (`ARRAY[…]::text[]` when a segment is not ASCII-alphanumeric), and a `CAST` to the member's
store type unless it is a string mapping, `decode(…, 'base64')` for `bytea`, `jsonb` for a
primitive collection or a `json`/`jsonb` scalar (checked in that order, as Npgsql does). PostgreSQL uses an expression index only for a query whose expression
matches it, so this is not cosmetic: until 5.4.0 every member was `"col" -> 'A' ->> 'B'` text,
and indexes on nested or typed members applied, enforced, and served no query
(`NpgsqlJsonMemberRenderingTests` asserts each rendering against EF's own `ToQueryString()`).
Date and time members stay text — their casts are not IMMUTABLE and cannot appear in an index —
and the part records a `TextFallback`; `ValidateQueryUsableLeadingParts` rejects a non-unique
index that starts with one (target model only). A path that *ends* at the JSON-mapped complex
property — or at a complex collection, which is always JSON — resolves to the container column
as a plain **column** part (so a whole-document GIN needs no runtime wiring); a complex property
nested inside the document resolves to a `->` extraction yielding `jsonb`. A table-split complex
property stays unresolved: there is no single column to stand for it.
nested inside the document resolves to a `jsonb` extraction. A table-split complex property
stays unresolved: there is no single column to stand for it.

**Rendering changes need `ComplexIndexAnnotations.RenderingVersion`.** Parts are resolved at diff
time on *both* sides, so changing how a member renders produces no migration at all — both sides
resolve alike and existing databases keep the old index forever. Every declaration writes the
version onto the model; a snapshot without it (scaffolded before 5.4.0) is resolved by the old
rules, which is what turns the change into one drop-and-create per affected index. Default index
names come from `ResolvedIndexPart.NameToken` (container plus path), never from the rendered SQL,
so a rendering change never renames an index; and `ResolvedIndexPart` equality ignores both
internal members, or identical SQL would still rebuild. Exclusion constraint filters pass
`renderingVersion: 1` and keep the old rendering: rebuilding a constraint buys no enforcement.
- `ResolveTemplatePart` — substitutes template placeholders with quoted columns or parenthesized
JSON extractions. Since 5.2.0 the core implements it, quoting through the `QuoteIdentifier`
virtual (ANSI by default; the SQL Server satellite brackets), so satellites override neither.
Expand Down
4 changes: 2 additions & 2 deletions Directory.Build.props
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<Project>
<PropertyGroup>
<Version>5.3.0</Version>
<Version>5.4.0</Version>
<Authors>CaffeinatedCoder</Authors>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
Expand Down Expand Up @@ -63,7 +63,7 @@
-->
<PropertyGroup>
<EnablePackageValidation>true</EnablePackageValidation>
<PackageValidationBaselineVersion>5.2.0</PackageValidationBaselineVersion>
<PackageValidationBaselineVersion>5.3.0</PackageValidationBaselineVersion>
</PropertyGroup>

<!--
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ EF Core 8.0 introduced complex properties, but migration tooling doesn't automat
- **Composite Indexes**: Define multi-column indexes spanning both scalar and nested properties with a single, intuitive expression — with per-column `ASC`/`DESC` ordering via `DbOrder.Asc`/`DbOrder.Desc`
- **Expression Indexes** *(PostgreSQL)*: Index arbitrary SQL expressions such as `lower(email)` or `to_tsvector('english', body)` — including on plain, non-complex entities
- **Typed Expression Indexes** *(PostgreSQL)*: Write `HasExpressionIndex(x => x.Email.ToLower())` and let the package translate it — property paths resolve to real columns at migration time
- **JSON Member Indexes** *(PostgreSQL)*: Index members of complex properties mapped with `ToJson()` — the same `HasComplexIndex` declaration becomes a `(col ->> 'Member')` expression index automatically
- **JSON Member Indexes** *(PostgreSQL)*: Index members of complex properties mapped with `ToJson()` — the same `HasComplexIndex` declaration becomes an expression index written the way Npgsql's queries read the member (`col ->> 'Member'`, `col #>> '{A,B}'`, cast to its type), so the queries use it
- **Temporal Constraints** *(PostgreSQL 18)*: Declare `UNIQUE … WITHOUT OVERLAPS` constraints to guarantee no two rows occupy overlapping time periods — the database enforces scheduling integrity for you
- **Exclusion Constraints** *(PostgreSQL)*: Declare `EXCLUDE USING gist (… WITH =, … WITH &&) WHERE (…)` constraints — filtered overlap protection (e.g. ignore soft-deleted rows), on any supported PostgreSQL version
- **SQL Server Options** *(SQL Server)*: Clustered, covering (`INCLUDE`), online-built, fill-factor, and data-compression index options on complex-property indexes — rendered by the stock SQL Server generator, no runtime wiring
Expand Down
4 changes: 2 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ remedy for those is to upgrade.

| Version | Supported |
|---|---|
| 5.3.x | ✅ |
| < 5.3 | ❌ |
| 5.4.x | ✅ |
| < 5.4 | ❌ |

### For how long

Expand Down
30 changes: 27 additions & 3 deletions docs/postgresql-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,9 +199,33 @@ builder.HasComplexIndex(x => x.Name.ShortName, isUnique: true, indexName: "ux_em
// ALTER: CREATE UNIQUE INDEX "ux_employer_short_name" ON employers (("name" ->> 'ShortName'));
```

Nested complex types become `->` segments (`("profile" -> 'Address' ->> 'City')`), and
`HasJsonPropertyName` is honored. Members are extracted as **text**; for typed comparisons or
ordering semantics use `HasExpressionIndex` with an explicit cast.
Each member is rendered **exactly the way Npgsql's queries read it**, because PostgreSQL uses an
expression index only for a query whose expression matches it. That decides the path operator and
the cast:

| Member | Index expression | Serves |
|---|---|---|
| `string`, enum stored as string | `"name" ->> 'ShortName'` | `x.Name.ShortName == …` |
| nested member | `"profile" #>> '{Address,City}'` | `x.Profile.Address.City == …` |
| `int`, `long`, `short`, `decimal`, `double`, `bool`, `Guid`, enum | `CAST("profile" ->> 'Rank' AS integer)` (the member's store type) | `x.Profile.Rank == …`, `> …`, `ORDER BY` |
| `byte[]` | `decode("profile" ->> 'Blob', 'base64')` | `x.Profile.Blob == …` |
| primitive collection, `JsonDocument` or other `jsonb` member | `"profile" -> 'Tags'` (`jsonb`) | a GIN over it |
| `DateTime`, `DateTimeOffset`, `DateOnly`, `TimeOnly` | `"profile" ->> 'At'` (text) | uniqueness only |

`HasJsonPropertyName` is honored throughout. Date and time members are the exception: Npgsql's
queries cast them to `timestamp with time zone`, `date` or `time`, and PostgreSQL cannot index those
casts (the conversion from text is not `IMMUTABLE`), so no index can match the query. The member
stays text, a **unique** index over it still enforces uniqueness, and a **non-unique** index that
starts with it is rejected at `migrations add`, since no query would ever use it. Put another part
first, or map the member to a regular column.

> **Upgrading from 5.3 or earlier:** before 5.4.0 every member was extracted as text with `->`
> segments, which only matched the query for a top-level string. Such indexes enforced uniqueness
> but no query used them. The first `migrations add` after upgrading drops and re-creates each
> affected index under its existing name; indexes on top-level string members are untouched.
> Until that migration exists, `has-pending-model-changes` reports changes and `Migrate()` raises
> EF Core's pending-model-changes error. The re-create is a plain `CREATE INDEX` that blocks writes
> while it builds; on a large table, declare the index with `IsCreatedConcurrently()` first.

### Indexing the whole document

Expand Down
21 changes: 21 additions & 0 deletions src/EFCore.ComplexIndexes.PostgreSQL/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,27 @@ Changes to the PostgreSQL satellite, newest first. The
[root changelog](https://github.com/CaffeinatedCoder/EFCore.ComplexIndexes/blob/main/CHANGELOG.md)
covers all three packages.

## 5.4.0

- **Fixed:** indexes on `ToJson()` members are written the way Npgsql's queries read the member —
`->>` or `#>> '{A,B}'`, cast to the member's store type unless it is a string, `decode(…,
'base64')` for `byte[]`, `jsonb` for a primitive collection or a `json`/`jsonb` member — so
PostgreSQL can use them. Before,
every member was `"doc" -> 'A' ->> 'B'` text, which matched a query only for a top-level string:
indexes on nested or typed members applied, enforced uniqueness, and served no query. Applies to
index parts, typed expression indexes and index filters; exclusion constraint filters keep their
rendering. Upgrading rebuilds each affected index once, under its existing name, at the first
`migrations add`.
- **Changed:** a non-unique index that starts with a `DateTime`, `DateTimeOffset`, `DateOnly` or
`TimeOnly` JSON member is rejected at `migrations add`: the queries cast the member to a type
PostgreSQL cannot index from text, so the index could never be used. Unique ones are allowed.
- **Fixed:** names are checked across kinds at `migrations add`: constraint names per table
(temporal, temporal foreign key and exclusion constraints against each other and against EF's
keys, foreign keys and check constraints) and index names per schema (complex indexes and the
index behind every unique, exclusion and temporal constraint, against EF's indexes and keys).
Such a clash failed at apply time with 42710 or 42P07 — or, against an exclusion constraint,
whose ADD is preceded by `DROP CONSTRAINT IF EXISTS`, silently dropped the other constraint.

## 5.2.0

- **Changed:** an index, exclusion constraint, temporal constraint or temporal foreign key whose
Expand Down
Loading
Loading