chore: absorb query into packages/query - #13896

Reintroduces Query::LOGICAL_TYPES (now a list of Method cases), uses
Method->value for shape() string concat, migrates lingering
Query::TYPE_AND/OR/ELEM_MATCH references in tests to Method cases, and
resolves PHPStan level-max issues across Builder/MongoDB, Parser/MongoDB,
and assorted test files (tighter array shapes, explicit instanceof
guards, extracted CollectingVisitor, widened mergeIntoCollection pipeline
types, removed dead helpers).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

CI runners are too variable to meet the <1-5us/op targets these tests
assert against; only run them on dedicated hardware.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…ggregate pipelines

update() built update operators before filters, but binding replacement
walks the serialized op JSON in key order (filter first, then update),
so the wrong bindings ended up in each slot — filters silently matched
nothing and updates set the wrong values.

Also, empty stdClass instances (BSON "{}") were being lost when the
integration client round-tripped queries through json_decode(assoc:true),
degrading operators like $documentNumber into invalid BSON arrays.
Decode as objects and preserve empty stdClass so the MongoDB driver
still encodes them as documents.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Window function aliases were produced by \$setWindowFields but then
dropped by the subsequent \$project stage because they weren't in the
select list, so callers never saw the generated column.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

… and MongoDB parser

Agent-Logs-Url: https://github.com/utopia-php/query/sessions/7b5bbec4-5637-4fb7-bde6-4b35dcdd4921

Co-authored-by: abnegate <[email protected]>

…entifier escaping

- Serializer: pass a stricter precedence to the right child for
  non-commutative left-associative operators (-, /, %) so trees like
  a - (b - c) serialise with parens and are not silently rewritten as
  a - b - c.
- Serializer: always separate a prefix unary operator from its operand
  so '-' followed by a negative numeric literal or nested unary can no
  longer collide into '--' and become a MySQL line comment.
- Serializer: escape backslashes in string literals before escaping
  single quotes, so a value ending in '\\' cannot break out of the
  quoted literal.
- Parser: un-double escaped delimiters in backtick, double-quoted and
  bracket-quoted identifiers (e.g. `foo``bar` -> foo`bar).
- MySQL tokenizer: mirror the single-quote branch's backslash-skip
  inside "..." so a # after an escaped quote stays inside the string
  literal and is not rewritten to --.
- MySQL tokenizer: replace # with -- (not '-- ') so the resulting line
  comment text matches the original.

- buildJoinStages now throws UnsupportedException when a non-equality join
  operator is passed — $lookup localField/foreignField only supports equality.
- update() rejects setRaw/setCase/conflictRaw sets instead of silently dropping
  them; these have no clean MongoDB translation.
- buildWindowFunctions rejects multi-argument window functions (COVAR(a, b))
  and requires ORDER BY for RANK/DENSE_RANK/ROW_NUMBER at build time instead
  of failing at runtime inside MongoDB.
- buildFieldExists now emits {$type: 10} for IS NULL and
  {$exists: true, $ne: null} for IS NOT NULL so the result mirrors SQL's
  IS NULL semantics (present and explicitly null) instead of conflating
  missing and null documents.
- insertOrIgnore builds its operation descriptor directly instead of
  round-tripping through json_decode, avoiding empty-stdClass corruption.
- buildDistinct resolves each attribute once per field instead of twice,
  halving attribute-hook invocations.
- Imports stdClass at the top and drops leading backslashes at call sites.
- Adds regression tests for every fix above, including the window-function
  projection preservation from commit 06ceaca.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

MySQL's default SQL mode treats backslash as an escape character, so values
ending in `\` can escape the closing single quote and break out of quoted
strings. The previous `str_replace("'", "''", $value)` only doubled single
quotes, leaving backslashes unescaped.

- Escape backslash before single quote in every DDL string-literal path:
  column comments, default values, ENUM values, table comments, column
  comments, partition names, collation option values, sequence names.
- ClickHouse enum escape now also doubles the backslash before applying
  `\'` quote escape so a value like `\'` cannot terminate the literal.
- `PostgreSQL::createCollation()` now validates option keys against
  `/^[A-Za-z_][A-Za-z0-9_]*$/` and throws `ValidationException` for keys
  that would allow SQL injection (e.g. `"provider = 'x', danger"`).
- `PostgreSQL::tablesample()` validates method against `BERNOULLI|SYSTEM`.
- `PostgreSQL::explain()` validates format against `TEXT|XML|JSON|YAML|''`.

Adds regression tests covering backslash-escaped enum/default/comment in
MySQL and ClickHouse, invalid collation option keys, and invalid
TABLESAMPLE method / EXPLAIN format on PostgreSQL.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Per project rule, prefer `array_push($items, ...$new)` over `array_merge`
in loops (merge copies the entire array every iteration) and over
per-element `foreach`-forwarding when a bulk append suffices.

- Add `Builder::addBindings(array)` that does one `array_push` spread
  instead of N method calls. Replace every
  `foreach ($x->bindings as $b) { \$this->addBinding(\$b); }` pattern
  across Builder.php and the MySQL, PostgreSQL, ClickHouse, MongoDB,
  SQL sub-builders.
- Replace `array_merge` in the recursive AST walks
  (`astWhereToQueries`, `astExpressionToSingleQuery`) and in
  `Query::validate()`'s nested-query recursion with `array_push` spreads.
- Import `stdClass` in `Builder/MongoDB.php` and `Schema/MongoDB.php`
  instead of referencing `\stdClass` with a leading backslash.
- Import `ValidationException` at the top of `Query.php` so the two
  `throw new \Utopia\Query\Exception\ValidationException(...)` sites in
  `Query::page()` use the short name.

No behaviour change. All unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…932148

…b-builder-guards

# Conflicts:
#	src/Query/Builder/MongoDB.php

…mes, parser depth, and tokenizer bounds

- Builder::selectCast() now rejects cast types that contain characters
  outside [A-Za-z0-9_(), ] to prevent raw-SQL injection through the
  CAST target.
- Builder::selectWindow() now rejects function strings that do not
  match the shape identifier(...), blocking trailing clauses, comment
  markers, and statement terminators.
- AST Parser enforces a recursion depth limit (256) on parseExpression
  to prevent stack overflow on deeply-nested parenthesised input.
- Tokenizer::readString() now throws ValidationException when a
  backslash appears at EOF, replacing a silent pos-past-length bug.
- MongoDB builder rejects field names that are empty or start with
  '$' across set, push, pull, pullAll, addToSet, increment, multiply,
  rename, unsetFields, currentDate, popFirst, popLast, updateMin,
  updateMax, and pushEach.
- Added unit tests for fromNone() and selectCast() across MySQL,
  PostgreSQL, SQLite, and ClickHouse dialects.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…cit opt-in

Raw queries bypass the binding/escaping pipeline, so accepting them from
JSON lets an attacker smuggle arbitrary SQL through any endpoint that calls
Query::parse on untrusted input. parse, parseQuery, and parseQueries now
throw ValidationException when they encounter Method::Raw unless the caller
explicitly passes allowRaw: true (for trusted round-trips such as in-memory
caches). The flag propagates into nested logical queries so Raw cannot be
hidden inside an or/and/elemMatch either.

Also rewrite groupByType() from an IIFE-per-arm match (true) pattern to
plain switch/case with direct mutations: one Closure allocation per call
per arm disappears and the control flow reads in source order. While there,
fix the cursor fallback that previously coerced a null cursor value to the
row-count limit; it now stays null.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Replace stringly-typed $direction ('ASC'/'DESC') and $nulls
('FIRST'/'LAST'/null) on OrderByItem with OrderDirection and
NullsPosition enum types. Producers (Parser, Builder) and consumers
(Serializer, Builder order-by emission) updated to use enum cases
and enum->value for SQL emission. Emitted SQL shape is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

… introduce MongoDB enums

Replace 13 parallel update-operation fields (pushOps, pullOps, addToSetOps,
incOps, unsetOps, renameOps, mulOps, popOps, pullAllOps, minOps, maxOps,
currentDateOps, pushEachOps) with a single updateOperations table keyed by
UpdateOperator enum value. Each operator's payload shape is constructed in
its setter, and buildUpdate() iterates once over the table to emit the final
update document.

Introduce three enums to eliminate magic strings throughout the builder:
Operation (find/insertMany/updateMany/deleteMany/updateOne/aggregate),
UpdateOperator (\$set/\$push/\$pull/etc.), and PipelineStage (\$match/\$group/
etc.). Also add an UpdateOperation readonly value object for typed consumers.

JSON output shape and all public method signatures are preserved — this is a
pure internal restructuring with compile-time safety guarantees.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…patialDistanceFilter DTO

QuotesIdentifiers::quote() now skips explode/array_map/implode for dotless
identifiers (the vast majority of inputs), directly wrapping and escaping
in one pass. In the dotted path, only the final segment may be bare '*' —
intermediate '*' segments are now quoted as literal identifiers instead of
silently passed through.

Method::sqlFunction() provides a single authoritative mapping from
aggregate/statistical/bitwise Method cases to standard SQL function names,
returning null for non-aggregation methods. Callers can migrate in later
cycles.

SpatialDistanceFilter is a typed readonly DTO replacing the opaque
[geometry, distance, meters] 3-tuple at the read sites in MariaDB, MySQL,
and PostgreSQL builders. Write sites in Query.php are untouched; the DTO
normalizes the tuple via fromTuple() so callers get named fields and
compile-time safety.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…ckHouse regex-based afterBuild with structured slot hooks

Builder::build() was a ~466-line monolith handling SELECT, CTE, UNION,
15+ clause sections, and alias qualification inline. It now orchestrates
named private helpers (buildCtePrefix, buildSelectClause, buildFromClause,
buildJoinsClause, buildWhereClause, buildGroupByClause, buildHavingClause,
buildWindowClause, buildOrderByClause, buildLimitClause, buildLockingClause,
buildUnionSuffix) plus a prepareAliasQualification step and a
compileWindowSelect / compileOrderByList / buildAggregationAliasMap trio.
Each helper has a single responsibility and emits its bindings in document
order, preserving byte-identical SQL output.

ClickHouse used to compile via parent::build() then post-hoc regex-rewrite
the SQL to splice in ARRAY JOIN, raw ASOF joins, GROUP BY modifier,
LIMIT BY, and SETTINGS. That approach could match keywords inside string
literals or identifiers and required a hand-rolled placeholder counter
(preg_match_all on '?') to reposition the LIMIT BY binding.

Builder now exposes four no-op protected hooks that subclasses override
to emit dialect-specific fragments at the correct position during build():

  buildAfterJoinsClause()    after JOINs, before WHERE
  buildAfterGroupByClause()  after GROUP BY, before HAVING
  buildAfterOrderByClause()  after ORDER BY, before LIMIT
  buildSettingsClause()      trailing settings, before UNION suffix

ClickHouse overrides all four. Its ARRAY JOIN / raw ASOF joins / PREWHERE
collapse into buildAfterJoinsClause; groupByModifier into
buildAfterGroupByClause; LIMIT BY (with its count binding added at the
moment of emission) into buildAfterOrderByClause; SETTINGS into
buildSettingsClause. The ClickHouse::build() override,
injectBeforeFirstKeyword, findKeywordPosition, and the preg_match_all
placeholder-counting hack are all deleted -- bindings ordering is now
naturally correct because emission happens in document order.

A new regression test asserts that identifiers (settings_table,
array_join_col, limit_by_col, order_by_col) and bound string literals
containing SQL clause keywords ('LIMIT 1 SETTINGS foo', 'ARRAY JOIN tags',
'PREWHERE condition') round-trip through the builder untouched -- a
guarantee the regex-based version could not make.

Full unit suite (4164 tests, 10714 assertions) passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…tate machine

- PostgreSQL.alterColumnType: validate $type allowlist and reject ;/--/block
  comments plus 1024-char cap on USING expression
- PostgreSQL.createPartition: same semicolon/comment/length rejection on the
  partition expression
- Schema.compileIndexColumns: validate collation against identifier allowlist
  and require orders to be exactly ASC/DESC (OrderDirection enum)
- Schema/Index: widen collation allowlist to permit quoted identifiers
- SQL.createProcedure / PostgreSQL.createTrigger+createProcedure: document
  trust requirement and reject \$\$ inside dollar-quoted PL/pgSQL bodies
- Parser/SQL: replace naive byte scans in extractKeyword and classifyCTE
  with a shared state machine that skips single/double/backtick quoted
  strings, dollar-quoted bodies, line comments and block comments — keeps
  parenthesis depth honest and prevents keyword misclassification when
  payloads hide DML inside string literals or comments
- Parser/MongoDB: validate skipBsonString and skipBsonBinary length fields
  against the remaining buffer (and reject negative lengths on 32-bit PHP)
  so a crafted strLen=0xFFFFFFFF cannot drive reads past the wire buffer

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…kenizer coverage

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

… SpatialDistanceFilter DTO from worktree-agent-a25589ce

…ree-agent-add57390

# Conflicts:
#	tests/Query/Builder/MongoDBTest.php

…s from worktree-agent-a4a0da49

# Conflicts:
#	tests/Query/Builder/ClickHouseTest.php

- MongoDBTest: assertIsArray($op['projection'|'filter']) before assertArrayHasKey
  on mixed-typed map access (reserved-word/unicode tests from af4b444b).
- ClickHouseClient: fall back to original query when preg_replace_callback
  returns null.
- QueryParseTest: assertInstanceOf(Query::class) before calling getMethod()
  on array<mixed> element.

…e Case\Builder to Case\Expression; README BuildResult->Plan

- README no longer references the non-existent BuildResult class; every build()/insert()/update()/delete() returns Plan
- #[\Override] attributes on every overriding method across Builder, SQL, MySQL, MariaDB, PostgreSQL, SQLite, ClickHouse, MongoDB, AST serializers and AST visitors (343 attributes). PHP verifies correctness at load-time
- Case namespace: Builder->Expression (fluent), Expression->Result (DTO). Resolves naming collision with top-level Builder. All callers and tests updated
- README CASE example rewritten to the real API (no Builder::case() method)

Adds tests exercising the exact attack vectors closed by:

- d203ed7 (DDL validation + wire-parser state machine):
  alterColumnType/createPartition input rejection, extractKeyword
  ignores keywords inside quoted strings and block comments, classifyCTE
  ignores INSERT hidden in a string literal.
- 5662d27 (cast/window/mongo/parser/tokenizer bounds):
  MongoDB builder rejects dollar-prefixed and empty field names.
- c5a4ed3 (DDL backslash escaping):
  MySQL enum with trailing backslash, createCollation rejects invalid
  option keys, PostgreSQL tablesample rejects injected method token.
- 4eb2996 (AST tightening): DDL default value with backslash is
  doubled so a trailing backslash cannot escape the closing quote.
- ff64121 (Method::Raw): parse/parseQuery reject Raw by default, accept
  when allowRaw=true, and still reject Raw nested inside Or.

Covers Features that previously had no dedicated unit test file. Each
test file exercises the happy path, NULL/empty-input edge cases,
dialect-specific quoting, and binding-order properties using assertSame.

Added:

- Feature/SpatialTest
- Feature/LateralJoinsTest
- Feature/BitwiseAggregatesTest
- Feature/StatisticalAggregatesTest
- Feature/PostgreSQL/VectorSearchTest
- Feature/PostgreSQL/MergeTest
- Feature/PostgreSQL/ReturningTest
- Feature/PostgreSQL/OrderedSetAggregatesTest
- Feature/ClickHouse/ArrayJoinsTest
- Feature/ClickHouse/AsofJoinsTest
- Feature/MongoDB/AtlasSearchTest
- Feature/MongoDB/PipelineStagesTest
- Feature/MongoDB/FieldUpdatesTest
- Feature/MongoDB/ArrayPushModifiersTest

…0 methods)

Brings ClickHouseIntegrationTest.php from 15 to 20 test methods, covering
the gaps against MySQLIntegrationTest and PostgreSQLIntegrationTest:

- testSelectWithBetween
- testSelectWithStartsWithAndContains
- testSelectWithCaseExpression
- testSelectWithArrayJoin
- testSelectWithExistsSubquery

All new tests exercise real ClickHouse behaviour via the existing
ClickHouseClient harness and pass against clickhouse-server 24.

SQL compileFilter accepted search/regex/exists on nested JOIN ON while
toAst() silently rewrote them to Raw, so build() and toAst() disagreed.

SQL compilation prefixes unqualified Query::on operands with the base-table alias when joins are present; toAst() now applies the same qualification so AST consumers resolve the same JOIN columns.

feat: nest join ON conditions as Query objects

Index types already use IndexType; column order was still a raw ASC/DESC
string. Schema\Order is the matching enum so call sites do not keep
magic strings next to typed index types.

(feat): add Schema\Order for index column directions

Postgres carries NOT NULL through an ALTER COLUMN ... TYPE, so a column
that moves between required and optional keeps its old constraint. MySQL
resets it as part of MODIFY COLUMN, which is why only Postgres needs the
constraint altered on its own.

Co-Authored-By: Claude Opus 5 <[email protected]>

feat(postgres): alter a column's nullability

…684093'

git-subtree-dir: packages/query
git-subtree-mainline: b866958
git-subtree-split: 67f89d9

# Conflicts:
#	composer.lock

# Conflicts:
#	composer.lock

# Conflicts:
#	composer.json
#	composer.lock

# Conflicts:
#	composer.lock

# Conflicts:
#	composer.json
#	composer.lock

# Conflicts:
#	composer.lock

# Conflicts:
#	composer.lock