--
326520769 by kuat <[email protected]>:

BEGIN_PUBLIC
- update copybara mapping
- drop -std=c++17 copts since it causes linking issues with absl (which has its own string_view). The flag is now passed directly via bazelrc invocation
- fix lifetime issues with string_view in tests
- update dependencies
- manually unroll status macros
END_PUBLIC
--
325896067 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Clean-up iteration variable code.
END_PUBLIC

Since we have to allocate multiple a frame at comprehension start to properly handle nested comprehensions, preallocating map slots for iter vars doesn't provide much of an advantage any more.

benchmarks: https://docs.google.com/spreadsheets/d/1MB7JpdjKa3VbZsgNOObySUIruFroiJybHlI4NEVZ8Ao/edit?usp=sharing

There may be a slight regression for smaller expressions, but it's hard to tell if it's significant given the noise from benchmarking on my cloudtop.
--
325885243 by kuat <[email protected]>:

BEGIN_PUBLIC
Refactor CreateMessage, a direct protobuf dependency, from CelValue definition.
END_PUBLIC

Tested:
Regular TAP presubmit

CLEANUP=y
BUG=158496650
croupier-relnote: n/a
RELEASE_NOTIFY=n/a
EXP_FLAG=REFACTOR_WITH_NO_FUNCTIONAL_CHANGE
PROD_RISK=none
BUG=0

Move third_party/cel/cpp/eval/public/cel_value.h to third_party/cel/cpp/eval/public/structs/cel_proto_wrapper.h

clang-migrate spec:
old_header: "third_party/cel/cpp/eval/public/cel_value.h"
new_header: "third_party/cel/cpp/eval/public/structs/cel_proto_wrapper.h"
renames {
  kind: CLASS_STATIC_METHOD
  old_symbol: "google::api::expr::runtime::CelValue::CreateMessage"
  new_symbol: "google::api::expr::runtime::CelProtoWrapper::CreateMessage"
}
class_scope_symbol_rename_only: true
--
325497260 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add benchmarks for comprehensions.
END_PUBLIC

Add a benchmark to confirm unknowns cleanup doesn't affect performance on comprehensions.
--
325359772 by CEL Dev Team <[email protected]>:

LSC Migrate usage of `StatusOr::operator bool` to `StatusOr::ok`, in preparation to remove `StatusOr::operator bool`. This enables the release of `absl::StatusOr` in Abseil.

More details: go/lsc-statusor-operator-bool,
go/elimination-of-statusor-operator-bool.

Tested:
    TAP --sample ran all affected tests and none failed
    http://test/OCL:325229099:BASE:325216829:1596725828334:344b6130
--
325291978 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add bzl_library rules for .bzl files without one.
END_PUBLIC

Add bzl_library rules for .bzl files without one.

Seeding google3 with bzl_library rules will allow us to start treating .bzl like other languages in google3. Strict dependencies, test setup and best practices.

For third_party rules, existence of bzl_library rules will not be enforced, but
it is a best practice. If you wish to not add bzl_library rules for your
package feel free to say so in a CL comment.

More information: go/seed-bzl-library

#rebuilder

Tested:
    TAP found no affected targets. No targets were built or tested.
    http://test/OCL:325061836:BASE:325038549:1596652230512:54c3464
--
324715519 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add basic implementation of reference resolution in checked Exprs
END_PUBLIC
--
323856311 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Introduce max recursion depth to the Parser to prevent stack overflows.
END_PUBLIC

I also:
* Removed test sharding since it interacts poorly with the reproduction command
  given in the auto-created fuzz bug
* Added relevant testcase to the test corpus
--
323847106 by CEL Dev Team <[email protected]>:

Migrates uses of deprecated map types to recommended types.

THIS IS NOT A TRIVIAL BUT SAFE CHANGE; THIS IS A NON-TRIVIAL AND PROBABLY SAFE CHANGE (your tests passed).  THIS CHANGE WILL LIKELY:
Alter the order of iteration for your unordered containers.  The new containers do not have a stable iteration order under any circumstances.
Save you RAM and CPU.
Change the default hash you are using to a better default hash.
Alter the hash key that is used by default for requests (custom hashers remain the same).  This can affect capacity during a rollout by causing a spike in cache misses until the new cache keys are warm.

IF THIS CHANGE DOES NOT BUILD, your targets are not registered with any TAP builds (go/tap-blueprint).

Because of pointer stability, containers that provide pointer stability are being migrated to node_hash_map.  Code that does not need pointer stability for elements in these containers can be switched to a flat_hash_map.  Reviewer provided edits (go/edit-from-critique) or follow up CLs are greatly appreciated.

These changes will save 100s of Terabytes of RAM and 100s-1000s of kGCUs and also put us in a better place for responding to hash flooding attacks in the future.  These fall under the go/capacity-cy currently affecting our fleet.

We are collectively managing thousands of CLs for this effort.  Please do not ask us to fix pre-existing warnings.  Reviewer provided edits (go/edit-from-critique) for nits are greatly appreciated as they save us a huge amount of time.  Owners are encouraged to change the types further in follow up CLs.

For details on this lsc, see go/map-marauder-lsc
For details on the new maps, see go/hashmaps and go/swisstable

Tested:
    tap_presubmit: http://test/OCL:323413933:BASE:323384321:1596006698288:943588ca
    Some tests failed; test failures are believed to be unrelated to this CL
--
323833894 by kuat <[email protected]>:

BEGIN_PUBLIC
Fix a crash in the optimizing call for binary cond visitor, as found by a fuzzer.
Fixes #65
END_PUBLIC
--
323004345 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Move container implementations to a subdirectory under /public
END_PUBLIC

context: One part of isolating the arena usage is maintaining a set of container implementations that we need to know how to allocate. This refactor makes that a bit neater.

croupier-relnote: n/a
--
322721429 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Fix out-of-range issues with ConditionalAnd / ConditionalOr inside c++ parser.
END_PUBLIC

This is the equivalent of
cel-expr/cel-go@f5f8bc2
for C++.
--
322681023 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Truncate the number of error messages returned by the parser. For certain
inputs, the parser allocates an unbounded amount of memory just for returning
error messages.
END_PUBLIC

This fixes the forge OOM issue. Also refactored errorMessage handling entirely into source_factory where I believe it belongs.
--
322677963 by jimlarson <[email protected]>:

Refactor expression printer into test utilities.

BEGIN_PUBLIC
Refactor expression printer into test utilities.

Also allows expression printer to be immutable and idempotent - it was
previously one-shot.
END_PUBLIC
--
321841589 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Fix C++ parser crash in ParserVisitor::visitFieldInitializerList.
END_PUBLIC
--
321660001 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Fix C++ CEL Parser crash when attempting to report error, and add relevant
reproducing test case to unit test.
END_PUBLIC
--
321591306 by kuat <[email protected]>:

BEGIN_PUBLIC
Set C++17 compiler option.
END_PUBLIC
--
321531133 by CEL Dev Team <[email protected]>:

Add Unrecoverable Unknown (error injection) support to CEL evaluator.

BEGIN_PUBLIC
Implement unrecoverable unknown support (error injection) in CEL evaluator.
Introduce a new field error_paths_ in Activation. Then, in evaluation of Select and Ident expressions, a CelError is returned as a result if error_paths_ contains the current path.
Terminal unknowns can be marked by adding paths to this field.
A boolean flag InterpreterOptions.enable_error_injection is added to guard this feature.
END_PUBLIC

go/terminal-unknown

RELNOTES: Add Unrecoverable Unknown (error injection) support to CEL evaluator.
--
321009523 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add optional parameter for MissingOverloads error to say which function is missing an overload.
END_PUBLIC
--
320984872 by tswadell <[email protected]>:

BEGIN_PUBLIC
Import of cel-go v0.5.1 from Github(https://github.com/google/cel-go.git).
END_PUBLIC

Included changes:

  - 199f791885764c1736178a1132f08706efef8410 Add runtime support for timestamp from Unix epoch (#358)
  - aa4b2a7d6f496f7fb8642a0719aea7bb2ae28f18 Implement a feature flag mechanism. (#354)
  - f3a472121b19f1042707024dc5b3283c2e3dfb68 Introduce a sync.Pool for ANTLR Lexer / Parser instances ...
  - abf60946ee520b24520c843249efc94b7ef378dd Fix namespaced function resolution (#341)
  - 3023710f92aa1469e7c0734514b080e15fbcb6a5 Fix the doc comments referring to the extant ClearBuiltin...
  - 623e8a635bf528c113d2592279fb6bc5310a3cb7 Stable sort errors by location (#343)
  - c4c3df541e541a8cc2fa0b5760fc88cf29f5e58d Introduce decls.NewVar and decls.NewConst (#342)
--
320472813 by CEL Dev Team <[email protected]>:

LSC: Replace the C++ MOCK_METHOD<n> family of macros with the new MOCK_METHOD in third_party/cel.  See go/totw/164.

BEGIN_PUBLIC
Use the new Google Mock MOCK_METHOD macro instead of the older MOCK_METHOD<n> macros.
END_PUBLIC

#updatemockmethod #codehealth
--
320250626 by mikedanese <[email protected]>:

implement google.protobuf.Any support in cel-cpp

The current implementation drops them silently in SetMessage.

BEGIN_PUBLIC
implement google.protobuf.Any support in cel-cpp
END_PUBLIC
--
319801546 by CEL Dev Team <[email protected]>:

Prototype for IndexMetadataExpressionEvaluator.

This is a class that takes a set of go/cel expressions (one for topic-level filtering, another for sp-level) and executes them on a MultiMDU representing a topic. The assumption is that each MDU has an IndexMetadata proto bound to the userdata field.

BEGIN_PUBLIC
Internal change
END_PUBLIC
--
319276695 by CEL Dev Team <[email protected]>:

This CL introduces a view layer for pToken attributes (for scalar pToken, context, operation), which would mix derived & basic attributes together. It doesn't require pre-materialization of attributes in a container, proto, etc.

Attributes are typed using the CEL values, with a CEL-less type-safe wrapper layer provided in typed_views.h

Views are provided over various protocol-buffer based representations, and support boxed-types and derived attribute staleness. More view implementations would expose a derived attribute cache, SecurityContext, and so on.

Basic attributes currently present in PTokenInternal are added to the PTokenAttributes registry so that they can be referenced using the type-safe lookups in typed_views.h

Tested:
  Unit tests are added for the new classes. No existing code is changed.

BEGIN_PUBLIC
internal change
END_PUBLIC
--
319266365 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add helper for creating CelAttributePatterns.
END_PUBLIC
--
318192649 by CEL Dev Team <[email protected]>:

This generated CL removes comments next to a licenses() rule in the BUILD file.

It's desirable to remove these comments as they are easy to fall out of sync with the LICENSE file causing confusion.
--
317682809 by kuat <[email protected]>:

BEGIN_PUBLIC
Remove superfluous include.
END_PUBLIC
--
316972268 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add basic gtest matchers for cel values.
END_PUBLIC
--
316850412 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Internal change
END_PUBLIC

Make third_party/cel/cpp/parser:parer library visible to
//privacy/ptoken/extraction
--
316223239 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Internal change
END_PUBLIC

Make third_party/cel/cpp/parser:parer library visible to
//video/youtube/abuse/enforcement/protection
--
315923824 by kmilsht <[email protected]>:

BEGIN_PUBLIC
Fixing null pointer situation when using dynamically constructed message of well-known types (google.protobuf) with CelValue::CreateMessage
END_PUBLIC
--
315271343 by CEL Dev Team <[email protected]>:

LSC: Replace cord.ToString() with std::string(cord)

This method is deprecated and will be removed.

LSC document: go/lsc-absl-cord-members
Bug: b/157745958

http://refactor/4636887511536581949 Master CL created by go/Zwingli.

Tested:
    tap_presubmit: http://test/OCL:314910097:BASE:314916222:1591474813453:da5b614c
    Some tests failed; test failures are believed to be unrelated to this CL
--
315006421 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add attr trail to iter_var. This should allow unknown paths to be maintained in comprehensions.
END_PUBLIC

modified rollback of cl/302120264
--
314991879 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Add debugging utility for printing cel values.
END_PUBLIC
--
314609599 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Move ternary builtin to a custom step instead of a built-in function. This helps make the special error handling semantics consistent with shortcutting on or off.
END_PUBLIC
--
314566279 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Use the logical step instead of built-in functions for handling or/and. This makes it a bit easier to handle special processing for errors and unknowns.
END_PUBLIC
--
314232099 by CEL Dev Team <[email protected]>:

Implements support for using the CEL parser in third_party in unit tests instead of creating CEL ASTs through helper functions.

Updated a first round of unit tests to use the parser and removed the helper functions.

BEGIN_PUBLIC
BUILD file visibility changes
END_PUBLIC
--
311339636 by CEL Dev Team <[email protected]>:

BEGIN_PUBLIC
Internal change
END_PUBLIC

Make third_party/cel/cpp/parser:parer library visible to
//cloud/ml/api/conversation/service/handler/cobweb.
--
311335037 by timdn <[email protected]>:

BEGIN_PUBLIC
Add CEL multi-threaded parsing test

On the Open Source version, this test breaks with Antlr 4.7.1, while it
suceeds with 4.7.2, which has received thread-safety improvements (breakage
rate was beyond 95% during my tests).
END_PUBLIC

On the internal version, this also works on 4.7.1, probably due to some
sanitization during builds. However, an upgrade to 4.7.2 seems advisable.

PiperOrigin-RevId: 326520769

Co-authored-by: CEL Dev Team <[email protected]>