https://docs.google.com/spreadsheets/d/1MB7JpdjKa3VbZsgNOObySUIruFroiJybHlI4NEVZ8Ao/edit?usp=sharing" rel="nofollow">https://docs.google.com/spreadsheets/d/1MB7JpdjKa3VbZsgNOObySUIruFroiJybHlI4NEVZ8Ao/edit?usp=sharing There may be a slight regression for smaller expressions, but it's hard to tell if it's significant given the noise from benchmarking on my cloudtop. -- 325885243 by kuat <kuat@google.com>: BEGIN_PUBLIC Refactor CreateMessage, a direct protobuf dependency, from CelValue definition. END_PUBLIC Tested: Regular TAP presubmit CLEANUP=y BUG=158496650 croupier-relnote: n/a RELEASE_NOTIFY=n/a EXP_FLAG=REFACTOR_WITH_NO_FUNCTIONAL_CHANGE PROD_RISK=none BUG=0 Move third_party/cel/cpp/eval/public/cel_value.h to third_party/cel/cpp/eval/public/structs/cel_proto_wrapper.h clang-migrate spec: old_header: "third_party/cel/cpp/eval/public/cel_value.h" new_header: "third_party/cel/cpp/eval/public/structs/cel_proto_wrapper.h" renames { kind: CLASS_STATIC_METHOD old_symbol: "google::api::expr::runtime::CelValue::CreateMessage" new_symbol: "google::api::expr::runtime::CelProtoWrapper::CreateMessage" } class_scope_symbol_rename_only: true -- 325497260 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add benchmarks for comprehensions. END_PUBLIC Add a benchmark to confirm unknowns cleanup doesn't affect performance on comprehensions. -- 325359772 by CEL Dev Team <cel-dev@google.com>: LSC Migrate usage of `StatusOr::operator bool` to `StatusOr::ok`, in preparation to remove `StatusOr::operator bool`. This enables the release of `absl::StatusOr` in Abseil. More details: go/lsc-statusor-operator-bool, go/elimination-of-statusor-operator-bool. Tested: TAP --sample ran all affected tests and none failed http://test/OCL:325229099:BASE:325216829:1596725828334:344b6130 -- 325291978 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add bzl_library rules for .bzl files without one. END_PUBLIC Add bzl_library rules for .bzl files without one. Seeding google3 with bzl_library rules will allow us to start treating .bzl like other languages in google3. Strict dependencies, test setup and best practices. For third_party rules, existence of bzl_library rules will not be enforced, but it is a best practice. If you wish to not add bzl_library rules for your package feel free to say so in a CL comment. More information: go/seed-bzl-library #rebuilder Tested: TAP found no affected targets. No targets were built or tested. http://test/OCL:325061836:BASE:325038549:1596652230512:54c3464 -- 324715519 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add basic implementation of reference resolution in checked Exprs END_PUBLIC -- 323856311 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Introduce max recursion depth to the Parser to prevent stack overflows. END_PUBLIC I also: * Removed test sharding since it interacts poorly with the reproduction command given in the auto-created fuzz bug * Added relevant testcase to the test corpus -- 323847106 by CEL Dev Team <cel-dev@google.com>: Migrates uses of deprecated map types to recommended types. THIS IS NOT A TRIVIAL BUT SAFE CHANGE; THIS IS A NON-TRIVIAL AND PROBABLY SAFE CHANGE (your tests passed). THIS CHANGE WILL LIKELY: Alter the order of iteration for your unordered containers. The new containers do not have a stable iteration order under any circumstances. Save you RAM and CPU. Change the default hash you are using to a better default hash. Alter the hash key that is used by default for requests (custom hashers remain the same). This can affect capacity during a rollout by causing a spike in cache misses until the new cache keys are warm. IF THIS CHANGE DOES NOT BUILD, your targets are not registered with any TAP builds (go/tap-blueprint). Because of pointer stability, containers that provide pointer stability are being migrated to node_hash_map. Code that does not need pointer stability for elements in these containers can be switched to a flat_hash_map. Reviewer provided edits (go/edit-from-critique) or follow up CLs are greatly appreciated. These changes will save 100s of Terabytes of RAM and 100s-1000s of kGCUs and also put us in a better place for responding to hash flooding attacks in the future. These fall under the go/capacity-cy currently affecting our fleet. We are collectively managing thousands of CLs for this effort. Please do not ask us to fix pre-existing warnings. Reviewer provided edits (go/edit-from-critique) for nits are greatly appreciated as they save us a huge amount of time. Owners are encouraged to change the types further in follow up CLs. For details on this lsc, see go/map-marauder-lsc For details on the new maps, see go/hashmaps and go/swisstable Tested: tap_presubmit: http://test/OCL:323413933:BASE:323384321:1596006698288:943588ca Some tests failed; test failures are believed to be unrelated to this CL -- 323833894 by kuat <kuat@google.com>: BEGIN_PUBLIC Fix a crash in the optimizing call for binary cond visitor, as found by a fuzzer. Fixes #65 END_PUBLIC -- 323004345 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Move container implementations to a subdirectory under /public END_PUBLIC context: One part of isolating the arena usage is maintaining a set of container implementations that we need to know how to allocate. This refactor makes that a bit neater. croupier-relnote: n/a -- 322721429 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Fix out-of-range issues with ConditionalAnd / ConditionalOr inside c++ parser. END_PUBLIC This is the equivalent of cel-expr/cel-go@f5f8bc2 for C++. -- 322681023 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Truncate the number of error messages returned by the parser. For certain inputs, the parser allocates an unbounded amount of memory just for returning error messages. END_PUBLIC This fixes the forge OOM issue. Also refactored errorMessage handling entirely into source_factory where I believe it belongs. -- 322677963 by jimlarson <jimlarson@google.com>: Refactor expression printer into test utilities. BEGIN_PUBLIC Refactor expression printer into test utilities. Also allows expression printer to be immutable and idempotent - it was previously one-shot. END_PUBLIC -- 321841589 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Fix C++ parser crash in ParserVisitor::visitFieldInitializerList. END_PUBLIC -- 321660001 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Fix C++ CEL Parser crash when attempting to report error, and add relevant reproducing test case to unit test. END_PUBLIC -- 321591306 by kuat <kuat@google.com>: BEGIN_PUBLIC Set C++17 compiler option. END_PUBLIC -- 321531133 by CEL Dev Team <cel-dev@google.com>: Add Unrecoverable Unknown (error injection) support to CEL evaluator. BEGIN_PUBLIC Implement unrecoverable unknown support (error injection) in CEL evaluator. Introduce a new field error_paths_ in Activation. Then, in evaluation of Select and Ident expressions, a CelError is returned as a result if error_paths_ contains the current path. Terminal unknowns can be marked by adding paths to this field. A boolean flag InterpreterOptions.enable_error_injection is added to guard this feature. END_PUBLIC go/terminal-unknown RELNOTES: Add Unrecoverable Unknown (error injection) support to CEL evaluator. -- 321009523 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add optional parameter for MissingOverloads error to say which function is missing an overload. END_PUBLIC -- 320984872 by tswadell <tswadell@google.com>: BEGIN_PUBLIC Import of cel-go v0.5.1 from Github(https://github.com/google/cel-go.git). END_PUBLIC Included changes: - 199f791885764c1736178a1132f08706efef8410 Add runtime support for timestamp from Unix epoch (#358) - aa4b2a7d6f496f7fb8642a0719aea7bb2ae28f18 Implement a feature flag mechanism. (#354) - f3a472121b19f1042707024dc5b3283c2e3dfb68 Introduce a sync.Pool for ANTLR Lexer / Parser instances ... - abf60946ee520b24520c843249efc94b7ef378dd Fix namespaced function resolution (#341) - 3023710f92aa1469e7c0734514b080e15fbcb6a5 Fix the doc comments referring to the extant ClearBuiltin... - 623e8a635bf528c113d2592279fb6bc5310a3cb7 Stable sort errors by location (#343) - c4c3df541e541a8cc2fa0b5760fc88cf29f5e58d Introduce decls.NewVar and decls.NewConst (#342) -- 320472813 by CEL Dev Team <cel-dev@google.com>: LSC: Replace the C++ MOCK_METHOD<n> family of macros with the new MOCK_METHOD in third_party/cel. See go/totw/164. BEGIN_PUBLIC Use the new Google Mock MOCK_METHOD macro instead of the older MOCK_METHOD<n> macros. END_PUBLIC #updatemockmethod #codehealth -- 320250626 by mikedanese <mikedanese@google.com>: implement google.protobuf.Any support in cel-cpp The current implementation drops them silently in SetMessage. BEGIN_PUBLIC implement google.protobuf.Any support in cel-cpp END_PUBLIC -- 319801546 by CEL Dev Team <cel-dev@google.com>: Prototype for IndexMetadataExpressionEvaluator. This is a class that takes a set of go/cel expressions (one for topic-level filtering, another for sp-level) and executes them on a MultiMDU representing a topic. The assumption is that each MDU has an IndexMetadata proto bound to the userdata field. BEGIN_PUBLIC Internal change END_PUBLIC -- 319276695 by CEL Dev Team <cel-dev@google.com>: This CL introduces a view layer for pToken attributes (for scalar pToken, context, operation), which would mix derived & basic attributes together. It doesn't require pre-materialization of attributes in a container, proto, etc. Attributes are typed using the CEL values, with a CEL-less type-safe wrapper layer provided in typed_views.h Views are provided over various protocol-buffer based representations, and support boxed-types and derived attribute staleness. More view implementations would expose a derived attribute cache, SecurityContext, and so on. Basic attributes currently present in PTokenInternal are added to the PTokenAttributes registry so that they can be referenced using the type-safe lookups in typed_views.h Tested: Unit tests are added for the new classes. No existing code is changed. BEGIN_PUBLIC internal change END_PUBLIC -- 319266365 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add helper for creating CelAttributePatterns. END_PUBLIC -- 318192649 by CEL Dev Team <cel-dev@google.com>: This generated CL removes comments next to a licenses() rule in the BUILD file. It's desirable to remove these comments as they are easy to fall out of sync with the LICENSE file causing confusion. -- 317682809 by kuat <kuat@google.com>: BEGIN_PUBLIC Remove superfluous include. END_PUBLIC -- 316972268 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add basic gtest matchers for cel values. END_PUBLIC -- 316850412 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Internal change END_PUBLIC Make third_party/cel/cpp/parser:parer library visible to //privacy/ptoken/extraction -- 316223239 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Internal change END_PUBLIC Make third_party/cel/cpp/parser:parer library visible to //video/youtube/abuse/enforcement/protection -- 315923824 by kmilsht <kmilsht@google.com>: BEGIN_PUBLIC Fixing null pointer situation when using dynamically constructed message of well-known types (google.protobuf) with CelValue::CreateMessage END_PUBLIC -- 315271343 by CEL Dev Team <cel-dev@google.com>: LSC: Replace cord.ToString() with std::string(cord) This method is deprecated and will be removed. LSC document: go/lsc-absl-cord-members Bug: b/157745958 http://refactor/4636887511536581949 Master CL created by go/Zwingli. Tested: tap_presubmit: http://test/OCL:314910097:BASE:314916222:1591474813453:da5b614c Some tests failed; test failures are believed to be unrelated to this CL -- 315006421 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add attr trail to iter_var. This should allow unknown paths to be maintained in comprehensions. END_PUBLIC modified rollback of cl/302120264 -- 314991879 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Add debugging utility for printing cel values. END_PUBLIC -- 314609599 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Move ternary builtin to a custom step instead of a built-in function. This helps make the special error handling semantics consistent with shortcutting on or off. END_PUBLIC -- 314566279 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Use the logical step instead of built-in functions for handling or/and. This makes it a bit easier to handle special processing for errors and unknowns. END_PUBLIC -- 314232099 by CEL Dev Team <cel-dev@google.com>: Implements support for using the CEL parser in third_party in unit tests instead of creating CEL ASTs through helper functions. Updated a first round of unit tests to use the parser and removed the helper functions. BEGIN_PUBLIC BUILD file visibility changes END_PUBLIC -- 311339636 by CEL Dev Team <cel-dev@google.com>: BEGIN_PUBLIC Internal change END_PUBLIC Make third_party/cel/cpp/parser:parer library visible to //cloud/ml/api/conversation/service/handler/cobweb. -- 311335037 by timdn <timdn@google.com>: BEGIN_PUBLIC Add CEL multi-threaded parsing test On the Open Source version, this test breaks with Antlr 4.7.1, while it suceeds with 4.7.2, which has received thread-safety improvements (breakage rate was beyond 95% during my tests). END_PUBLIC On the internal version, this also works on 4.7.1, probably due to some sanitization during builds. However, an upgrade to 4.7.2 seems advisable. PiperOrigin-RevId: 326520769

Uh oh!

Commits

Commits on Aug 13, 2020

Export of internal changes