Skip to content

Initial implementation of opt-out for IAST - #5713

Closed
manuel-alvarez-alvarez wants to merge 1 commit into
masterfrom
malvarez/iast-opt-out
Closed

Initial implementation of opt-out for IAST#5713
manuel-alvarez-alvarez wants to merge 1 commit into
masterfrom
malvarez/iast-opt-out

Conversation

@manuel-alvarez-alvarez

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Aug 12, 2023

Copy link
Copy Markdown
Member

What Does This Do

Motivation

Additional Notes

@manuel-alvarez-alvarez manuel-alvarez-alvarez added tag: no release notes Changes to exclude from release notes comp: asm iast Application Security Management (IAST) labels Aug 12, 2023
@manuel-alvarez-alvarez
manuel-alvarez-alvarez force-pushed the malvarez/iast-opt-out branch 2 times, most recently from 285cba0 to ae49143 Compare August 13, 2023 09:59
@manuel-alvarez-alvarez
manuel-alvarez-alvarez changed the base branch from master to malvarez/fix-servlet-response-inst August 13, 2023 10:00
@manuel-alvarez-alvarez
manuel-alvarez-alvarez force-pushed the malvarez/fix-servlet-response-inst branch from 1a735b8 to 60353d3 Compare August 13, 2023 11:18
Base automatically changed from malvarez/fix-servlet-response-inst to master August 14, 2023 08:45
@manuel-alvarez-alvarez
manuel-alvarez-alvarez force-pushed the malvarez/iast-opt-out branch 2 times, most recently from 0909806 to 4305eda Compare August 14, 2023 15:35
@pr-commenter

pr-commenter Bot commented Aug 14, 2023 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~dbf342291c 1.20.0-SNAPSHOT~4d5358abad
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 66 cases.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:insecure-bank:iast_OPT_OUT:GlobalTracer better
[-22.106ms; -9.485ms] or [-7.671%; -3.291%]		 272.375ms 288.171ms
scenario:insecure-bank:iast_OPT_OUT:Remote Config better
[-60.422µs; -25.213µs] or [-8.936%; -3.729%]		 633.315µs 676.133µs
scenario:petclinic:iast:IAST worse
[+459.438µs; +824.173µs] or [+3.306%; +5.930%]		 14.541ms 13.899ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~4d5358abad, baseline=1.20.0-SNAPSHOT~dbf342291c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.007 s) : 0, 1007326
Total [baseline] (8.726 s) : 0, 8726014
Agent [candidate] (994.541 ms) : 0, 994541
Total [candidate] (8.64 s) : 0, 8639678
section appsec
Agent [baseline] (1.071 s) : 0, 1071302
Total [baseline] (8.741 s) : 0, 8741332
Agent [candidate] (1.071 s) : 0, 1071239
Total [candidate] (8.685 s) : 0, 8685476
section iast
Agent [baseline] (1.096 s) : 0, 1096196
Total [baseline] (9.2 s) : 0, 9200072
Agent [candidate] (1.098 s) : 0, 1098194
Total [candidate] (9.173 s) : 0, 9173031
section iast_OPT_OUT
Agent [baseline] (1.001 s) : 0, 1000543
Total [baseline] (8.658 s) : 0, 8657576
Agent [candidate] (998.991 ms) : 0, 998991
Total [candidate] (8.66 s) : 0, 8659555
section profiling
Agent [baseline] (1.155 s) : 0, 1155193
Total [baseline] (8.868 s) : 0, 8867821
Agent [candidate] (1.161 s) : 0, 1160668
Total [candidate] (8.883 s) : 0, 8883297
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.007 s -
Agent appsec 1.071 s 63.976 ms (6.4%)
Agent iast 1.096 s 88.87 ms (8.8%)
Agent iast_OPT_OUT 1.001 s -6.784 ms (-0.7%)
Agent profiling 1.155 s 147.867 ms (14.7%)
Total tracing 8.726 s -
Total appsec 8.741 s 15.318 ms (0.2%)
Total iast 9.2 s 474.058 ms (5.4%)
Total iast_OPT_OUT 8.658 s -68.438 ms (-0.8%)
Total profiling 8.868 s 141.807 ms (1.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 994.541 ms -
Agent appsec 1.071 s 76.697 ms (7.7%)
Agent iast 1.098 s 103.653 ms (10.4%)
Agent iast_OPT_OUT 998.991 ms 4.449 ms (0.4%)
Agent profiling 1.161 s 166.127 ms (16.7%)
Total tracing 8.64 s -
Total appsec 8.685 s 45.799 ms (0.5%)
Total iast 9.173 s 533.353 ms (6.2%)
Total iast_OPT_OUT 8.66 s 19.877 ms (0.2%)
Total profiling 8.883 s 243.62 ms (2.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~4d5358abad, baseline=1.20.0-SNAPSHOT~dbf342291c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (624.675 ms) : 0, 624675
BytebuddyAgent [candidate] (615.647 ms) : 0, 615647
GlobalTracer [baseline] (290.437 ms) : 0, 290437
GlobalTracer [candidate] (286.937 ms) : 0, 286937
AppSec [baseline] (50.439 ms) : 0, 50439
AppSec [candidate] (50.718 ms) : 0, 50718
Remote Config [baseline] (676.993 µs) : 0, 677
Remote Config [candidate] (673.045 µs) : 0, 673
Telemetry [baseline] (6.096 ms) : 0, 6096
Telemetry [candidate] (6.039 ms) : 0, 6039
section appsec
BytebuddyAgent [baseline] (611.118 ms) : 0, 611118
BytebuddyAgent [candidate] (611.021 ms) : 0, 611021
GlobalTracer [baseline] (283.685 ms) : 0, 283685
GlobalTracer [candidate] (284.313 ms) : 0, 284313
AppSec [baseline] (135.732 ms) : 0, 135732
AppSec [candidate] (135.279 ms) : 0, 135279
Remote Config [baseline] (726.771 µs) : 0, 727
Remote Config [candidate] (717.811 µs) : 0, 718
Telemetry [baseline] (5.772 ms) : 0, 5772
Telemetry [candidate] (5.761 ms) : 0, 5761
section iast
BytebuddyAgent [baseline] (722.059 ms) : 0, 722059
BytebuddyAgent [candidate] (723.279 ms) : 0, 723279
GlobalTracer [baseline] (272.04 ms) : 0, 272040
GlobalTracer [candidate] (271.93 ms) : 0, 271930
AppSec [baseline] (47.324 ms) : 0, 47324
AppSec [candidate] (47.53 ms) : 0, 47530
Remote Config [baseline] (555.373 µs) : 0, 555
Remote Config [candidate] (556.38 µs) : 0, 556
Telemetry [baseline] (5.781 ms) : 0, 5781
Telemetry [candidate] (5.728 ms) : 0, 5728
IAST [baseline] (13.945 ms) : 0, 13945
IAST [candidate] (14.552 ms) : 0, 14552
section iast_OPT_OUT
BytebuddyAgent [baseline] (620.336 ms) : 0, 620336
BytebuddyAgent [candidate] (624.553 ms) : 0, 624553
GlobalTracer [baseline] (288.171 ms) : 0, 288171
GlobalTracer [candidate] (272.375 ms) : 0, 272375
AppSec [baseline] (50.403 ms) : 0, 50403
AppSec [candidate] (50.424 ms) : 0, 50424
Remote Config [baseline] (676.133 µs) : 0, 676
Remote Config [candidate] (633.315 µs) : 0, 633
Telemetry [baseline] (6.088 ms) : 0, 6088
Telemetry [candidate] (6.364 ms) : 0, 6364
IAST [candidate] (10.296 ms) : 0, 10296
section profiling
ProfilingAgent [baseline] (73.931 ms) : 0, 73931
ProfilingAgent [candidate] (75.452 ms) : 0, 75452
BytebuddyAgent [baseline] (623.765 ms) : 0, 623765
BytebuddyAgent [candidate] (626.785 ms) : 0, 626785
GlobalTracer [baseline] (348.996 ms) : 0, 348996
GlobalTracer [candidate] (349.303 ms) : 0, 349303
AppSec [baseline] (49.032 ms) : 0, 49032
AppSec [candidate] (49.241 ms) : 0, 49241
Remote Config [baseline] (642.957 µs) : 0, 643
Remote Config [candidate] (640.04 µs) : 0, 640
Telemetry [baseline] (5.96 ms) : 0, 5960
Telemetry [candidate] (5.933 ms) : 0, 5933
Profiling [baseline] (73.957 ms) : 0, 73957
Profiling [candidate] (75.476 ms) : 0, 75476
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~4d5358abad, baseline=1.20.0-SNAPSHOT~dbf342291c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.006 s) : 0, 1006230
Total [baseline] (9.273 s) : 0, 9272831
Agent [candidate] (988.96 ms) : 0, 988960
Total [candidate] (9.179 s) : 0, 9179026
section appsec
Agent [baseline] (1.07 s) : 0, 1069913
Total [baseline] (9.239 s) : 0, 9238585
Agent [candidate] (1.077 s) : 0, 1077217
Total [candidate] (9.279 s) : 0, 9278541
section iast
Agent [baseline] (1.09 s) : 0, 1089716
Total [baseline] (9.4 s) : 0, 9399637
Agent [candidate] (1.092 s) : 0, 1092070
Total [candidate] (9.438 s) : 0, 9438028
section profiling
Agent [baseline] (1.155 s) : 0, 1154893
Total [baseline] (9.445 s) : 0, 9445283
Agent [candidate] (1.158 s) : 0, 1158174
Total [candidate] (9.393 s) : 0, 9393002
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.006 s -
Agent appsec 1.07 s 63.682 ms (6.3%)
Agent iast 1.09 s 83.485 ms (8.3%)
Agent profiling 1.155 s 148.663 ms (14.8%)
Total tracing 9.273 s -
Total appsec 9.239 s -34.246 ms (-0.4%)
Total iast 9.4 s 126.806 ms (1.4%)
Total profiling 9.445 s 172.452 ms (1.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 988.96 ms -
Agent appsec 1.077 s 88.257 ms (8.9%)
Agent iast 1.092 s 103.11 ms (10.4%)
Agent profiling 1.158 s 169.214 ms (17.1%)
Total tracing 9.179 s -
Total appsec 9.279 s 99.515 ms (1.1%)
Total iast 9.438 s 259.002 ms (2.8%)
Total profiling 9.393 s 213.975 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~4d5358abad, baseline=1.20.0-SNAPSHOT~dbf342291c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (623.812 ms) : 0, 623812
BytebuddyAgent [candidate] (612.603 ms) : 0, 612603
GlobalTracer [baseline] (290.339 ms) : 0, 290339
GlobalTracer [candidate] (285.173 ms) : 0, 285173
AppSec [baseline] (50.41 ms) : 0, 50410
AppSec [candidate] (50.136 ms) : 0, 50136
Remote Config [baseline] (677.096 µs) : 0, 677
Remote Config [candidate] (669.503 µs) : 0, 670
Telemetry [baseline] (6.083 ms) : 0, 6083
Telemetry [candidate] (6.007 ms) : 0, 6007
section appsec
BytebuddyAgent [baseline] (610.202 ms) : 0, 610202
BytebuddyAgent [candidate] (613.457 ms) : 0, 613457
GlobalTracer [baseline] (283.597 ms) : 0, 283597
GlobalTracer [candidate] (287.537 ms) : 0, 287537
AppSec [baseline] (135.469 ms) : 0, 135469
AppSec [candidate] (135.404 ms) : 0, 135404
Remote Config [baseline] (723.873 µs) : 0, 724
Remote Config [candidate] (728.698 µs) : 0, 729
Telemetry [baseline] (5.769 ms) : 0, 5769
Telemetry [candidate] (5.851 ms) : 0, 5851
section iast
BytebuddyAgent [baseline] (717.46 ms) : 0, 717460
BytebuddyAgent [candidate] (718.773 ms) : 0, 718773
GlobalTracer [baseline] (270.107 ms) : 0, 270107
GlobalTracer [candidate] (270.702 ms) : 0, 270702
AppSec [baseline] (47.542 ms) : 0, 47542
AppSec [candidate] (47.542 ms) : 0, 47542
Remote Config [baseline] (562.74 µs) : 0, 563
Remote Config [candidate] (551.908 µs) : 0, 552
Telemetry [baseline] (5.763 ms) : 0, 5763
Telemetry [candidate] (5.703 ms) : 0, 5703
IAST [baseline] (13.899 ms) : 0, 13899
IAST [candidate] (14.541 ms) : 0, 14541
section profiling
ProfilingAgent [baseline] (74.739 ms) : 0, 74739
ProfilingAgent [candidate] (76.037 ms) : 0, 76037
BytebuddyAgent [baseline] (623.524 ms) : 0, 623524
BytebuddyAgent [candidate] (623.753 ms) : 0, 623753
GlobalTracer [baseline] (348.198 ms) : 0, 348198
GlobalTracer [candidate] (349.629 ms) : 0, 349629
AppSec [baseline] (49.007 ms) : 0, 49007
AppSec [candidate] (49.185 ms) : 0, 49185
Remote Config [baseline] (639.691 µs) : 0, 640
Remote Config [candidate] (646.859 µs) : 0, 647
Telemetry [baseline] (5.973 ms) : 0, 5973
Telemetry [candidate] (5.943 ms) : 0, 5943
Profiling [baseline] (74.763 ms) : 0, 74763
Profiling [candidate] (76.063 ms) : 0, 76063
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~dbf342291c 1.20.0-SNAPSHOT~4d5358abad
config baseline candidate
end_time 2023-08-21T07:45:35 2023-08-21T08:05:30
start_time 2023-08-21T07:45:18 2023-08-21T08:05:12
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 26 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~4d5358abad, baseline=1.20.0-SNAPSHOT~dbf342291c
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.133 µs) : 340, 380
.   : milestone, 360,
appsec (652.725 µs) : 633, 673
.   : milestone, 653,
iast (456.226 µs) : 435, 477
.   : milestone, 456,
iast_FULL (522.876 µs) : 502, 544
.   : milestone, 523,
iast_INACTIVE (432.801 µs) : 412, 454
.   : milestone, 433,
iast_OPT_OUT (436.165 µs) : 415, 457
.   : milestone, 436,
profiling (434.64 µs) : 414, 455
.   : milestone, 435,
tracing (434.585 µs) : 414, 455
.   : milestone, 435,
section candidate
no_agent (359.356 µs) : 339, 380
.   : milestone, 359,
appsec (661.184 µs) : 640, 682
.   : milestone, 661,
iast (452.31 µs) : 431, 473
.   : milestone, 452,
iast_FULL (523.915 µs) : 503, 545
.   : milestone, 524,
iast_INACTIVE (436.041 µs) : 416, 456
.   : milestone, 436,
iast_OPT_OUT (435.566 µs) : 415, 456
.   : milestone, 436,
profiling (429.823 µs) : 409, 450
.   : milestone, 430,
tracing (428.957 µs) : 409, 449
.   : milestone, 429,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.133 µs [340.175 µs, 380.092 µs] -
appsec 652.725 µs [632.522 µs, 672.929 µs] 292.592 µs (81.2%)
iast 456.226 µs [435.304 µs, 477.148 µs] 96.092 µs (26.7%)
iast_FULL 522.876 µs [502.14 µs, 543.611 µs] 162.742 µs (45.2%)
iast_INACTIVE 432.801 µs [411.929 µs, 453.673 µs] 72.668 µs (20.2%)
iast_OPT_OUT 436.165 µs [415.11 µs, 457.22 µs] 76.032 µs (21.1%)
profiling 434.64 µs [414.036 µs, 455.244 µs] 74.507 µs (20.7%)
tracing 434.585 µs [413.928 µs, 455.243 µs] 74.452 µs (20.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.356 µs [338.95 µs, 379.763 µs] -
appsec 661.184 µs [640.358 µs, 682.01 µs] 301.828 µs (84.0%)
iast 452.31 µs [431.443 µs, 473.177 µs] 92.953 µs (25.9%)
iast_FULL 523.915 µs [503.03 µs, 544.8 µs] 164.559 µs (45.8%)
iast_INACTIVE 436.041 µs [415.674 µs, 456.408 µs] 76.685 µs (21.3%)
iast_OPT_OUT 435.566 µs [415.036 µs, 456.095 µs] 76.209 µs (21.2%)
profiling 429.823 µs [409.283 µs, 450.363 µs] 70.467 µs (19.6%)
tracing 428.957 µs [408.75 µs, 449.164 µs] 69.601 µs (19.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~4d5358abad, baseline=1.20.0-SNAPSHOT~dbf342291c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.312 ms) : 1292, 1331
.   : milestone, 1312,
appsec (1.672 ms) : 1648, 1697
.   : milestone, 1672,
iast (1.457 ms) : 1433, 1481
.   : milestone, 1457,
profiling (1.478 ms) : 1454, 1503
.   : milestone, 1478,
tracing (1.475 ms) : 1451, 1499
.   : milestone, 1475,
section candidate
no_agent (1.339 ms) : 1320, 1358
.   : milestone, 1339,
appsec (1.664 ms) : 1640, 1688
.   : milestone, 1664,
iast (1.453 ms) : 1429, 1478
.   : milestone, 1453,
profiling (1.476 ms) : 1451, 1502
.   : milestone, 1476,
tracing (1.435 ms) : 1411, 1459
.   : milestone, 1435,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.312 ms [1.292 ms, 1.331 ms] -
appsec 1.672 ms [1.648 ms, 1.697 ms] 360.386 µs (27.5%)
iast 1.457 ms [1.433 ms, 1.481 ms] 145.49 µs (11.1%)
profiling 1.478 ms [1.454 ms, 1.503 ms] 166.475 µs (12.7%)
tracing 1.475 ms [1.451 ms, 1.499 ms] 163.494 µs (12.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.339 ms [1.32 ms, 1.358 ms] -
appsec 1.664 ms [1.64 ms, 1.688 ms] 325.21 µs (24.3%)
iast 1.453 ms [1.429 ms, 1.478 ms] 114.426 µs (8.5%)
profiling 1.476 ms [1.451 ms, 1.502 ms] 137.178 µs (10.2%)
tracing 1.435 ms [1.411 ms, 1.459 ms] 95.862 µs (7.2%)

@smola smola removed the tag: no release notes Changes to exclude from release notes label Aug 14, 2023
@manuel-alvarez-alvarez

manuel-alvarez-alvarez commented Feb 5, 2024

Copy link
Copy Markdown
Member Author

Deprecated by #6621

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp: asm iast Application Security Management (IAST)

Projects

None yet

Milestone

No milestone

Development

See More

Successfully merging this pull request may close these issues.

2 participants

@manuel-alvarez-alvarez @smola