Open
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1061843
Total [baseline] (8.733 s) : 0, 8732647
Agent [candidate] (1.072 s) : 0, 1072362
Total [candidate] (8.742 s) : 0, 8742277
section iast
Agent [baseline] (1.23 s) : 0, 1229614
Total [baseline] (9.4 s) : 0, 9400122
Agent [candidate] (1.239 s) : 0, 1238575
Total [candidate] (9.379 s) : 0, 9378932
gantt
title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.226 ms) : 0, 1226
BytebuddyAgent [baseline] (626.51 ms) : 0, 626510
BytebuddyAgent [candidate] (632.234 ms) : 0, 632234
AgentMeter [baseline] (28.953 ms) : 0, 28953
AgentMeter [candidate] (29.321 ms) : 0, 29321
GlobalTracer [baseline] (257.099 ms) : 0, 257099
GlobalTracer [candidate] (259.422 ms) : 0, 259422
AppSec [baseline] (32.716 ms) : 0, 32716
AppSec [candidate] (33.016 ms) : 0, 33016
Debugger [baseline] (61.954 ms) : 0, 61954
Debugger [candidate] (64.666 ms) : 0, 64666
Remote Config [baseline] (624.683 µs) : 0, 625
Remote Config [candidate] (624.549 µs) : 0, 625
Telemetry [baseline] (12.162 ms) : 0, 12162
Telemetry [candidate] (9.241 ms) : 0, 9241
Flare Poller [baseline] (4.509 ms) : 0, 4509
Flare Poller [candidate] (6.28 ms) : 0, 6280
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (794.424 ms) : 0, 794424
BytebuddyAgent [candidate] (801.28 ms) : 0, 801280
AgentMeter [baseline] (11.267 ms) : 0, 11267
AgentMeter [candidate] (11.606 ms) : 0, 11606
GlobalTracer [baseline] (247.387 ms) : 0, 247387
GlobalTracer [candidate] (248.943 ms) : 0, 248943
AppSec [baseline] (31.502 ms) : 0, 31502
AppSec [candidate] (34.713 ms) : 0, 34713
Debugger [baseline] (68.174 ms) : 0, 68174
Debugger [candidate] (64.768 ms) : 0, 64768
Remote Config [baseline] (537.813 µs) : 0, 538
Remote Config [candidate] (545.319 µs) : 0, 545
Telemetry [baseline] (8.581 ms) : 0, 8581
Telemetry [candidate] (8.674 ms) : 0, 8674
Flare Poller [baseline] (3.462 ms) : 0, 3462
Flare Poller [candidate] (3.47 ms) : 0, 3470
IAST [baseline] (27.145 ms) : 0, 27145
IAST [candidate] (27.254 ms) : 0, 27254
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1071141
Total [baseline] (10.879 s) : 0, 10879044
Agent [candidate] (1.072 s) : 0, 1071726
Total [candidate] (10.915 s) : 0, 10915073
section appsec
Agent [baseline] (1.244 s) : 0, 1243982
Total [baseline] (10.998 s) : 0, 10998075
Agent [candidate] (1.241 s) : 0, 1240724
Total [candidate] (10.969 s) : 0, 10968868
section iast
Agent [baseline] (1.239 s) : 0, 1238565
Total [baseline] (11.22 s) : 0, 11219922
Agent [candidate] (1.237 s) : 0, 1237439
Total [candidate] (11.244 s) : 0, 11243740
section profiling
Agent [baseline] (1.195 s) : 0, 1194700
Total [baseline] (10.91 s) : 0, 10909550
Agent [candidate] (1.198 s) : 0, 1197576
Total [candidate] (11.078 s) : 0, 11077816
gantt
title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (631.721 ms) : 0, 631721
BytebuddyAgent [candidate] (632.767 ms) : 0, 632767
AgentMeter [baseline] (29.321 ms) : 0, 29321
AgentMeter [candidate] (29.195 ms) : 0, 29195
GlobalTracer [baseline] (259.052 ms) : 0, 259052
GlobalTracer [candidate] (258.845 ms) : 0, 258845
AppSec [baseline] (33.374 ms) : 0, 33374
AppSec [candidate] (33.172 ms) : 0, 33172
Debugger [baseline] (64.463 ms) : 0, 64463
Debugger [candidate] (65.515 ms) : 0, 65515
Remote Config [baseline] (618.784 µs) : 0, 619
Remote Config [candidate] (605.775 µs) : 0, 606
Telemetry [baseline] (10.775 ms) : 0, 10775
Telemetry [candidate] (9.035 ms) : 0, 9035
Flare Poller [baseline] (4.522 ms) : 0, 4522
Flare Poller [candidate] (5.306 ms) : 0, 5306
section appsec
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (661.556 ms) : 0, 661556
BytebuddyAgent [candidate] (658.39 ms) : 0, 658390
AgentMeter [baseline] (12.0 ms) : 0, 12000
AgentMeter [candidate] (11.975 ms) : 0, 11975
GlobalTracer [baseline] (258.302 ms) : 0, 258302
GlobalTracer [candidate] (258.411 ms) : 0, 258411
AppSec [baseline] (168.112 ms) : 0, 168112
AppSec [candidate] (168.019 ms) : 0, 168019
Debugger [baseline] (67.2 ms) : 0, 67200
Debugger [candidate] (67.274 ms) : 0, 67274
Remote Config [baseline] (644.596 µs) : 0, 645
Remote Config [candidate] (661.532 µs) : 0, 662
Telemetry [baseline] (9.643 ms) : 0, 9643
Telemetry [candidate] (9.494 ms) : 0, 9494
Flare Poller [baseline] (3.781 ms) : 0, 3781
Flare Poller [candidate] (3.764 ms) : 0, 3764
IAST [baseline] (25.394 ms) : 0, 25394
IAST [candidate] (25.456 ms) : 0, 25456
section iast
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (799.887 ms) : 0, 799887
BytebuddyAgent [candidate] (798.57 ms) : 0, 798570
AgentMeter [baseline] (11.546 ms) : 0, 11546
AgentMeter [candidate] (11.382 ms) : 0, 11382
GlobalTracer [baseline] (248.922 ms) : 0, 248922
GlobalTracer [candidate] (248.867 ms) : 0, 248867
AppSec [baseline] (33.143 ms) : 0, 33143
AppSec [candidate] (34.259 ms) : 0, 34259
Debugger [baseline] (67.589 ms) : 0, 67589
Debugger [candidate] (67.018 ms) : 0, 67018
Remote Config [baseline] (537.254 µs) : 0, 537
Remote Config [candidate] (542.937 µs) : 0, 543
Telemetry [baseline] (8.747 ms) : 0, 8747
Telemetry [candidate] (8.722 ms) : 0, 8722
Flare Poller [baseline] (3.503 ms) : 0, 3503
Flare Poller [candidate] (3.508 ms) : 0, 3508
IAST [baseline] (27.34 ms) : 0, 27340
IAST [candidate] (27.333 ms) : 0, 27333
section profiling
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (685.393 ms) : 0, 685393
BytebuddyAgent [candidate] (686.119 ms) : 0, 686119
AgentMeter [baseline] (8.533 ms) : 0, 8533
AgentMeter [candidate] (8.62 ms) : 0, 8620
GlobalTracer [baseline] (216.171 ms) : 0, 216171
GlobalTracer [candidate] (217.119 ms) : 0, 217119
AppSec [baseline] (32.695 ms) : 0, 32695
AppSec [candidate] (32.695 ms) : 0, 32695
Debugger [baseline] (67.022 ms) : 0, 67022
Debugger [candidate] (67.509 ms) : 0, 67509
Remote Config [baseline] (626.54 µs) : 0, 627
Remote Config [candidate] (655.529 µs) : 0, 656
Telemetry [baseline] (9.105 ms) : 0, 9105
Telemetry [candidate] (9.156 ms) : 0, 9156
Flare Poller [baseline] (3.747 ms) : 0, 3747
Flare Poller [candidate] (3.843 ms) : 0, 3843
ProfilingAgent [baseline] (99.162 ms) : 0, 99162
ProfilingAgent [candidate] (99.843 ms) : 0, 99843
Profiling [baseline] (99.769 ms) : 0, 99769
Profiling [candidate] (100.417 ms) : 0, 100417
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 3 performance regressions! Performance is the same for 16 metrics, 17 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section baseline
no_agent (19.156 ms) : 18958, 19354
. : milestone, 19156,
appsec (18.657 ms) : 18467, 18847
. : milestone, 18657,
code_origins (17.632 ms) : 17458, 17807
. : milestone, 17632,
iast (17.402 ms) : 17228, 17576
. : milestone, 17402,
profiling (18.835 ms) : 18645, 19025
. : milestone, 18835,
tracing (17.438 ms) : 17266, 17610
. : milestone, 17438,
section candidate
no_agent (19.101 ms) : 18907, 19295
. : milestone, 19101,
appsec (18.823 ms) : 18629, 19017
. : milestone, 18823,
code_origins (17.5 ms) : 17328, 17671
. : milestone, 17500,
iast (17.595 ms) : 17418, 17771
. : milestone, 17595,
profiling (19.913 ms) : 19713, 20114
. : milestone, 19913,
tracing (17.671 ms) : 17498, 17843
. : milestone, 17671,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section baseline
no_agent (1.197 ms) : 1185, 1210
. : milestone, 1197,
iast (3.062 ms) : 3023, 3101
. : milestone, 3062,
iast_FULL (5.764 ms) : 5706, 5821
. : milestone, 5764,
iast_GLOBAL (3.556 ms) : 3498, 3614
. : milestone, 3556,
profiling (2.179 ms) : 2158, 2200
. : milestone, 2179,
tracing (1.841 ms) : 1826, 1856
. : milestone, 1841,
section candidate
no_agent (1.185 ms) : 1174, 1197
. : milestone, 1185,
iast (3.31 ms) : 3263, 3357
. : milestone, 3310,
iast_FULL (5.895 ms) : 5836, 5954
. : milestone, 5895,
iast_GLOBAL (3.512 ms) : 3452, 3571
. : milestone, 3512,
profiling (2.001 ms) : 1982, 2019
. : milestone, 2001,
tracing (1.88 ms) : 1863, 1897
. : milestone, 1880,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
. : milestone, 1480,
appsec (2.537 ms) : 2481, 2592
. : milestone, 2537,
iast (2.261 ms) : 2192, 2330
. : milestone, 2261,
iast_GLOBAL (2.318 ms) : 2248, 2388
. : milestone, 2318,
profiling (2.117 ms) : 2060, 2173
. : milestone, 2117,
tracing (2.088 ms) : 2034, 2143
. : milestone, 2088,
section candidate
no_agent (1.482 ms) : 1470, 1493
. : milestone, 1482,
appsec (3.797 ms) : 3575, 4018
. : milestone, 3797,
iast (2.266 ms) : 2196, 2335
. : milestone, 2266,
iast_GLOBAL (2.314 ms) : 2244, 2384
. : milestone, 2314,
profiling (2.085 ms) : 2030, 2139
. : milestone, 2085,
tracing (2.081 ms) : 2027, 2135
. : milestone, 2081,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~4b9b2257f8, baseline=1.60.0-SNAPSHOT~daf2c01407
dateFormat X
axisFormat %s
section baseline
no_agent (15.208 s) : 15208000, 15208000
. : milestone, 15208000,
appsec (15.036 s) : 15036000, 15036000
. : milestone, 15036000,
iast (18.206 s) : 18206000, 18206000
. : milestone, 18206000,
iast_GLOBAL (17.913 s) : 17913000, 17913000
. : milestone, 17913000,
profiling (14.754 s) : 14754000, 14754000
. : milestone, 14754000,
tracing (14.833 s) : 14833000, 14833000
. : milestone, 14833000,
section candidate
no_agent (15.236 s) : 15236000, 15236000
. : milestone, 15236000,
appsec (14.981 s) : 14981000, 14981000
. : milestone, 14981000,
iast (18.176 s) : 18176000, 18176000
. : milestone, 18176000,
iast_GLOBAL (17.83 s) : 17830000, 17830000
. : milestone, 17830000,
profiling (15.472 s) : 15472000, 15472000
. : milestone, 15472000,
tracing (15.09 s) : 15090000, 15090000
. : milestone, 15090000,
|
449dd10 to
8981bb1
Compare
8981bb1 to
74326e0
Compare
c227a36 to
f7d45ac
Compare
Contributor
pawel-big-lebowski
left a comment
There was a problem hiding this comment.
Pushing first round of comments.
Main concern: do we have to call advice from within an advice?
| Thread.currentThread() | ||
| .getContextClassLoader() | ||
| .loadClass("datadog.trace.instrumentation.spark.SparkLauncherAdvice"); | ||
| Method finishMethod = adviceClass.getDeclaredMethod("finishLauncherSpan", int.class); |
Contributor
There was a problem hiding this comment.
Why do we need reflection to call this? SparkLauncherAdvice is a class we can modify to expose methods required. Also, I don't think that calling advice from advice is a recommended pattern.
Pls explain why relying on AppHandleListener aint good enough?
| Class<?> adviceClass = | ||
| Thread.currentThread() | ||
| .getContextClassLoader() | ||
| .loadClass("datadog.trace.instrumentation.spark.SparkLauncherAdvice"); |
Contributor
There was a problem hiding this comment.
Would it make sense to have separate SparkLauncherSpanBuilder class instead of calling advice from within the advice?
Contributor
Author
There was a problem hiding this comment.
The class is still quite small and we are sure to only emit 1 span from here (will never get child), so for now I'd push to keep it as is and we can revisit next week/next time we have to add code to this class
| Map<String, String> conf = (Map<String, String>) confField.get(builder); | ||
| if (conf != null) { | ||
| for (Map.Entry<String, String> entry : conf.entrySet()) { | ||
| if (SparkConfAllowList.canCaptureJobParameter(entry.getKey())) { |
Contributor
There was a problem hiding this comment.
Can't we use datadog.trace.instrumentation.spark.SparkConfAllowList#getRedactedSparkConf same way datadog.trace.instrumentation.spark.AbstractDatadogSparkListener#captureJobParameters ?
| } | ||
| } | ||
|
|
||
| public static synchronized void createLauncherSpan(String resource, Object launcher) { |
Contributor
There was a problem hiding this comment.
Is public necessary?
Contributor
Author
There was a problem hiding this comment.
yes we need it for real I've hit a lot of IllegalAccessError.
When the advice calls createLauncherSpan() or new AppHandleListener(), that code is actually inside SparkLauncher which is in a different package than datadog.trace.instrumentation.spark. When those methods were private, I had IllegalAccessError and no span emitted
| span.setError(true); | ||
| span.setTag(DDTags.ERROR_TYPE, "Spark Application " + state); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Isn't this a right place to finishLauncherSpan instead of relying on SparkExitAdvice?
| private static final Logger log = LoggerFactory.getLogger(SparkLauncherAdvice.class); | ||
|
|
||
| // Same default pattern as spark.redaction.regex in Spark source | ||
| private static final Pattern CONF_REDACTION_PATTERN = |
Contributor
There was a problem hiding this comment.
If you add SparkConfAllowList as helper class in advice, you should be able to use its redaction methods.
Contributor
Author
There was a problem hiding this comment.
Thanks, makes sense addressed !
390dafa to
559ce6c
Compare
f4c59f3 to
f57bf18
Compare
Contributor
pawel-big-lebowski
left a comment
There was a problem hiding this comment.
Keeping the code within a single advice makes it more readable and makes the outcome easier to predict. Thanks for making this change.
| } | ||
| } | ||
|
|
||
| public static class AppHandleListener implements SparkAppHandle.Listener { |
Contributor
There was a problem hiding this comment.
SparkLauncherAdvice currently has multiple responsibilities and several static methods. It might be cleaner to extract AppHandleListener into its own class and make AgentSpan launcherSpan a member there.
This would improve separation of concerns: the advice would only inject/register the listener, and the listener would handle creating and emitting the launcher spans.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Motivation
Get spark.launcher.launch spans for SparkAppHandler to monitor the launcher that can fail independently of the app it starts.
Span example: https://ddstaging.datadoghq.com/apm/traces?query=job_flow_id%3A%2A&agg_m=count&agg_m_source=base&agg_t=count&cols=core_service%2Ccore_resource_name%2Clog_duration%2Clog_http.method%2Clog_http.status_code&fromUser=false&graphType=flamegraph&historicalData=true&messageDisplay=inline&query_translation_version=v0&shouldShowLegend=true&sort=desc&spanID=6071379317785568039&spanType=all&spanViewType=metadata&sparkMetricsSections=io%2Cmemory%2CcpuTable&storage=hot&timeHint=1771578766670&trace=AwAAAZx6UrVOWC0CIQAAABhBWng2VXI0dEFBRFp2V0ZWbnpsclVfSjkAAAAkZjE5YzdhNTQtMjA0My00OTg2LTgyZmMtZDNmMjg4ZmY3ZGU0AABLaA&traceID=6998256f000000007df290e2863efdc1&traceQuery=&view=spans&start=1771540655768&end=1771583855768&paused=false
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.