Repository navigation
Follow-ups to #14208: doc fixes, OTLP AI evaluation tests and e2e download hardening - #14209
Merged
Merged
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
4 open issues
This test relies on reflective access to mutate a private field, which is brittle and can break… · New The download writes directly to the final jar path and leaves${MYSQL_URL}unquoted. If curl… · New This introduces an unconditionalexit 0on successful download, whereas the previous version… · New Subject–verb agreement:services.[].rateandservices.[].durationshould use 'have' (plural),… · New
What changed in this PR
Follow-up changes after #14208 to correct documentation, close test gaps around OTLP AI evaluation / trace analysis behavior, and harden an e2e dependency download step.
Changes:
- Hardened MySQL driver download in e2e setup with bounded retries and curl failure handling.
- Added unit tests for OTLP AI evaluation behavior and for analysis running even when a span is vetoed from storage.
- Optimized and expanded coverage for gateway host parsing, plus multiple documentation corrections.
| File | Description |
|---|---|
| test/e2e-v2/script/prepare/setup-oap/download-mysql.sh | Adds retry + timeout + --fail to prevent saving HTTP error bodies as a jar. |
| oap-server/server-receiver-plugin/otel-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OTLPAnalysisBeforeStorageHandlerTest.java | New test asserting TAL analysis runs even if a span listener vetoes persistence. |
| oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/span/OtlpAIEvaluationAnalysisTest.java | New tests for shipped otlp-ai-evaluation behavior (dedup, failures, vote handling). |
| oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/span/AliasRegistrarTest.java | Adds cases covering host parsing edge cases, including empty/non-ASCII ports. |
| oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/span/AliasRegistrar.java | Replaces substring+stream digit check with in-place loop. |
| docs/en/setup/service-agent/virtual-genai.md | Documents service.layer=OTLP_GENERAL requirement for native OTLP AI evaluation only. |
| docs/en/setup/backend/trace-sampling.md | Fixes setting name to default.rate and corrects examples/typos. |
| docs/en/setup/backend/trace-analyzer.md | Clarifies OTLP AI evaluation layer scoping and service.instance fallback. |
| docs/en/concepts-and-designs/tal.md | Documents labelLimit compatibility requirement when sharing metrics. |
| docs/en/concepts-and-designs/tal-otlp.md | Clarifies OTLP meta fallbacks and timestamp/latency normalization behavior. |
| docs/en/changes/changes.md | Updates changelog to capture doc fixes and metadata behavior. |
| .claude/skills/new-monitoring-feature/SKILL.md | Corrects guidance about browser/iOS metrics sources and anti-pattern note. |
Review effort: Lite
Copilot stopped reviewing on behalf of
wu-sheng due to an error
November 10, 2026 09:21
wankai123
previously approved these changes
Nov 10, 2026
…nload hardening Docs: - tal.md: blocks or rules that share a metric must also agree on its labelLimit; an omitted one counts as 50. - tal-otlp.md: the meta table states the service.instance fallback of serviceInstance, the start and end time fallbacks, and the latency clamp. - trace-sampling.md: the setting is named default.rate throughout; the duration.rate and default.rare typos are fixed, and the example rates are 3500 and 5500 at the 1/10000 precision. - virtual-genai.md, trace-analyzer.md: AI evaluation of natively stored OTLP spans needs service.layer=OTLP_GENERAL, the layer otlp-ai-evaluation is declared for; the Virtual GenAI metrics and the Zipkin-path evaluation do not. trace-analyzer.md's instance row names the service.instance fallback. The changelog's otlp-ai-evaluation line no longer says every OpenTelemetry rule needs OTLP_GENERAL. - new-monitoring-feature skill: iOS does get service_cpm and the other core OAL metrics, from IOSHTTPSpanListener; browser has its own OAL metrics (browser_app_*); the anti-pattern note points at the mini-program pattern. Code: - AliasRegistrar.hostOf checks the port in place instead of through a substring and a character stream, with the same result for every input; a new AliasRegistrarTest case pins it. Tests: - OtlpAIEvaluationAnalysisTest runs the shipped otlp-ai-evaluation rule through TraceAnalyzerServiceImpl: another rule's drop vote does not stop the sample, a span sent again is sampled once while the dedup window remembers it, and a failed dispatch fails only its rule, whose persist vote is then not counted. - OTLPAnalysisBeforeStorageHandlerTest: a span that a span listener vetoes still reaches the TAL analysis, and only the other span is stored. E2E: - download-mysql.sh makes up to five download attempts, 5 seconds apart, each with curl --fail and --max-time 60, and exits non-zero when all fail, so an HTTP error such as a 429 is not saved as the driver jar. After curl is installed, the download takes at most about 320 seconds, and if every attempt fails the OAP does not start. curl's --retry is not used: it sleeps for as long as a 429's Retry-After asks.
wu-sheng
force-pushed
the
fix/tal-followups
branch
from
November 10, 2026 09:54
a657bf5 to
edcd45e
Compare
wankai123
approved these changes
Nov 10, 2026
Moutia-Ben-Yahia
pushed a commit
to HotOpenSourcing/skywalking
that referenced
this pull request
Nov 10, 2026
…2e download hardening (apache#14209)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Fix the doc inaccuracies and test gaps left after #14208
These are the follow-ups #14208 left out of scope, plus two gaps in its own docs.
Docs
tal.md: blocks or rules that share a metric must also agree on itslabelLimit(TALMetricDeclaration.equalscompares it); an omitted one counts as 50.tal-otlp.md: themetatable now states what the receiver does:serviceInstancefalls back toservice.instance, a start or end time left at 0 is replaced by the other one (or by the arrival time when both are 0), andlatencyis clamped to0..Integer.MAX_VALUE.trace-sampling.md: the setting is nameddefault.ratethroughout, theduration.rateanddefault.raretypos are fixed, and the example rates are3500/5500(35% / 55% at the 1/10000 precision) instead of35/55.virtual-genai.md,trace-analyzer.md: AI evaluation of natively stored OTLP spans needsservice.layer=OTLP_GENERAL, the layerotlp-ai-evaluationis declared for. The Virtual GenAI metrics and the Zipkin-path evaluation don't need it.trace-analyzer.md's instance row names theservice.instancefallback. The changelog'sotlp-ai-evaluationline no longer says every OpenTelemetry rule needsOTLP_GENERAL.new-monitoring-featureskill: iOS does getservice_cpmand the other core OAL metrics, fromIOSHTTPSpanListener. Browser has its own OAL metrics (browser_app_*), and the anti-pattern note now points at the mini-program pattern.Code
AliasRegistrar.hostOf, used by the gateway check, checks the port in place instead of through a substring and a character stream. The result is the same for every input: an old-vs-new run over 2,327,680 addresses (every UTF-16 character as the port, plus 2 million random strings) found no difference. A newAliasRegistrarTestcase pins the forms, including the empty port and non-ASCII digits.Tests for documented behaviour of the OTLP AI evaluation that had no dedicated test:
OtlpAIEvaluationAnalysisTestruns the shippedotlp-ai-evaluationrule throughTraceAnalyzerServiceImpl:persist {}vote ignored, with a control showing the vote counts when the dispatch succeeds.OTLPAnalysisBeforeStorageHandlerTest: a span that a span listener vetoes still reaches the TAL analysis, and only the other span is stored.Four mutations were checked, each making its matching test fail: stopping at the first drop vote, counting a failed rule's vote, turning dedup off, and skipping the analysis of a vetoed span.
E2E
download-mysql.shusedcurl -Lo, which exits 0 on an HTTP 429 and saves the error body as the driver jar, and had no timeout. It now makes up to five attempts, 5 seconds apart, each with--failand--max-time 60, and exits non-zero when all fail. After curl is installed, the download takes at most about 320 seconds, and if every attempt fails the OAP does not start, since the entrypoint runs it with&&. The attempts are retried by the script, not by curl's--retry: that sleeps for as long as a 429'sRetry-Afterasks, and with--retry-max-timeadded, curl 8.19 exited 0 without writing the file whenRetry-Afterexceeded it.Retry-After: 1000(5 attempts, exit 1 after 20 seconds, no file) and one that never answers (exit 1 after 5 timed-out attempts), and inside the OAP base imageeclipse-temurin:25-jre, where it installed curl 8.18 and downloaded the jar.curl -Lo, and no OAP log was collected to confirm it.Verification
Full unit-test suite (4,163 tests, 0 failures, 0 errors, 5 skipped), checkstyle, the
-Pallbuild with javadoc, and the license-header check of every tracked file pass.Add a unit test to verify that the fix works.
Explain briefly why the bug exists and how to fix it.
If this pull request closes/resolves/fixes an existing issue, replace the issue number. Not applicable; there is no issue.
Update the
CHANGESlog.