Move the native trace analysis from Java listeners to TAL rules (SWIP-17) by wu-sheng · Pull Request #14208 · apache/skywalking

wu-sheng · 2026-11-09T11:35:38Z

Move the SkyWalking native trace analysis from Java listeners to TAL rules

If this is non-trivial feature, paste the links/URLs to the design doc. SWIP-17 (docs/en/swip/SWIP-17.md, synced to what was built), discussed in [SWIP-17] [DISCUSS] Move the Native Trace Analysis from Java Listeners to TAL #14205.
Update the documentation to include this new feature.
Tests(including UT, IT, E2E) are added to verify the new feature. Unit and comparison tests added; existing E2E cases rerun.
If it's UI related, attach the screenshots below. (Not UI related.)

What changed

Native analysis as TAL rules. The analysis of SkyWalking agents' segments runs as seven TAL rule files on a new sw-native input: segment, RPC entry/exit/logic endpoints, cross-thread endpoints, network alias, virtual cache/database/MQ/GenAI, and AI evaluation. The rules ship as config/tal/native-*.yaml, listed in the new trace-analyzer setting nativeTalFiles (SW_TRACE_NATIVE_TAL_FILES). trace-analyzer hosts the runner, and ISegmentParserService moved there. The listeners and the AnalysisListenerFactory SPI left production and are kept as the comparison baseline in test/script-cases/trace-analysis-listener-v1.
The native input. span is the agent's protobuf SpanObject. The segment is segment(), and the runner's segment-wide results are functions: segmentStartTime(), segmentLatency(), segmentIsError(), segmentTags(), segmentSampled(), segmentBytes().
Language additions, for every input: assignment with =, foreach, log, typed as Layer / as <Enum>, requiredModules, source {} (an existing source, an OAL source or a record such as DatabaseSlowStatement), and more functions (isEmpty, toJson, formatServiceName, …).
Native-only constructs: build {}, where a native rule makes its outputs, first {}, segmentDef and writeNetworkAlias.
Records are sources. A rule of either input writes a record with source {}. The OTLP outputType key, sink {} and the TALOutputBuilder interface are removed; the shipped otlp-slow-sql rule writes source DatabaseSlowStatement {}. AI evaluation samples are a new GenAIEvaluationSample source, which ai-evaluation's dispatcher hands to its evaluation service.
OTLP AI evaluation as a TAL rule. The new otlp-ai-evaluation rule (tal/otlp-ai-evaluation.yaml, added to the default talFiles) samples natively stored OTLP GenAI spans in place of the span listener's OTLP path. Zipkin, and OTLP stored as Zipkin spans, keep the listener. meta.traceStorage tells a rule how the span is stored; matchGenAIModel and the two ai-evaluation functions serve every input.
Extension points: TALSourceTypeProvider gains constructs, helper functions and bind(ModuleManager). TALFunctionProvider (server-core) lets other modules contribute functions, each provider declaring the inputs it serves; ai-evaluation contributes evaluationActive() and evaluationSampleRate(). A module takes an output of its own as a source with its SourceDispatcher.
Docs.
- The tracing docs are reorganized by where traces come from.
- New pages: tal-otlp.md and tal-sw-native.md beside the shared tal.md, native-trace.md, native-trace-analysis.md and tracing-overview.md.
- SWIP-17 and the changelog are updated.

Behaviour

Metrics, topology and stored traces stay the same, apart from:

Repeated tag key: tag lookups (tag()) and tag maps keep the first value of a repeated key, and a duplicate key no longer aborts the cache analysis. Rules that iterate the raw tag list keep their previous behavior.
Native AI evaluation samples: they are deferred to the build phase, so a segment that fails before its builds run emits none.
Feature, bare-host gateway match: a bare host now matches a gateway listed at port 80, as OTLP already did. Its debug lines follow the lookups made.
Script loggers: logs emitted by the ported rules use org.apache.skywalking.trace.analysis.script.<file>; runner and shared-service diagnostics use Java logger names.
Startup: SW_TRACE_ANALYZER=- stops the boot while the native trace receiver or the Kafka fetcher needs the module.
OTLP rule files: a custom rule file written for 11.1.0 that uses outputType or sink {} fails to load until its record moves to source {}. The shipped slow-statement record is unchanged.
OTLP AI evaluation: like every OTLP rule, otlp-ai-evaluation sees the spans of services that set service.layer=OTLP_GENERAL; a GenAI application that sets no layer is no longer evaluated. A deployment that overrides talFiles adds otlp-ai-evaluation. Zipkin spans, and OTLP spans stored as Zipkin spans, are evaluated as before. The rule runs with the other TAL rules, before the receiver checks the span listeners' veto and the rules' persist {} votes, where the listener ran inside the native store after both: a span that a custom rule or listener keeps out of storage is now evaluated, though no shipped rule or listener vetoes a GenAI span. With dedupWindow above 0, a span sent again to the same node, while that node's dedup set still holds it, is not evaluated again. A failure while an evaluation sample is handed over no longer drops the span, as the native store's catch did when the listener threw.
OTLP source dispatch: a source is dispatched at its statement, so a failure while dispatching fails the rule, which is caught and skipped, and its persist {} vote is not counted; the runner used to write the slow-statement record after the rule and keep the vote on a failed write.

Verification

v1-v2 checker (test/script-cases/trace-analysis-v1-v2-checker): the listeners and the rules run the same constructed segments across 21 configuration profiles, cold and warm. The comparison is at TAL's outputs: every source and record, stateful collaborator call, log line and failure. The documented differences have explicit expectations. A JaCoCo gate requires every branch of the moved listener classes to be reached, except 5 allowlisted unreachable ones (exact counts). The v1 events are written to target/ for logic review.
Compiled code vs the old code: the 11 rule classes generated by the production compiler were decompiled (FernFlower) and reviewed rule by rule against the listeners and the runner against TraceAnalyzer. They match, apart from the documented differences above.
Slow-statement record, old vs new: the shipped otlp-slow-sql rule before this change, with the SlowSQL builder written as the OTLP runner wrote it, and after, with source DatabaseSlowStatement {}, ran the same 32 constructed OTLP spans in two worktrees. They cover every peer-naming fallback, the server-cluster lookup, the cache and kind aborts, latency and statement edges, cutting, unicode, an over-long peer name and timestamp edges. Every DatabaseSlowStatement (25 written, 7 refused) and every persist vote matched.
OTLP AI evaluation, old vs new: the listener's OTLP path before this change and the otlp-ai-evaluation rule with the real dispatcher after ran the same 79 constructed OTLP spans in two worktrees, at rate 1,000,000, at rate 500,000 over 60 seeded trace ids, and inactive. All 237 outcomes matched (76, 43 and 0 sampled), with every AIEvaluationContext field equal, including a span with a zero start time.
A/B on identical input: the pre-PR OAP image and the new one consumed the same agent segments from Kafka, each with its own consumer group and BanyanDB.
- Steady state: 178 distinct query pairs. 177 completed and matched; one MQ endpoint-topology query errored on both sides.
- What matched: the service, instance and endpoint lists, global/service/instance/endpoint topology, selected service, instance, endpoint, relation, virtual-DB and virtual-MQ metrics per minute (including selected percentiles), and the trace list.
- Network alias rows: read separately from both BanyanDBs; the mapping and time fields matched (BanyanDB row versions differ).
- Full window: the only differences were three client-side relation metric queries at 10:27 UTC, right after traffic started at 10:26:57, consistent with alias-cache warm-up.
E2E, locally, after the output-model change: 7 cases, 203 checks, all pass: storage/banyandb, otlp-traces/native/banyandb (the slow-statement source), virtual-genai, mixed-virtual-genai, otlp-virtual-genai/native, otlp-virtual-genai, zipkin-virtual-genai. The two cases with natively stored OTLP GenAI spans now set service.layer=OTLP_GENERAL on their OTLP application, which otlp-ai-evaluation needs.
E2E, locally, before the output-model change: 11 native-agent cases, 312 checks, all pass:
- storage/banyandb, gateway, virtual-mq, server-clusters/native;
- virtual-genai, mixed-virtual-genai;
- banyandb/trace-sampling, kafka/simple-so11y;
- lua, python, go.
Unit tests: full backend suite, 4,157 tests reported, 0 failures, 0 errors, 5 skipped. Checkstyle passed. Javadoc completed successfully, with warnings.

Performance

NativeTraceBenchmark (JMH, in the checker module) runs both engines in the same run, with two forks and logging off. It uses plain module lookups, and every emitted source is consumed by a per-thread blackhole. Results are segments per second, and bytes allocated per segment.

Workload	Java listeners	TAL rules	Rules vs Java	Allocation, Java → TAL
Fixture mix, cold alias cache	464.4k ±8.3k	466.0k ±9.8k	+0.4%	8.06 → 7.64 KB (−5.2%)
Fixture mix, warm alias cache	463.0k ±5.9k	450.0k ±10.9k	−2.8%	8.09 → 7.73 KB (−4.4%)
Wide segment, cold	31.8k ±0.6k	29.8k ±0.7k	−6.4%	79.5 → 65.8 KB (−17.2%)
Wide segment, warm	24.5k ±0.3k	23.8k ±0.4k	−3.0%	92.2 → 74.0 KB (−19.7%)
4 threads, cold	105.7k ±3.5k	109.4k ±1.6k	+3.5%	79.3 → 66.2 KB (−16.6%)
4 threads, warm	83.7k ±2.7k	90.5k ±2.4k	+8.1%	92.2 → 74.0 KB (−19.7%)

Fixture mix: the checker's constructed segments minus the six intentional-difference ones, 103 when measured, run under one default-derived configuration with evaluation inactive. One operation is one segment, cycling through the list.
Wide segment: one 40-span segment, with 8 references on its entry span and 12 tags on each span. "4 threads" means four such segments analyzed concurrently, each by one thread, on shared collaborators.
Cold / warm alias cache: OAP's network address alias cache, which maps a dialed address to the real service. Cold is empty: every exit lookup misses and the peer becomes a virtual service. Warm holds fixture-generated and synthetic alias rows: the wide workloads preload every exit peer, while the fixture mix still includes misses and exceptional alias cases.
Confidence intervals: only wide/cold (−6.4%) and 4-threads/warm (+8.1%) have non-overlapping 99.9% confidence intervals in this run; the intervals overlap for the other workloads.

The measurements cover the native analysis. The changes behind them:

Shared TAL compiler changes: compiled rules hold their function objects in fields; literal enum and string comparisons compile to == / equals; a built-in Layer literal compiles to its constant.
Native-only: the native functions read services bound once at startup, and the native runner reuses one context per segment and resets only what a native rule can change.

The OTLP runner is unchanged apart from no longer writing records itself; the shipped OTLP slow-SQL rule writes the same record through source {}.

TAL skill

A new contributor skill, .claude/skills/tal/SKILL.md, summarizes how to work with TAL:

the two inputs and where the rule files live;
the compiler and runtime code map, and the generated-code performance rules;
dumping the generated classes (SW_DYNAMIC_CLASS_ENGINE_DEBUG) and decompiling them;
the v1-v2 checker, the branch gate, and the benchmark.

Follow-ups (out of scope)

The shared AliasRegistrar.hostOf still uses a character stream when it checks a single-colon port suffix; that helper is unchanged.
Older doc inaccuracies that remain unresolved: the OTLP meta-table fallbacks, the labelLimit sentence, the iOS advice in the new-monitoring-feature skill, and typos in the trace sampling doc.
If this pull request closes/resolves/fixes an existing issue, replace the issue number. Not applicable; this PR does not close an issue.
Update the CHANGES log.

Copilot

🔵 Needs a closer look

The native-trace migration spans generated TAL execution and module startup ordering, requiring maintainer validation of output parity and boot behavior unavailable in this source-only review.

No open issues 🎉

What changed in this PR

Moves SkyWalking’s native trace analysis into the TAL-based trace-analyzer, while retaining the Java listeners as a test-only comparison baseline.

Changes:

Routes native segments through TAL rules, with native execution state, output builders and function extension points.
Adds comparison-checker support and compiler/runtime test fixtures.
Updates configuration, module dependencies and tracing documentation.

File	Description
test/script-cases/trace-analysis-v1-v2-checker/src/test/resources/checker-allowed-misses.yml	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/trace/analysis/checker/CheckerWireIndex.java	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/trace/analysis/checker/CheckerNamingControl.java	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/trace/analysis/checker/CheckerFrameClassifier.java	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/trace/analysis/checker/CheckerEvaluationService.java	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/trace/analysis/checker/CheckerDifference.java	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/trace/analysis/checker/CheckerApplicationYml.java	Updated as part of this pull request.
test/script-cases/trace-analysis-v1-v2-checker/src/test/java/org/apache/skywalking/oap/server/core/cache/CheckerAliasCacheLoader.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/test/java/org/apache/skywalking/oap/server/receiver/trace/provider/parser/listener/MockReceiver.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.analyzer.provider.trace.parser.listener.AnalysisListenerFactory	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/SpanTags.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/SegmentParserServiceImpl.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/SegmentParserListenerManager.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/ListenerManagerAssembly.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/vservice/VirtualServiceProcessor.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/vservice/VirtualGenAIProcessor.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/SegmentListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/LocalAnalysisListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/FirstAnalysisListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/ExitAnalysisListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/EntryAnalysisListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/EndpointSourceBuilder.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/EndpointDependencyBuilder.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/CommonAnalysisListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/AnalysisListenerFactory.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/AnalysisListener.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/ISegmentParserListenerManager.java	Updated as part of this pull request.
test/script-cases/trace-analysis-listener-v1/pom.xml	Updated as part of this pull request.
test/script-cases/pom.xml	Updated as part of this pull request.
test/e2e-v2/cases/storage/expected/config-dump.yml	Updated as part of this pull request.
pom.xml	Updated as part of this pull request.
oap-server/server-tools/data-generator/src/main/resources/application.yml	Updated as part of this pull request.
oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/restapi/SegmentGeneratorHandler.java	Updated as part of this pull request.
oap-server/server-tools/data-generator/src/main/java/org/apache/skywalking/module/DataGeneratorModuleProvider.java	Updated as part of this pull request.
oap-server/server-starter/src/main/resources/tal/native-network-alias.yaml	Updated as part of this pull request.
oap-server/server-starter/src/main/resources/tal/native-ai-evaluation.yaml	Updated as part of this pull request.
oap-server/server-starter/src/main/resources/application.yml	Updated as part of this pull request.
oap-server/server-receiver-plugin/skywalking-trace-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/trace/provider/TraceModuleProvider.java	Updated as part of this pull request.
oap-server/server-receiver-plugin/skywalking-trace-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/trace/provider/handler/v8/rest/TraceSegmentReportHandler.java	Updated as part of this pull request.
oap-server/server-receiver-plugin/skywalking-trace-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/trace/provider/handler/v8/grpc/TraceSegmentReportServiceHandler.java	Updated as part of this pull request.
oap-server/server-receiver-plugin/skywalking-trace-receiver-plugin/pom.xml	Updated as part of this pull request.
oap-server/server-receiver-plugin/otel-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/otel/otlp/ShippedRuleOverRecordedTraceTest.java	Updated as part of this pull request.
oap-server/server-fetcher-plugin/kafka-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/analyzer/agent/kafka/provider/KafkaFetcherProvider.java	Updated as part of this pull request.
oap-server/server-fetcher-plugin/kafka-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/analyzer/agent/kafka/provider/handler/TraceSegmentHandler.java	Updated as part of this pull request.
oap-server/server-fetcher-plugin/kafka-fetcher-plugin/pom.xml	Updated as part of this pull request.
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/TALOutputBuilder.java	Updated as part of this pull request.
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/TALFunctionProvider.java	Updated as part of this pull request.
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/TALFunctionContext.java	Updated as part of this pull request.
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/DefaultScopeDefine.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/resources/tal-native-files/third.yaml	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/resources/tal-native-files/second.yaml	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/resources/tal-native-files/otlp-rule.yaml	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/resources/tal-native-files/first.yaml	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/resources/tal-native-files/duplicate.yaml	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/NativeRuleFilesTest.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestTypedOutput.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestSpanSourceTypeProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestSpanChild.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestRefineProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestNestedSpanInput.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestNativeTwiceProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestNativeSideProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestNativeRecord.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestMetadataSideProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestInputProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestFunctionProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TestDuplicateFunctionProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TALWorkedExampleTest.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TALTestModules.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/test/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TALExecutionTest.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/resources/META-INF/services/org.apache.skywalking.oap.trace.analyzer.v2.spi.TALSourceTypeProvider	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/spi/TALConstruct.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/TraceAnalyzerModuleProvider.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/TraceAnalyzerModuleConfig.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/nativetrace/SwNativeSegmentState.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/nativetrace/NetworkAliasMatch.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/nativetrace/ISegmentParserService.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/provider/nativetrace/GenAIModelMatch.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/module/TraceAnalyzerModule.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TalExpression.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/TalDSL.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/dsl/QueuedBuild.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/TALRuntimeRoles.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/TALRootResolver.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/TALOutputBuilders.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/TALLocalVar.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/TALInputs.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/rt/TalNativeOutputs.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/java/org/apache/skywalking/oap/trace/analyzer/v2/compiler/ExprType.java	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/antlr4/org/apache/skywalking/tal/rt/grammar/TALParser.g4	Updated as part of this pull request.
oap-server/analyzer/trace-analyzer/src/main/antlr4/org/apache/skywalking/tal/rt/grammar/TALLexer.g4	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.core.source.TALOutputBuilder	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.core.source.TALFunctionProvider	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/java/org/apache/skywalking/oap/server/ai/evaluation/listener/GenAIEvaluationSampleBuilder.java	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/java/org/apache/skywalking/oap/server/ai/evaluation/AIEvaluationTALFunctions.java	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/java/org/apache/skywalking/oap/server/ai/evaluation/AIEvaluationSamplingConfig.java	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/java/org/apache/skywalking/oap/server/ai/evaluation/AIEvaluationProvider.java	Updated as part of this pull request.
oap-server/analyzer/ai-evaluation/src/main/java/org/apache/skywalking/oap/server/ai/evaluation/AIEvaluationModule.java	Updated as part of this pull request.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/NativeTraceAnalysisConfigImpl.java	Updated as part of this pull request.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/NativeTraceAnalysisConfig.java	Updated as part of this pull request.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/AnalyzerModuleProvider.java	Updated as part of this pull request.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/module/AnalyzerModule.java	Updated as part of this pull request.
docs/menu.yml	Updated as part of this pull request.
docs/en/swip/readme.md	Updated as part of this pull request.
docs/en/setup/backend/zipkin-trace.md	Updated as part of this pull request.
docs/en/setup/backend/uninstrumented-gateways.md	Updated as part of this pull request.
docs/en/setup/backend/tracing-overview.md	Updated as part of this pull request.
docs/en/setup/backend/trace-sampling.md	Updated as part of this pull request.
docs/en/setup/backend/server-clusters.md	Updated as part of this pull request.
docs/en/setup/backend/otlp-trace.md	Updated as part of this pull request.
docs/en/setup/backend/otlp-trace-practices/message-queue.md	Updated as part of this pull request.
docs/en/setup/backend/otlp-trace-practices/gateway-overview.md	Updated as part of this pull request.
docs/en/setup/backend/native-trace.md	Updated as part of this pull request.
docs/en/setup/backend/configuration-vocabulary.md	Updated as part of this pull request.
docs/en/setup/backend/backend-wechat-mini-program-monitoring.md	Updated as part of this pull request.
docs/en/setup/backend/backend-alipay-mini-program-monitoring.md	Updated as part of this pull request.
docs/en/changes/changes.md	Updated as part of this pull request.

_{Review effort: Balanced}

…-17) The analysis of SkyWalking agents' segments now runs as seven TAL rule files (config/tal/native-*.yaml, listed in trace-analyzer's nativeTalFiles) on a new sw-native input whose span is the agent's protobuf SpanObject. trace-analyzer hosts the runner and ISegmentParserService. The listeners and the AnalysisListenerFactory SPI leave production and stay as the comparison baseline in test/script-cases/trace-analysis-listener-v1. TAL gains assignment, foreach, log, typed Layer/enum values, requiredModules, more functions and source {} for every input, and for the native input build {}, first {}, segmentDef and writeNetworkAlias, with function extension points for inputs (TALSourceTypeProvider) and modules (TALFunctionProvider). Records are sources: the OTLP outputType key, sink {} and TALOutputBuilder are removed, the shipped otlp-slow-sql rule writes source DatabaseSlowStatement {}, and AI evaluation samples are a GenAIEvaluationSample source that ai-evaluation's dispatcher takes. The new otlp-ai-evaluation rule samples natively stored OTLP GenAI spans in place of the span listener's OTLP path, for services that set service.layer=OTLP_GENERAL; Zipkin keeps the listener. Metrics, topology and stored traces stay the same, except that tag lookups and tag maps keep the first value of a repeated key, and native AI evaluation samples are deferred to the build phase, so a failure before their builds run emits none. A failure while an OTLP rule dispatches a source fails that rule, as any rule failure does. A bare host now matches a gateway listed at port 80 for agent traces, as it already did for OpenTelemetry traces. A custom OTLP rule file that uses outputType or sink {} fails to load until its record moves to source {}. The v1-v2 checker compares the listeners and the rules at TAL's outputs across 21 profiles with a JaCoCo branch gate, and NativeTraceBenchmark compares their throughput and allocation: in the recorded JMH run, TAL's mean throughput ranged from 6.4% lower to 8.1% higher than the listeners, while allocation fell by 4.4-19.7%. The tracing docs are reorganized by source, SWIP-17 is synced to what was built, and a tal contributor skill is added.

…nload hardening Docs: - tal.md: blocks or rules that share a metric must also agree on its labelLimit; an omitted one counts as 50. - tal-otlp.md: the meta table states the service.instance fallback of serviceInstance, the start and end time fallbacks, and the latency clamp. - trace-sampling.md: the setting is named default.rate throughout; the duration.rate and default.rare typos are fixed, and the example rates are 3500 and 5500 at the 1/10000 precision. - virtual-genai.md, trace-analyzer.md: AI evaluation of natively stored OTLP spans needs service.layer=OTLP_GENERAL, the layer otlp-ai-evaluation is declared for; the Virtual GenAI metrics and the Zipkin-path evaluation do not. trace-analyzer.md's instance row names the service.instance fallback. The changelog's otlp-ai-evaluation line no longer says every OpenTelemetry rule needs OTLP_GENERAL. - new-monitoring-feature skill: iOS does get service_cpm and the other core OAL metrics, from IOSHTTPSpanListener; browser has its own OAL metrics (browser_app_*); the anti-pattern note points at the mini-program pattern. Code: - AliasRegistrar.hostOf checks the port in place instead of through a substring and a character stream, with the same result for every input; a new AliasRegistrarTest case pins it. Tests: - OtlpAIEvaluationAnalysisTest runs the shipped otlp-ai-evaluation rule through TraceAnalyzerServiceImpl: another rule's drop vote does not stop the sample, a span sent again is sampled once while the dedup window remembers it, and a failed dispatch fails only its rule, whose persist vote is then not counted. - OTLPAnalysisBeforeStorageHandlerTest: a span that a span listener vetoes still reaches the TAL analysis, and only the other span is stored. E2E: - download-mysql.sh makes up to five download attempts, 5 seconds apart, each with curl --fail and --max-time 60, and exits non-zero when all fail, so an HTTP error such as a 429 is not saved as the driver jar. After curl is installed, the download takes at most about 320 seconds, and if every attempt fails the OAP does not start. curl's --retry is not used: it sleeps for as long as a 429's Retry-After asks.

…nload hardening (#14209) Docs: - tal.md: blocks or rules that share a metric must also agree on its labelLimit; an omitted one counts as 50. - tal-otlp.md: the meta table states the service.instance fallback of serviceInstance, the start and end time fallbacks, and the latency clamp. - trace-sampling.md: the setting is named default.rate throughout; the duration.rate and default.rare typos are fixed, and the example rates are 3500 and 5500 at the 1/10000 precision. - virtual-genai.md, trace-analyzer.md: AI evaluation of natively stored OTLP spans needs service.layer=OTLP_GENERAL, the layer otlp-ai-evaluation is declared for; the Virtual GenAI metrics and the Zipkin-path evaluation do not. trace-analyzer.md's instance row names the service.instance fallback. The changelog's otlp-ai-evaluation line no longer says every OpenTelemetry rule needs OTLP_GENERAL. - new-monitoring-feature skill: iOS does get service_cpm and the other core OAL metrics, from IOSHTTPSpanListener; browser has its own OAL metrics (browser_app_*); the anti-pattern note points at the mini-program pattern. Code: - AliasRegistrar.hostOf checks the port in place instead of through a substring and a character stream, with the same result for every input; a new AliasRegistrarTest case pins it. Tests: - OtlpAIEvaluationAnalysisTest runs the shipped otlp-ai-evaluation rule through TraceAnalyzerServiceImpl: another rule's drop vote does not stop the sample, a span sent again is sampled once while the dedup window remembers it, and a failed dispatch fails only its rule, whose persist vote is then not counted. - OTLPAnalysisBeforeStorageHandlerTest: a span that a span listener vetoes still reaches the TAL analysis, and only the other span is stored. E2E: - download-mysql.sh makes up to five download attempts, 5 seconds apart, each with curl --fail and --max-time 60, and exits non-zero when all fail, so an HTTP error such as a 429 is not saved as the driver jar. After curl is installed, the download takes at most about 320 seconds, and if every attempt fails the OAP does not start. curl's --retry is not used: it sleeps for as long as a 429's Retry-After asks.

…2e download hardening (apache#14209)

wu-sheng requested a balanced review from Copilot November 9, 2026 11:35

Copilot started reviewing on behalf of wu-sheng November 9, 2026 11:37

View session

wu-sheng added this to the 11.2.0 milestone Nov 9, 2026

wu-sheng added backend OAP backend related. feature New feature labels Nov 9, 2026

wu-sheng requested a review from wankai123 November 9, 2026 11:41

Copilot AI reviewed Nov 9, 2026

View reviewed changes

wu-sheng force-pushed the swip-17-tal-native-trace branch 3 times, most recently from 4bd9b4c to 282437e Compare November 10, 2026 03:55

wu-sheng force-pushed the swip-17-tal-native-trace branch from 282437e to 382f5e6 Compare November 10, 2026 04:29

wankai123 approved these changes Nov 10, 2026

View reviewed changes

wu-sheng merged commit 06fb9ae into master Nov 10, 2026
484 of 487 checks passed

wu-sheng deleted the swip-17-tal-native-trace branch November 10, 2026 06:49

wu-sheng mentioned this pull request Nov 10, 2026

Follow-ups to #14208: doc fixes, OTLP AI evaluation tests and e2e download hardening #14209

Merged

3 of 4 tasks

Moutia-Ben-Yahia pushed a commit to HotOpenSourcing/skywalking that referenced this pull request Nov 10, 2026

Follow-ups to apache#14208: doc fixes, OTLP AI evaluation tests and e…

f39b455

…2e download hardening (apache#14209)

Navigation Menu

Repository navigation

Move the native trace analysis from Java listeners to TAL rules (SWIP-17) - #14208

Move the native trace analysis from Java listeners to TAL rules (SWIP-17)#14208
wu-sheng merged 1 commit into
masterfrom
swip-17-tal-native-trace

wu-sheng commented Nov 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wu-sheng commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Move the SkyWalking native trace analysis from Java listeners to TAL rules

What changed

Behaviour

Verification

Performance

TAL skill

Follow-ups (out of scope)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wu-sheng commented Nov 9, 2026 •

edited

Loading