Skip to content

feat(server): add embedded REST Gremlin execution - #3388

Open
contrueCT wants to merge 3 commits into
apache:masterfrom
hugegraph:task/gremlin-embedded-rest
Open

contrueCT wants to merge 3 commits into
apache:masterfrom
hugegraph:task/gremlin-embedded-rest

Conversation

@contrueCT

@contrueCT contrueCT commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the PR

Add opt-in in-process execution for REST /gremlin to remove its internal HTTP forwarding hop. gremlinserver.rest_execution_mode defaults to forward; embedded uses this Server's existing Gremlin runtime.

Builds on the merged Java 17 (#3261) and TinkerPop 3.8.1 (#3263) upgrades. Design and investigation.

Main Changes

forward:  REST -> GremlinClient -> HTTP -> GremlinExecutor
embedded: REST ------------------------> GremlinExecutor
                                         materialize / commit / serialize
  • Reuse the live graph manager, workers and JSON serializer, with authentication context and transaction callbacks on the worker.
  • Preserve request/error handling, blocking admission and REST/evaluation timeout behavior; validate supported local topology and wire startup/shutdown.
  • Add differential contract tests and reproducible HTTP/RocksDB comparison scripts.

Readiness, new overload controls, authentication-cache consistency and global shutdown redesign remain separate tasks.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests.
  • Need tests and can be verified as follows:
    • Java 17: formatting, full clean compile and Apache RAT checks passed.
    • 47 focused tests passed, covering REST/Gremlin contracts, authentication context, Memory/RocksDB transactions and bootstrap.
    • Standalone RocksDB: 76 measurement groups / 316,981 responses validated, including commit/rollback/restart checks; 80,200 successful writes reconciled with stored counts.

Draft follow-up: at concurrency 32 with 1,000-result responses, the extended comparison's p95 medians were 77.94 -> 68.50 ms, while p99 medians were 90.71 -> 109.59 ms (forward -> embedded). Investigate the tail behavior at matched offered load; these were closed-loop runs on a shared host. HStore, full TinkerPop suites and real authentication-store integration remain unverified.

Does this PR potentially affect the following parts?

  • Dependencies
  • Modify configurations
  • The public API
  • Other affects (Server startup/shutdown integration)
  • Nope

Documentation Status

  • Doc - TODO: required documentation is pending; complete it before merging.
  • Doc - Done: documentation is included here or linked below.
  • Doc - No Need: no user-visible documentation is affected.

Documentation: REST Gremlin execution modes, linked from the Server README and shipped configuration.

Visual overview

Forwarded versus opt-in embedded Gremlin execution

Draft validation boundaries

Draft execution-mode overview. Removing an internal HTTP hop does not itself establish performance, compatibility or acceptance.

Conceptual illustrations; not test, benchmark, security, or release-acceptance evidence.

- Add selectable forward and embedded REST executors with forward as default.
- Reuse the existing Gremlin workers, serializer, auth context and transactions.
- Integrate executor startup, shutdown and supported-topology validation.
- Add differential contract tests, usage docs and reproducible benchmarks.
@codecov

codecov Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 81 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.44%. Comparing base (c2ec570) to head (427b47d).
⚠️ Report is 26 commits behind head on master.

Files with missing lines Patch % Lines
...graph/api/gremlin/EmbeddedGremlinRestExecutor.java 77.65% 28 Missing and 14 partials ⚠️
...egraph/api/gremlin/GremlinRestExecutorFactory.java 63.41% 8 Missing and 7 partials ⚠️
...n/java/org/apache/hugegraph/server/RestServer.java 44.44% 4 Missing and 1 partial ⚠️
...ava/org/apache/hugegraph/dist/HugeGraphServer.java 50.00% 5 Missing ⚠️
...egraph/api/gremlin/ForwardGremlinRestExecutor.java 77.77% 1 Missing and 3 partials ⚠️
.../apache/hugegraph/api/gremlin/GremlinQueryAPI.java 50.00% 4 Missing ⚠️
...org/apache/hugegraph/server/ApplicationConfig.java 63.63% 3 Missing and 1 partial ⚠️
...a/org/apache/hugegraph/api/gremlin/GremlinAPI.java 80.00% 0 Missing and 1 partial ⚠️
...che/hugegraph/api/gremlin/GremlinRestExecutor.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3388      +/-   ##
============================================
+ Coverage     55.93%   56.44%   +0.51%     
- Complexity     2277     6782    +4505     
============================================
  Files           774      779       +5     
  Lines         68534    72558    +4024     
  Branches       9333    10689    +1356     
============================================
+ Hits          38334    40955    +2621     
- Misses        26634    27902    +1268     
- Partials       3566     3701     +135     
Flag Coverage Δ *Carryforward flag
commons 96.37% <ø> (ø) Carriedforward from d1fc963
hstore 35.81% <ø> (+<0.01%) ⬆️ Carriedforward from d1fc963
pd 49.15% <ø> (-0.03%) ⬇️ Carriedforward from d1fc963
server_hbase 41.83% <ø> (-0.07%) ⬇️ Carriedforward from d1fc963
server_memory 56.13% <72.72%> (+1.80%) ⬆️
server_rocksdb 42.45% <12.12%> (+0.10%) ⬆️
store 31.93% <ø> (ø) Carriedforward from d1fc963

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Initialize the executor mock used by the isolated server stop fixture.
- Assert admission shutdown precedes Gremlin and REST shutdown.
- Validate the timeout regression with 48 Java 17 regression tests.
@contrueCT
contrueCT marked this pull request as ready for review October 27, 2026 16:50
imbajin
imbajin previously approved these changes Oct 28, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: Embedded REST execution preserves worker-side context, transaction completion and serialization behavior, with explicit configuration guards. Evidence: Exact-head source review, comparison with pinned TinkerPop 3.8.1 behavior, passing server memory/RocksDB and macOS ARM CI; local runtime tests were not rerun. Score: 8.4/10.

- Resolve every configured Gremlin target address before enabling embedded mode.
- Reject mixed loopback and non-loopback DNS answers.
- Match the selected listener bind independent of target answer order.
- Add deterministic IPv4, IPv6, wildcard and mixed-address regression tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants