Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
79cf2f5
Parse the SQL an ORM renders: quoted names, projections, bind parameters
kartikparsoya-eng Oct 13, 2026
a4c6bbb
Server-to-server endpoint: subscribe a query, be told when it is touched
kartikparsoya-eng Oct 13, 2026
c36a6b3
Name the s2s task for tokio-console; console feature on the server
kartikparsoya-eng Oct 13, 2026
5558a53
README: the server-to-server endpoint
kartikparsoya-eng Oct 13, 2026
fb31451
Parser: aliased projected columns; anything else before FROM is refused
kartikparsoya-eng Oct 13, 2026
8f6458e
S2S rows mode: the hydrated result on accept, deltas per transaction
kartikparsoya-eng Oct 14, 2026
2f14622
Parser: = ANY / != ALL with binary array binds, as ORMs render IN lists
kartikparsoya-eng Oct 14, 2026
125056b
S2S: the projection's output names travel with the accepted frame
kartikparsoya-eng Oct 14, 2026
8b49b85
S2S joins: parse Diesel's joined FROM into the engine's tree, ship ro…
kartikparsoya-eng Oct 14, 2026
108f785
Passthrough Diesel connection wrapper
kartikparsoya-eng Oct 13, 2026
d49f82a
Resolve Postgres type metadata through the inner connection
kartikparsoya-eng Oct 13, 2026
f85916d
Design notes: interception point, two freshness models, engine contract
kartikparsoya-eng Oct 13, 2026
4cd795e
Owned rows that decode like live ones
kartikparsoya-eng Oct 13, 2026
e405a0f
Hyperswitch integration patch
kartikparsoya-eng Oct 13, 2026
8dec1c7
Serve repeated reads from a cache the engine keeps honest
kartikparsoya-eng Oct 13, 2026
d4d83c9
Engine client: subscribe over the S2S endpoint, drop entries it touches
kartikparsoya-eng Oct 13, 2026
c6ffd94
Live test for an engine restart; docs for the protocol and a local setup
kartikparsoya-eng Oct 13, 2026
237c1ce
Commit-to-cache latency test on any table, driven by environment
kartikparsoya-eng Oct 13, 2026
7f84114
Hyperswitch integration: live test of config lookups through the wrapper
kartikparsoya-eng Oct 13, 2026
c978da7
Rows mode: serve results from the engine's rows, no Postgres read
kartikparsoya-eng Oct 14, 2026
9ccbc99
Hyperswitch patch: rows-mode assertion
kartikparsoya-eng Oct 14, 2026
5eef9dd
Joins composed in the library from the engine's per-table rows
kartikparsoya-eng Oct 14, 2026
4f1db34
Engine client as a pool of sockets with a subscription cap
kartikparsoya-eng Oct 14, 2026
40ae00c
Workspace: the Diesel wrapper lives here as crates/streamgres-diesel
kartikparsoya-eng Oct 14, 2026
706d7b4
Routing flags per query and per connection pool
kartikparsoya-eng Oct 14, 2026
7da9e47
Cache clocks: idle time and time to live, from the environment
kartikparsoya-eng Oct 14, 2026
7cb0eaf
Settings per database: a cache per URL, a route per pool
kartikparsoya-eng Oct 14, 2026
7d698dc
Read-your-writes window after a write on a connection
kartikparsoya-eng Oct 14, 2026
febb1eb
diesel-async adapter, with subscribe split into two steps
kartikparsoya-eng Oct 14, 2026
0482296
Hyperswitch patch: the live test turns the read-your-writes window off
kartikparsoya-eng Oct 14, 2026
0c38ed4
S2S: bytea as hex text, enum binds as labels, OFFSET accepted
kartikparsoya-eng Oct 14, 2026
fb00b2f
Wrapper: refused queries remembered, bytea and OFFSET in rows mode, H…
kartikparsoya-eng Oct 14, 2026
2100a23
Times kept as microseconds; Zero still sees milliseconds
kartikparsoya-eng Oct 14, 2026
48f4be8
Wrapper: timestamps at any precision served from engine rows
kartikparsoya-eng Oct 14, 2026
fad45f4
Zero delete frames write a time key in milliseconds
kartikparsoya-eng Oct 14, 2026
18c14c1
S2S parser: enum arrays decode as text, infinite times refused, LIMIT…
kartikparsoya-eng Oct 14, 2026
3033751
S2S task: one delta entry per key, refusals unregister, bounded outgo…
kartikparsoya-eng Oct 14, 2026
ddfeb99
streamgres-diesel: review fixes for the cache, the engine client and …
kartikparsoya-eng Oct 14, 2026
38dad65
streamgres-diesel: review fixes for the diesel-async adapter
kartikparsoya-eng Oct 14, 2026
e05eb24
streamgres-diesel: design notes for the review fixes, coverage case f…
kartikparsoya-eng Oct 14, 2026
e56ab9b
Dockerfile: copy the workspace's crates so the locked build resolves
kartikparsoya-eng Oct 14, 2026
67c69c8
streamgres-diesel: second-review fixes, with the orderings forced in …
kartikparsoya-eng Oct 14, 2026
bd35086
Wrapper and parser fixes the example app turned up
kartikparsoya-eng Oct 14, 2026
3c4e254
streamgres-example: a ticket desk on the wrapper, as demo and test bed
kartikparsoya-eng Oct 14, 2026
e47c52f
demo: the compose stack runs end to end
kartikparsoya-eng Oct 14, 2026
e6c9417
Format the client crates with rustfmt
kartikparsoya-eng Oct 14, 2026
ee2edb2
Forget refusals on reconnect; say where each read was served from
kartikparsoya-eng Oct 14, 2026
7097d3f
A table added at runtime keeps the catalog's enum types
kartikparsoya-eng Oct 15, 2026
fc27680
Replace the downstream integration patch with a shape-coverage test i…
kartikparsoya-eng Oct 15, 2026
186eef2
Look a table's column types up again when a snapshot names a column t…
kartikparsoya-eng Oct 15, 2026
02f9bb2
Example app as a test bed: every shape, every kind of write, a self-c…
kartikparsoya-eng Oct 15, 2026
661bb65
Pin every storage session to UTC, so a time in a filter is read as th…
kartikparsoya-eng Oct 15, 2026
f600837
Example scenario: a wrong engine answer fails the load tests too
kartikparsoya-eng Oct 15, 2026
081ea3b
Test the flows that had no test
kartikparsoya-eng Oct 15, 2026
312b7fa
Guard the S2S endpoint with an optional token and a per-connection cap
kartikparsoya-eng Oct 19, 2026
4fb8c8b
Send every time in microseconds on the S2S path
kartikparsoya-eng Oct 19, 2026
7d83495
Bound S2S connections, announce the subscription limit, name connect …
kartikparsoya-eng Oct 19, 2026
790f2d5
Keep Zero's milliseconds for times nested in a map or a set
kartikparsoya-eng Oct 19, 2026
f871008
Rename the Diesel wrapper to diesel-streamgres and the example to str…
kartikparsoya-eng Oct 21, 2026
190de46
Merge main into feat/s2s-endpoint
kartikparsoya-eng Oct 21, 2026
598fb25
Rewrite the wrapper's design notes as a description of the code
kartikparsoya-eng Oct 21, 2026
db2e9d4
Fix the end-to-end review's findings
kartikparsoya-eng Oct 21, 2026
02fbab3
Let a closing S2S connection's writer end on its own
kartikparsoya-eng Oct 22, 2026
f012c0c
Move in-body comments into docs: the S2S endpoint and the wrapper's s…
kartikparsoya-eng Oct 22, 2026
2432b1a
Move in-body comments into docs: the parser and the S2S path's server…
kartikparsoya-eng Oct 22, 2026
095348d
Move in-body comments into docs: the wrapper's tests and the demo
kartikparsoya-eng Oct 22, 2026
92db61e
Say in the demo walk-through that a new enum takes two live reads
kartikparsoya-eng Oct 22, 2026
b2e199b
Refuse S2S queries too large for the engine's thread
kartikparsoya-eng Oct 22, 2026
8a1ed3b
Read every time PostgreSQL holds, so one infinity cannot stop the engine
kartikparsoya-eng Oct 22, 2026
de3beaa
Give up on a silent engine socket when its window ends, not at the ne…
kartikparsoya-eng Oct 22, 2026
c5c325f
Cap the WAL a crashed engine's slot can hold in the demo stack
kartikparsoya-eng Oct 22, 2026
86de8df
Never serve another database's rows: the engine names its database
kartikparsoya-eng Oct 22, 2026
0f5f8d7
Fix the engine-backed wrapper tests
kartikparsoya-eng Oct 22, 2026
dafcf5b
Lower the default silence window to 10 s
kartikparsoya-eng Oct 22, 2026
f4e6d7b
Report the engine's lag and read Postgres while it is behind
kartikparsoya-eng Oct 22, 2026
e0e355f
Wait for the reconnect in the silent-engine test rather than a fixed …
kartikparsoya-eng Oct 22, 2026
8994637
Merge main into feat/s2s-endpoint
kartikparsoya-eng Oct 22, 2026
d1c253e
Never take an engine on trust because a database identity read failed
kartikparsoya-eng Oct 22, 2026
4739359
Keep heartbeats numbered across reconnects and flush them
kartikparsoya-eng Oct 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# The image is built from the manifest and the sources alone.
# The images are built from the manifests and the sources alone: the
# workspace's crates are part of the manifest, their build output is not.
*
!Cargo.toml
!Cargo.lock
!clippy.toml
!src
!crates
crates/*/target
11 changes: 7 additions & 4 deletions .github/workflows/ci-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,17 @@ jobs:
- name: Install rust cache
uses: Swatinem/rust-cache@v2.7.7

- name: Clippy, warnings denied
run: cargo clippy --all-targets --locked -- -D warnings
- name: Clippy, warnings denied, every crate of the workspace
run: cargo clippy --workspace --all-targets --locked -- -D warnings

test:
name: Tests (accuracy)
runs-on: ubuntu-latest
env:
STREAMGRES_PG_DSN: postgresql://postgres:postgres@localhost:5432/postgres
# The Diesel wrapper's and the demo's tests read and write their own
# tables in the same database.
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand All @@ -77,8 +80,8 @@ jobs:
-c wal_level=logical -c max_replication_slots=32 -c max_wal_senders=32
until docker exec pg pg_isready -U postgres; do sleep 1; done

- name: Unit, scenario, randomized and live PostgreSQL tests
run: cargo test --release --locked
- name: Unit, scenario, randomized and live PostgreSQL tests, every crate
run: cargo test --release --locked --workspace

perf:
name: Benchmark against the base branch (performance)
Expand Down
14 changes: 8 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,22 +31,24 @@ Build and test:

```bash
cargo build
cargo test # unit and scenario tests
cargo clippy --all-targets -- -D warnings
cargo test --workspace # unit and scenario tests, every crate
cargo clippy --workspace --all-targets -- -D warnings
cargo run --bin streamgres # Streamgres demo: every check must end in PASS
cargo run --release --bin bench # benchmarks
cargo run --release --bin load # the server over the wire: latency, cores, memory (needs STREAMGRES_PG_DSN)
```

The tests against a real PostgreSQL run when `STREAMGRES_PG_DSN` is set and
report themselves skipped otherwise. To run them the way CI does:
report themselves skipped otherwise; the Diesel wrapper's and the demo's
tests read `DATABASE_URL`. To run them the way CI does:

```bash
docker run -d --name pg -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:17 \
-c wal_level=logical -c max_replication_slots=32 -c max_wal_senders=32

STREAMGRES_PG_DSN=postgresql://postgres:postgres@localhost:5432/postgres \
cargo test --release
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres \
cargo test --release --workspace
```

To run the server locally, follow the [Quick start](README.md#quick-start) in
Expand All @@ -58,8 +60,8 @@ Every pull request runs:

| Job | What it runs |
| --- | --- |
| Lints | `cargo clippy --all-targets --locked -- -D warnings` |
| Tests | `cargo test --release --locked` against PostgreSQL 17 |
| Lints | `cargo clippy --workspace --all-targets --locked -- -D warnings` |
| Tests | `cargo test --release --locked --workspace` against PostgreSQL 17 |
| Benchmark | `bench` on your branch and the base branch, five alternating rounds, compared by `scripts/bench-compare.py` |
| Image | builds `docker/server/Dockerfile` and drives it end to end with `scripts/smoke.mjs` |

Expand Down
Loading
Loading