Ruby Pinot Client

A Ruby client library for Apache Pinot. Mirrors the API of the Go client and supports HTTP JSON transport, multiple broker discovery strategies, and parameterized queries.

Installation

Add to your Gemfile:

gem "pinot-client", "~> 1.0"

Or install directly:

gem install pinot-client

Quick Start

Start a local Pinot cluster using Docker:

docker run -d \
  --name pinot-quickstart \
  -p 8000:8000 \
  apachepinot/pinot:1.5.0 QuickStart -type BATCH

Then query it from Ruby:

require "pinot"

client = Pinot.from_broker_list(["localhost:8000"])
resp   = client.execute_sql("baseballStats", "SELECT count(*) AS cnt FROM baseballStats")
puts resp.result_table.get_long(0, 0)  # => 97889

Examples

The examples/ directory contains runnable scripts demonstrating common usage patterns:

• basic_query.rb — simple SQL query and result iteration
• prepared_statement.rb — parameterized queries with PreparedStatement
• controller_discovery.rb — automatic broker discovery via Controller
• instrumentation.rb — attaching metrics/logging hooks
• multistage_query.rb — multi-stage query engine usage

Run any example against a local Pinot quickstart cluster:

ruby examples/basic_query.rb

Creating a Connection

From a broker list

client = Pinot.from_broker_list(["localhost:8000"])

For HTTPS brokers, include the scheme:

client = Pinot.from_broker_list(["https://pinot-broker.example.com"])

From a controller (dynamic broker discovery)

The client periodically polls the controller's /v2/brokers/tables API and automatically picks up broker changes.

client = Pinot.from_controller("localhost:9000")

Via gRPC

Requires the grpc gem (gem "grpc", "~> 1.65" in your Gemfile).

grpc_config = Pinot::GrpcConfig.new(
  broker_list: ["localhost:8090"],
  timeout:     10,
  extra_metadata: { "Authorization" => "Bearer <token>" }
)
config = Pinot::ClientConfig.new(grpc_config: grpc_config)
client = Pinot.from_config(config)

From ZooKeeper (dynamic broker discovery)

Requires the zk gem (gem "zk" in your Gemfile). Watches /EXTERNALVIEW/brokerResource and automatically picks up broker changes.

zk_config = Pinot::ZookeeperConfig.new(zk_path: "localhost:2181")
config = Pinot::ClientConfig.new(zookeeper_config: zk_config)
client = Pinot.from_config(config)

From a ClientConfig

config = Pinot::ClientConfig.new(
  broker_list:           ["localhost:8000"],
  http_timeout:          10,          # seconds — sets open_timeout, read_timeout, and write_timeout on the underlying Net::HTTP connection
  extra_http_header:     { "Authorization" => "Bearer <token>" },
  use_multistage_engine: false,
  controller_config: Pinot::ControllerConfig.new(
    controller_address:            "localhost:9000",
    update_freq_ms:                1000,
    extra_controller_api_headers:  { "Authorization" => "Bearer <token>" }
  )
)
client = Pinot.from_config(config)

validate! raises Pinot::ConfigurationError early if the config is invalid.

With TLS

tls = Pinot::TlsConfig.new(
  ca_cert_file:        "/path/to/ca.pem",
  client_cert_file:    "/path/to/client.crt",
  client_key_file:     "/path/to/client.key",
  insecure_skip_verify: false   # set true to skip cert verification
)
config = Pinot::ClientConfig.new(
  broker_list: ["https://pinot-broker.example.com:8000"],
  tls_config:  tls
)
client = Pinot.from_config(config)

Logging

By default the client logs warnings to stdout. Configure a custom logger globally or per-client:

# Global logger
Pinot::Logging.logger = Logger.new("pinot.log", level: Logger::DEBUG)

# Per-client logger via ClientConfig
config = Pinot::ClientConfig.new(
  broker_list: ["localhost:8000"],
  logger: Logger.new($stdout, level: Logger::INFO)
)
client = Pinot.from_config(config)

Instrumentation

Hook into every query execution for metrics, tracing, or alerting:

Pinot::Instrumentation.on_query = lambda do |event|
  puts "#{event[:table]} #{event[:query][0..50]} " \
       "#{event[:duration_ms].round(1)}ms " \
       "success=#{event[:success]}"
end

The event hash contains: :table, :query, :duration_ms (Float), :success (Boolean), :error (Exception or nil).

Executing Queries

Simple SQL

resp = client.execute_sql("baseballStats", "SELECT playerName, homeRuns FROM baseballStats LIMIT 5")

Parameterized queries

Use ? placeholders — the client substitutes and quotes values safely:

resp = client.execute_sql_with_params(
  "baseballStats",
  "SELECT playerName FROM baseballStats WHERE teamID = ? AND yearID >= ?",
  ["SFN", 2000]
)

Multistage engine

client.use_multistage_engine = true
resp = client.execute_sql("baseballStats", "SELECT teamID, count(*) FROM baseballStats GROUP BY teamID")

Or enable it upfront via ClientConfig:

config = Pinot::ClientConfig.new(
  broker_list:           ["localhost:8000"],
  use_multistage_engine: true
)
client = Pinot.from_config(config)

Per-request timeout

# Global default via config
config = Pinot::ClientConfig.new(
  broker_list:      ["localhost:8000"],
  query_timeout_ms: 5000
)

# One-off override per query
resp = client.execute_sql_with_timeout("myTable", "SELECT * FROM myTable", 3000)

Trace

client.open_trace
resp = client.execute_sql("baseballStats", "SELECT count(*) FROM baseballStats")
client.close_trace

Error Handling

All errors raised by the client inherit from Pinot::Error < StandardError:

Class	Raised when
Pinot::BrokerNotFoundError	No broker available (empty list or all offline)
Pinot::TableNotFoundError	Named table not found in broker map
Pinot::TransportError	Non-200 HTTP response from broker
Pinot::PreparedStatementClosedError	Operation on a closed prepared statement
Pinot::ConfigurationError	Invalid config (bad URL scheme, missing broker source)

Example:

begin
  resp = client.execute_sql("myTable", "SELECT * FROM myTable")
rescue Pinot::TableNotFoundError => e
  puts "Table missing: #{e.message}"
rescue Pinot::TransportError => e
  puts "Broker error: #{e.message}"
rescue Pinot::Error => e
  puts "Pinot error: #{e.message}"
end

Retry

Configure automatic retry with exponential backoff for transient errors (connection reset, timeout, HTTP 503):

config = Pinot::ClientConfig.new(
  broker_list:       ["localhost:8000"],
  max_retries:       3,
  retry_interval_ms: 200   # base interval; doubles each attempt (200ms, 400ms, 800ms)
)
client = Pinot.from_config(config)

Reading Results

execute_sql returns a Pinot::BrokerResponse. Results are in result_table:

rt = resp.result_table

rt.row_count          # => number of rows
rt.column_count       # => number of columns
rt.column_name(0)     # => "playerName"
rt.column_data_type(0)# => "STRING"

rt.get(0, 0)          # => raw cell value (Pinot::JsonNumber or String)
rt.get_string(0, 0)   # => String
rt.get_int(0, 1)      # => Integer (32-bit)
rt.get_long(0, 1)     # => Integer (64-bit)
rt.get_float(0, 2)    # => Float (32-bit range)
rt.get_double(0, 2)   # => Float (64-bit)

Stat fields on BrokerResponse:

resp.time_used_ms
resp.num_docs_scanned
resp.total_docs
resp.exceptions         # => Array<Pinot::PinotException>
resp.num_groups_limit_reached

Prepared Statements

Prepared statements are thread-safe and can be reused with different parameters.

stmt = client.prepare(
  "baseballStats",
  "SELECT playerName, sum(homeRuns) AS hr FROM baseballStats WHERE teamID = ? AND yearID >= ? GROUP BY playerName ORDER BY hr DESC LIMIT ?"
)

stmt.set_string(1, "NYA")
stmt.set_int(2, 2000)
stmt.set_int(3, 10)
resp = stmt.execute

# Or pass params inline:
resp = stmt.execute_with_params("BOS", 2000, 5)

# Reuse with new parameters:
stmt.clear_parameters
stmt.set_string(1, "LAA")
stmt.set_int(2, 2005)
stmt.set_int(3, 3)
resp = stmt.execute

stmt.close

Supported parameter types

Ruby type	SQL rendering
String	'value' (single-quote escaped)
Integer	42
Float	3.14
TrueClass / FalseClass	true / false
BigDecimal	'1234567890'
Time	'2023-01-01 12:00:00.000'

Running Tests

Unit tests

bundle install
bundle exec rspec spec/pinot/

Integration tests (requires a running Pinot cluster)

docker run -d --name pinot-quickstart -p 8000:8000 \
  apachepinot/pinot:1.5.0 QuickStart -type BATCH

# wait ~2 minutes for the cluster to load data, then:
bundle exec rspec spec/integration/ --format documentation

Environment variables:

Variable	Default	Description
BROKER_HOST	127.0.0.1	Pinot broker hostname
BROKER_PORT	8000	Pinot broker HTTP port

Releases

Publishing is explicit; pushes to main do not release a gem. Run the Release Decision workflow (release-decision.yml) with release_type set to patch, minor, or major to bump the selected branch's current version and release it after the Ruby matrix and Pinot integration tests pass. ref defaults to main. The default release_type: none records the decision without changing a version, tag, or package:

gh workflow run release-decision.yml -f release_type=none -f ref=main

The existing Release workflow (release.yml) still accepts an explicit tag and ref, including prerelease tags. Choose either entry point for a release; do not dispatch both for the same version.

Changelog

See CHANGELOG.md for a full history of releases.

License

Apache License 2.0 — see LICENSE.