Skip to content

SQLite: parse the full PRAGMA value grammar - #2469

Open
LucaCappelletti94 wants to merge 5 commits into
apache:mainfrom
LucaCappelletti94:sqlite-pragma-boolean-values
Open

LucaCappelletti94 wants to merge 5 commits into
apache:mainfrom
LucaCappelletti94:sqlite-pragma-boolean-values

Conversation

@LucaCappelletti94

@LucaCappelletti94 LucaCappelletti94 commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

PRAGMA values such as journal_mode = WAL, synchronous = NORMAL, cache_size = -2000 and optimize = 0x10002 were rejected. The earlier boolean handling also kept its own keyword list plus a value-type allowlist, and that duplicated allowlist is what rejected PRAGMA case_sensitive_like = true in the first place.

parse_pragma_value now follows SQLite's own definition of a pragma value, signed-number | name | signed-literal, and stores the result as an Expr. A bare name like WAL is kept as an identifier, so the parser no longer decides which value a given pragma accepts and leaves that to execution the way SQLite does. parse_pragma builds the statement in a single place.

This changes Statement::Pragma.value from Option<ValueWithSpan> to Option<Expr>.

@LucaCappelletti94
LucaCappelletti94 marked this pull request as ready for review August 31, 2026 13:28
Comment thread src/parser/mod.rs Outdated
Comment on lines +20170 to +20197
let span = self.peek_token_ref().span;
let v = match self.parse_one_of_keywords(&[
Keyword::TRUE,
Keyword::FALSE,
Keyword::ON,
Keyword::OFF,
Keyword::YES,
Keyword::NO,
]) {
Some(keyword) => Value::Boolean(matches!(
keyword,
Keyword::TRUE | Keyword::YES | Keyword::ON
))
.with_span(span),
None => self.parse_value()?,
};
match &v.value {
Value::SingleQuotedString(_) => Ok(v),
Value::DoubleQuotedString(_) => Ok(v),
Value::Number(_, _) => Ok(v),
Value::Boolean(_) => Ok(v),
Value::Placeholder(_) => Ok(v),
_ => {
self.prev_token();
self.expected_ref("number or string or ? placeholder", self.peek_token_ref())
self.expected_ref(
"boolean, number, string, or ? placeholder",
self.peek_token_ref(),
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could simplify the logic to have it accept whatever parse_value returns. and keep the sqlite semantics for validation upstream? that would have avoided this bug in the initial version

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked SQLite docs hoping for a limited set of allowed values, it states there is explicitly no such set, and I therefore switched to Expr

@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch from 0fa5f24 to 4dfcf28 Compare September 7, 2026 05:19
@LucaCappelletti94 LucaCappelletti94 changed the title Parse SQLite boolean PRAGMA values SQLite: parse the full PRAGMA value grammar Sep 7, 2026
@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch from 4dfcf28 to 582c4e4 Compare September 7, 2026 05:23
Comment thread src/parser/mod.rs
Comment on lines +20162 to 20175
if matches!(self.peek_token_ref().token, Token::Plus | Token::Minus) {
let op = match self.next_token().token {
Token::Plus => UnaryOperator::Plus,
_ => UnaryOperator::Minus,
};
return Ok(Expr::UnaryOp {
op,
expr: Box::new(Expr::Value(self.parse_value()?)),
});
}
match self.maybe_parse(|parser| parser.parse_value())? {
Some(value) => Ok(Expr::Value(value)),
None => Ok(Expr::Identifier(self.parse_identifier()?)),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm can this be replaced by a call to parse_expr()?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In terms of accepting things, yes it would work, but parse_expr() would over-extend the grammar accepting arbitrary expressions like PRAGMA p = 1 + 2, foo(bar), a AND b or (SELECT 1).

SQLite defines a pragma value as signed-number | name | signed-literal, so parse_pragma_value mirrors that production exactly while still leaving the per-pragma value validation to execution.

I understand I tend to lean much more strongly on the strict parser approach, but here I really believe allowing any expr would be excessively loose.

@codecov-commenter

codecov-commenter commented Sep 24, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.75000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.53%. Comparing base (b22bf61) to head (ef53be5).

Files with missing lines Patch % Lines
src/parser/mod.rs 92.30% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2469      +/-   ##
==========================================
+ Coverage   82.52%   82.53%   +0.01%     
==========================================
  Files          42       42              
  Lines       35533    35537       +4     
  Branches    35533    35537       +4     
==========================================
+ Hits        29323    29331       +8     
+ Misses       2675     2674       -1     
+ Partials     3535     3532       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch from e0cf34f to 1c2fb43 Compare October 2, 2026 16:14
@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch 3 times, most recently from e084e16 to 55d496f Compare October 23, 2026 22:21
@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch from 984bac7 to 0ae34c4 Compare October 24, 2026 11:34
@LucaCappelletti94 LucaCappelletti94 added enhancement New feature or request waiting on reviewer PR needs to be reviewed labels Oct 24, 2026
@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch 4 times, most recently from f41c3f2 to 8e0ba2a Compare October 26, 2026 18:58
@LucaCappelletti94
LucaCappelletti94 force-pushed the sqlite-pragma-boolean-values branch from 8e0ba2a to ef53be5 Compare October 29, 2026 18:16
Comment thread src/parser/mod.rs
) {
return self.expected_ref("number, name or string", self.peek_token_ref());
}
match self.try_parse(|parser| parser.parse_value()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be smt like this?

// A bare word that is not a literal (`WAL`, `NORMAL`) is a pragma name.
          if let Token::Word(_) = self.peek_token_ref().token {
              if let Some(value) = self.maybe_parse(|parser| parser.parse_value())? {
                  return Ok(Expr::Value(value));
              }
              return Ok(Expr::Identifier(self.parse_identifier()?));
          }
          Ok(Expr::Value(self.parse_value()?))

Current approach falls back to parse_identifier on any error, so real parse_value errors are hidden.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request SQLite waiting on reviewer PR needs to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants