[chiliec]

Fixes #380.

Problem

_consume_quoted_identifier, _consume_literal, and _consume_raw_string_literal set the token's end to token_len (the token length) instead of start + token_len. For any such token that is not at offset 0, end is smaller than start:

>>> from jmespath.lexer import Lexer
>>> [t for t in Lexer().tokenize('foo."q"') if t['type'] == 'quoted_identifier']
[{'type': 'quoted_identifier', 'value': 'q', 'start': 4, 'end': 2}]

Every other token type already uses start + len(...).

Fix

Set end to start + token_len in all three places. Search results are unaffected; only the position metadata changes.

>>> [t for t in Lexer().tokenize('foo."q"') if t['type'] == 'quoted_identifier']
[{'type': 'quoted_identifier', 'value': 'q', 'start': 4, 'end': 6}]

Tests

Added test_position_quoted_identifier, test_position_literal_not_at_start, and test_position_raw_string_not_at_start to tests/test_lexer.py, each asserting the fixed offsets and end >= start.

Validation:

$ python -m pytest tests/test_lexer.py -q
# before the fix: the 3 new tests FAIL (end < start)
# after the fix:  22 passed

$ python -m pytest tests/ -q
994 passed, 1 skipped