Represents a lexer that tokenizes input text. The Lexer takes the original expression string and returns an array of tokens parsed from that string. For example, the string "a + b" would result in tokens for a, +, and b.
Creates an instance of Lexer.
Lexer options.
Converts a character to its Unicode code point.
Character to convert.
Unicode code point.
Checks if a character is contained in a set of characters.
Character to check.
Set of characters.
True if character is in the set, false otherwise.
Checks if a character is an exponent operator.
True if character is an exponent operator, false otherwise.
Checks if a character is a valid identifier continuation.
True if character is a valid identifier continuation, false otherwise.
Checks if a character is a valid identifier start.
True if character is a valid identifier start, false otherwise.
Checks if a character is a number.
True if character is a number, false otherwise.
Checks if a character is whitespace.
True if character is whitespace, false otherwise.
Tokenizes the input text.
Input text to lex.
Array of tokens.
Peeks at the next character in the text.
Optional
Number of characters to peek.
Next character or false if end of text.
Peeks at the next multicharacter sequence in the text.
Next multicharacter sequence.
Reads and tokenizes an identifier from the text.
Reads and tokenizes a number from the text.
Reads and tokenizes a string from the text.
Quote character used for the string.
Throws a lexer error.
Error message.
Start index.
End index.
Lexer error.
Represents a lexer that tokenizes input text. The Lexer takes the original expression string and returns an array of tokens parsed from that string. For example, the string "a + b" would result in tokens for a, +, and b.