Represents a lexer that tokenizes input text. The Lexer takes the original expression string and returns an array of tokens parsed from that string. For example, the string "a + b" would result in tokens for a, +, and b.

Constructors

Properties

index: number
options: LexerOptions
text: string
tokens: Token[]

Methods

  • Converts a character to its Unicode code point.

    Parameters

    • ch: string

      Character to convert.

    Returns number

    Unicode code point.

  • Checks if a character is contained in a set of characters.

    Parameters

    • ch: string

      Character to check.

    • chars: string

      Set of characters.

    Returns boolean

    True if character is in the set, false otherwise.

  • Checks if a character is an exponent operator.

    Parameters

    • ch: string

      Character to check.

    Returns boolean

    True if character is an exponent operator, false otherwise.

  • Checks if a character is a valid identifier continuation.

    Parameters

    • ch: string

      Character to check.

    Returns boolean

    True if character is a valid identifier continuation, false otherwise.

  • Checks if a character is a valid identifier start.

    Parameters

    • ch: string

      Character to check.

    Returns boolean

    True if character is a valid identifier start, false otherwise.

  • Checks if a character is a number.

    Parameters

    • ch: string

      Character to check.

    Returns boolean

    True if character is a number, false otherwise.

  • Checks if a character is whitespace.

    Parameters

    • ch: string

      Character to check.

    Returns boolean

    True if character is whitespace, false otherwise.

  • Peeks at the next character in the text.

    Parameters

    • Optionali: number

      Number of characters to peek.

    Returns string | false

    Next character or false if end of text.

  • Peeks at the next multicharacter sequence in the text.

    Returns string

    Next multicharacter sequence.

  • Reads and tokenizes a string from the text.

    Parameters

    • quote: string

      Quote character used for the string.

    Returns void

  • Throws a lexer error.

    Parameters

    • error: string

      Error message.

    • Optionalstart: number

      Start index.

    • Optionalend: number

      End index.

    Returns void

    Lexer error.