# Class Lexer

Represents a lexer that tokenizes input text. The Lexer takes the original expression string and returns an array of tokens parsed from that string. For example, the string "a + b" would result in tokens for a, +, and b.

- Defined in src/core/parse/lexer/lexer.js:33

##### Index

### Constructors

constructor

### Properties

index
options
text
tokens

### Methods

codePointAt
handleUnicodeEscape
is
isExpOperator
isIdentifierContinue
isIdentifierStart
isNumber
isWhitespace
lex
peek
peekMultichar
readIdent
readNumber
readString
throwError

## Constructors

### constructor

- new Lexer(options: LexerOptions): Lexer

  Creates an instance of Lexer.

  #### Parameters

  - options: LexerOptions

    Lexer options.

  #### Returns Lexer

  - Defined in src/core/parse/lexer/lexer.js:38

## Properties

### index

index: number

- Defined in src/core/parse/lexer/lexer.js:50

### options

options: LexerOptions

- Defined in src/core/parse/lexer/lexer.js:40

### text

text: string

- Defined in src/core/parse/lexer/lexer.js:49

### tokens

tokens: Token[]

- Defined in src/core/parse/lexer/lexer.js:52

## Methods

### codePointAt

- codePointAt(ch: string): number

  Converts a character to its Unicode code point.

  #### Parameters

  - ch: string

    Character to convert.

  #### Returns number

  Unicode code point.

  - Defined in src/core/parse/lexer/lexer.js:171

### handleUnicodeEscape

- handleUnicodeEscape(): string

  #### Returns string

  - Defined in src/core/parse/lexer/lexer.js:330

### is

- is(ch: string, chars: string): boolean

  Checks if a character is contained in a set of characters.

  #### Parameters

  - ch: string

    Character to check.

  - chars: string

    Set of characters.

  #### Returns boolean

  True if character is in the set, false otherwise.

  - Defined in src/core/parse/lexer/lexer.js:101

### isExpOperator

- isExpOperator(ch: string): boolean

  Checks if a character is an exponent operator.

  #### Parameters

  - ch: string

    Character to check.

  #### Returns boolean

  True if character is an exponent operator, false otherwise.

  - Defined in src/core/parse/lexer/lexer.js:199

### isIdentifierContinue

- isIdentifierContinue(ch: string): boolean

  Checks if a character is a valid identifier continuation.

  #### Parameters

  - ch: string

    Character to check.

  #### Returns boolean

  True if character is a valid identifier continuation, false otherwise.

  - Defined in src/core/parse/lexer/lexer.js:156

### isIdentifierStart

- isIdentifierStart(ch: string): boolean

  Checks if a character is a valid identifier start.

  #### Parameters

  - ch: string

    Character to check.

  #### Returns boolean

  True if character is a valid identifier start, false otherwise.

  - Defined in src/core/parse/lexer/lexer.js:142

### isNumber

- isNumber(ch: string): boolean

  Checks if a character is a number.

  #### Parameters

  - ch: string

    Character to check.

  #### Returns boolean

  True if character is a number, false otherwise.

  - Defined in src/core/parse/lexer/lexer.js:122

### isWhitespace

- isWhitespace(ch: string): boolean

  Checks if a character is whitespace.

  #### Parameters

  - ch: string

    Character to check.

  #### Returns boolean

  True if character is whitespace, false otherwise.

  - Defined in src/core/parse/lexer/lexer.js:131

### lex

- lex(text: string): Token[]

  Tokenizes the input text.

  #### Parameters

  - text: string

    Input text to lex.

  #### Returns Token[]

  Array of tokens.

  - Defined in src/core/parse/lexer/lexer.js:48

### peek

- peek(i?: number): string | false

  Peeks at the next character in the text.

  #### Parameters

  - `Optional` i: number

    Number of characters to peek.

  #### Returns string | false

  Next character or false if end of text.

  - Defined in src/core/parse/lexer/lexer.js:110

### peekMultichar

- peekMultichar(): string

  Peeks at the next multicharacter sequence in the text.

  #### Returns string

  Next multicharacter sequence.

  - Defined in src/core/parse/lexer/lexer.js:180

### readIdent

- readIdent(): void

  Reads and tokenizes an identifier from the text.

  #### Returns void

  - Defined in src/core/parse/lexer/lexer.js:266

### readNumber

- readNumber(): void

  Reads and tokenizes a number from the text.

  #### Returns void

  - Defined in src/core/parse/lexer/lexer.js:225

### readString

- readString(quote: string): void

  Reads and tokenizes a string from the text.

  #### Parameters

  - quote: string

    Quote character used for the string.

  #### Returns void

  - Defined in src/core/parse/lexer/lexer.js:287

### throwError

- throwError(error: string, start?: number, end?: number): void

  Throws a lexer error.

  #### Parameters

  - error: string

    Error message.

  - `Optional` start: number

    Start index.

  - `Optional` end: number

    End index.

  #### Returns void

  #### Throws

  Lexer error.

  - Defined in src/core/parse/lexer/lexer.js:210