Options
All
  • Public
  • Public/Protected
  • All
Menu

Class Tokenizer

Hierarchy

  • Tokenizer

Index

Constructors

constructor

  • new Tokenizer(code: string, config: Config): Tokenizer

Properties

buffer

buffer: IToken[]

errorHandler

errorHandler: ErrorHandler

reader

reader: Reader

scanner

scanner: Scanner

trackLoc

trackLoc: boolean

trackRange

trackRange: boolean

Methods

errors

  • errors(): Error[]

getNextToken

  • getNextToken(): IToken