Class: Tokenizer

tokenizer. Tokenizer

new Tokenizer(template, grammar)

Represents a Tokenizer.
Parameters:
Name Type Description
template string The template string to tokenize.
grammar array The specified grammar to test against.
Source:

Methods

(static) Tokenizer#parse() → {array}

Parses the template into a series of tokens.
Source:
Returns:
The series of tokens.
Type
array