class Tokenizer

Simple lexical analyser.

Methods

__construct(array $patterns, string $flags = '')

No description

tokenize(string $input)

Tokenizes string.

static array
getCoordinates(string $text, int $offset)

Returns position of token in input string.

Details

at line 29
__construct(array $patterns, string $flags = '')

Parameters

array $patterns of [(int|string) token type => (string) pattern]
string $flags regular expression flags

at line 40
Stream tokenize(string $input)

Tokenizes string.

Parameters

string $input

Return Value

Stream

Exceptions

Exception

at line 74
static array getCoordinates(string $text, int $offset)

Returns position of token in input string.

Parameters

string $text
int $offset

Return Value

array of [line, column]