1. class
  2. Latte
  3. \
  4. Tokenizer

Tokenizer

class Tokenizer

internal  
 

Simple lexical analyser.

Constants

VALUE

OFFSET

TYPE

Methods

__construct(array $patterns, string $flags = '')

No description

array
tokenize(string $input)

Tokenizes string.

static array
getCoordinates(string $text, int $offset)

Returns position of token in input string.

Details

at line 37
__construct(array $patterns, string $flags = '')

No description

Parameters

array $patterns

of [(int) symbol type => pattern]
string $flags

regular expression flag

at line 48
array tokenize(string $input)

Tokenizes string.

Parameters

string $input

Return Value

array

at line 82
static array getCoordinates(string $text, int $offset)

Returns position of token in input string.

Parameters

string $text
int $offset

Return Value

array

Traits

Strict

Better OOP experience.