Interface TokensProvider

A "manual" provider of tokens.

Hierarchy

  • TokensProvider

Methods

  • The initial state of a language. Will be the state passed in to tokenize the first line.

    Returns IState

  • Tokenize a line given the state at the beginning of the line.

    Parameters

    Returns ILineTokens