Class CharSequence2TokenSequence

  • All Implemented Interfaces:
    AlphabetCarrying, java.io.Serializable

    public class CharSequence2TokenSequence
    extends Pipe
    implements java.io.Serializable
    Pipe that tokenizes a character sequence. Expects a CharSequence in the Instance data, and converts the sequence into a token sequence using the given regex or CharSequenceLexer. (The regex / lexer should specify what counts as a token.)
    See Also:
    Serialized Form
    • Constructor Detail

      • CharSequence2TokenSequence

        public CharSequence2TokenSequence​(CharSequenceLexer lexer)
      • CharSequence2TokenSequence

        public CharSequence2TokenSequence​(java.lang.String regex)
      • CharSequence2TokenSequence

        public CharSequence2TokenSequence​(java.util.regex.Pattern regex)
      • CharSequence2TokenSequence

        public CharSequence2TokenSequence()
    • Method Detail

      • pipe

        public Instance pipe​(Instance carrier)
        Description copied from class: Pipe
        Really this should be 'protected', but isn't for historical reasons.
        Overrides:
        pipe in class Pipe
      • main

        public static void main​(java.lang.String[] args)