Uses of Class
org.apache.sysds.runtime.transform.tokenize.Token

Packages that use Token

Package	Description
org.apache.sysds.runtime.transform.tokenize
org.apache.sysds.runtime.transform.tokenize.builder

Uses of Token in org.apache.sysds.runtime.transform.tokenize

Fields in org.apache.sysds.runtime.transform.tokenize with type parameters of type Token

Modifier and Type	Field	Description
List<Token>	DocumentRepresentation.tokens

Methods in org.apache.sysds.runtime.transform.tokenize that return types with arguments of type Token

Modifier and Type	Method	Description
List<Token>	DocumentRepresentation.getTokens()

Constructor parameters in org.apache.sysds.runtime.transform.tokenize with type arguments of type Token

Constructor	Description
Token(List<Token> subList)

Uses of Token in org.apache.sysds.runtime.transform.tokenize.builder

Methods in org.apache.sysds.runtime.transform.tokenize.builder that return types with arguments of type Token

Modifier and Type	Method	Description
List<Token>	TokenizerBuilderNgram.splitIntoNgrams(Token token, int minGram, int maxGram)
List<Token>	TokenizerBuilderWhitespaceSplit.splitToTokens(String text)

Methods in org.apache.sysds.runtime.transform.tokenize.builder with parameters of type Token

Modifier and Type	Method	Description
List<Token>	TokenizerBuilderNgram.splitIntoNgrams(Token token, int minGram, int maxGram)