org.apache.sysds.runtime.transform.tokenize

Interface TokenizerPre

All Superinterfaces:

Serializable

All Known Implementing Classes:

TokenizerPreNgram, TokenizerPreWhitespaceSplit

public interface TokenizerPre
extends Serializable

Method Summary

Modifier and Type	Method and Description
List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens>	tokenizePre(FrameBlock in)

Method Detail

tokenizePre

List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tokenizePre(FrameBlock in)