public class JTokeniser extends InternalModule
Modifier and Type | Field and Description |
---|---|
static int |
TOKEN_MAXLENGTH |
logger, state
MODULE_OFFLINE, MODULE_RUNNING
Constructor and Description |
---|
JTokeniser() |
JTokeniser(Locale locale) |
JTokeniser(MaryDataType inputType,
MaryDataType outputType,
Locale locale) |
JTokeniser(String locale) |
Modifier and Type | Method and Description |
---|---|
MaryData |
process(MaryData d)
Perform this module's processing on abstract "MaryData" input
d . |
protected void |
setTokenizerLanguage(String languageCode)
Set the tokenizer language to be different from the Locale of the module.
|
void |
startup()
Allow the module to start up, performing whatever is necessary to become operational.
|
getInputType, getLocale, getOutputType, getState, inputType, name, outputType, powerOnSelfTest, shutdown
public static final int TOKEN_MAXLENGTH
public JTokeniser()
public JTokeniser(String locale)
public JTokeniser(Locale locale)
public JTokeniser(MaryDataType inputType, MaryDataType outputType, Locale locale)
protected void setTokenizerLanguage(String languageCode)
languageCode
- the language-code to use, as a two-character string such as "de" or "en".public void startup() throws Exception
MaryModule
startup
in interface MaryModule
startup
in class InternalModule
Exception
- Exceptionpublic MaryData process(MaryData d) throws Exception
InternalModule
d
. Subclasses need to make sure that the
process()
method is thread-safe, because in server-mode, it will be called from different threads at the same
time. A sensible way to do this seems to be not to use any global or static variables, or to use them read-only.
process
in interface MaryModule
process
in class InternalModule
d
- doutputType()
encapsulating the processing result.
This method just returns its input. Subclasses should override this.
Exception
- ExceptionCopyright © 2000–2016 DFKI GmbH. All rights reserved.