WikipediaProcessor (MaryTTS 5.2 API)

java.lang.Object
- marytts.tools.dbselection.WikipediaProcessor

```
public class WikipediaProcessor
extends Object
```
WikipediaProcessor This program processes one by one the xml files split with wikipediaDumpSplitter. Each xml file is converted to an sql source file with mwdumper-2008-04-13.jar (org.mediawiki.dumper.Dumper) The tables names in the sql source are prefixed with the local (ex. en_US, de etc.) Each sql source is loaded in a mysql database, basically the tables local_text, local_page and local_revision are loaded. Once the tables are loaded the WikipediMarkupCleaner is used to extract clean text and a wordList, as a result two tables will be created in the database: local_cleanText and local_wordList (the wordList is also saved in a file).

Author:

Marcela Charfuelan.

Constructor Summary

Constructors
Constructor and Description

WikipediaProcessor()

Constructors
Constructor and Description
`WikipediaProcessor()`

Method Summary

Methods
Modifier and Type	Method and Description
`boolean`	`getDebug()`
`boolean`	`getDeleteCleanTextTable()`
`String`	`getListFile()`
`boolean`	`getLoadWikiTables()`
`String`	`getLocale()`
`int`	`getMaxTextLength()`
`int`	`getMinPageLength()`
`int`	`getMinTextLength()`
`String`	`getMysqlDB()`
`String`	`getMysqlHost()`
`String`	`getMysqlPasswd()`
`String`	`getMysqlUser()`
`String`	`getPageFile()`
`String`	`getRevisionFile()`
`String`	`getTestId()`
`String`	`getTextFile()`
`String`	`getWikiLog()`
`static void`	`main(String[] args)`
`void`	`setDebug(boolean bval)`
`void`	`setDeleteCleanTextTable(boolean bval)`
`void`	`setListFile(String str)`
`void`	`setLoadWikiTables(boolean bval)`
`void`	`setLocale(String str)`
`void`	`setMaxTextLength(int val)`
`void`	`setMinPageLength(int val)`
`void`	`setMinTextLength(int val)`
`void`	`setMysqlDB(String str)`
`void`	`setMysqlHost(String str)`
`void`	`setMysqlPasswd(String str)`
`void`	`setMysqlUser(String str)`
`void`	`setPageFile(String str)`
`void`	`setRevisionFile(String str)`
`void`	`setTestId(String str)`
`void`	`setTextFile(String str)`
`void`	`setWikiLog(String str)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail
- WikipediaProcessor
```
public WikipediaProcessor()
```

Method Detail

setLocale
```
public void setLocale(String str)
```

setMysqlHost
```
public void setMysqlHost(String str)
```

setMysqlDB
```
public void setMysqlDB(String str)
```

setMysqlUser
```
public void setMysqlUser(String str)
```

setMysqlPasswd

public void setMysqlPasswd(String str)

setListFile
```
public void setListFile(String str)
```

setTextFile
```
public void setTextFile(String str)
```

setPageFile
```
public void setPageFile(String str)
```

setRevisionFile

public void setRevisionFile(String str)

setWikiLog
```
public void setWikiLog(String str)
```

setTestId
```
public void setTestId(String str)
```

setMinPageLength

public void setMinPageLength(int val)

setMinTextLength

public void setMinTextLength(int val)

setMaxTextLength

public void setMaxTextLength(int val)

setDebug
```
public void setDebug(boolean bval)
```

setLoadWikiTables

public void setLoadWikiTables(boolean bval)

setDeleteCleanTextTable

public void setDeleteCleanTextTable(boolean bval)

getLocale
```
public String getLocale()
```

getMysqlHost
```
public String getMysqlHost()
```

getMysqlDB
```
public String getMysqlDB()
```

getMysqlUser
```
public String getMysqlUser()
```

getMysqlPasswd
```
public String getMysqlPasswd()
```

getListFile
```
public String getListFile()
```

getTextFile
```
public String getTextFile()
```

getPageFile
```
public String getPageFile()
```

getRevisionFile
```
public String getRevisionFile()
```

getWikiLog
```
public String getWikiLog()
```

getTestId
```
public String getTestId()
```

getMinPageLength
```
public int getMinPageLength()
```

getMinTextLength
```
public int getMinTextLength()
```

getMaxTextLength
```
public int getMaxTextLength()
```

getDebug
```
public boolean getDebug()
```

getLoadWikiTables
```
public boolean getLoadWikiTables()
```

getDeleteCleanTextTable

public boolean getDeleteCleanTextTable()

main

public static void main(String[] args)
                 throws Exception

Throws:: Exception

Class WikipediaProcessor

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

WikipediaProcessor

Method Detail

setLocale

setMysqlHost

setMysqlDB

setMysqlUser

setMysqlPasswd

setListFile

setTextFile

setPageFile

setRevisionFile

setWikiLog

setTestId

setMinPageLength

setMinTextLength

setMaxTextLength

setDebug

setLoadWikiTables

setDeleteCleanTextTable

getLocale

getMysqlHost

getMysqlDB

getMysqlUser

getMysqlPasswd

getListFile

getTextFile

getPageFile

getRevisionFile

getWikiLog

getTestId

getMinPageLength

getMinTextLength

getMaxTextLength

getDebug

getLoadWikiTables

getDeleteCleanTextTable

main