a processor for text data where the datalines are pointers to text files, does normalization More...
Inheritance diagram for nabu.processing.processors.textfile_processor.TextFileProcessor:
Public Member Functions | |
| def | __init__ (self, conf) |
| TextProcessor constructor. More... | |
| def | __call__ (self, dataline) |
| process the data in dataline More... | |
| def | write_metadata (self, datadir) |
| write the processor metadata to disk More... | |
 Public Member Functions inherited from nabu.processing.processors.processor.Processor | |
| def | __init__ (self, conf) |
| Processor constructor. More... | |
| def | __call__ (self, dataline) |
| process the data in dataline More... | |
| def | write_metadata (self, datadir) |
| write the processor metadata to disk More... | |
Public Attributes | |
| normalizer | |
| alphabet | |
| max_length | |
| sequence_length_histogram | |
| nonesymbol | |
| Public Attributes inherited from nabu.processing.processors.processor.Processor | |
| conf | |
Detailed Description
a processor for text data where the datalines are pointers to text files, does normalization
Constructor & Destructor Documentation
§ __init__()
| def nabu.processing.processors.textfile_processor.TextFileProcessor.__init__ | ( | self, | |
| conf | |||
| ) |
TextProcessor constructor.
- Parameters
-
conf processor configuration as a configparser
Member Function Documentation
§ __call__()
| def nabu.processing.processors.textfile_processor.TextFileProcessor.__call__ | ( | self, | |
| dataline | |||
| ) |
process the data in dataline
- Parameters
-
dataline a line of text
- Returns
- The normalized text as a string
§ write_metadata()
| def nabu.processing.processors.textfile_processor.TextFileProcessor.write_metadata | ( | self, | |
| datadir | |||
| ) |
write the processor metadata to disk
- Parameters
-
dir the directory where the metadata should be written
The documentation for this class was generated from the following file:
