kuhumcst/taggerXML
Modernized version of Eric Brill's Part Of Speech tagger.
Converts UTF-16 (BE/LE), UTF-32 (BE/LE), ISO-8859-N to UTF-8. Removes BOM and surrogate pairs from UTF-8, converting a codepoint between U-D800 and U-DBFF followed by a codepoint between U-DC00 and U-DFFF to one valid codepoint > U-FFFF.
This repository is cataloged as part of our automated global GitHub synchronization. Full telemetry, velocity snapshots, and code summaries are scheduled for continuous enrichment.
Modernized version of Eric Brill's Part Of Speech tagger.
Text segmenter and tokeniser for Danish, English and other languages. Reads an RTF or flat text file and outputs the text, one line per sentence & optionally tokenized.
Parse sgml, html and xml in a forgiving way.
Functions for upper/lower casing, for testing whether a character is a letter and for conversion between Unicode encodings UTF-8 and UTF-16