Sms2fr
Translate text message abbreviations to actual french.
This program is based on Rewriting the orthography of SMS messages, François Yvon, In Natural Language Engineering, volume 16, 2010. It uses two specific automata created using a machine learning method.
The first one is used to add the different syllable with a similar sound as those from the text (graphemic model).
The second one treats the probabilities of those changes considering the syntax of the text (syntactic model).
Notes:
The following example does not rely on actual weights and labels from the algorithm and these values were chosen as theoritical values.
The characters '#', '[' and ']' are considered special characters by the trained automata and not accepted in the original text.
References:
The method
Considering we enter 'bo' in the sms2fr
algorithm, the following automaton is created:
It is then composed with the graphemic model (weights will depend on frequency of usage in the language):
And it is finally composed with the syntactic model (weights will depend on probability of presence of this grapheme after the letter b):
The algorithm will then choose the path with the lightest weight (in the tropical weightset Rmin), here: 'beau'.
Initialization
Unfortunately, for lack of a binary format for automata, loading these automata takes about ten seconds.
Core algorithm
Examples
It is also possible to get multiple traduction proposition by using Vcsn's implementations of K shortest path algorithms.