Integrating this AI system for large-scale use was a challenge - training set and user base size both hugely exceeded the original academic use-case.
The translation time for a single sentence initially exceeded one minute in the industrial scenario, causing huge productivity problems for Translated.
I optimized the underlying C++ implementation of the language and translation models and achieved a speedup of 3x.
Finally, the team achieved a total speedup of about 20x when combined with other LM optimizations.