Overview of a couple of projects that involved transformer encoder and decoder recombination, analysis and exploitation. This includes modular and partially-shared encoders for machine translation, multilinguality analysis of BERT and GPT-like models as well as experiments on porting success of English large LMs to Estonian.
Mark Fišel is head of NLP at University of Tartu (Estonia) with focus on machine translation, speech synthesis, large language model usage, tuning and analysis, etc.