Translation with glossary and target "EN-GB" looses some words #111

Activity

1. JanEbbing commented on Jun 26, 2024

   

   JanEbbing

   on Jun 26, 2024

   Contributor

   More actions

   Im not 100% what your use case is, but you will get the highest possible translation quality by parsing structured data like this before feeding it into the API, for example in your case:

   text_to_translate = "| \\_VOEMI            | Data emissione operazione | Deve essere maggiore o uguale alla data di emissione della polizza e minore o uguale alla data di sistema.     |"
   special_tokens = ["\\_"]
   delimiter = "|"
   translator = deepl.Translator(...)
   
   translated_texts = []
   for text in text_to_translate.split(delimiter):
       if (not text.strip()) or any(map(lambda tok: text.contains(tok), special_tokens)):
           translated_texts.append(text)
           continue
       else:
           # you might want to trim the whitespace here as well with text.trim(), and maybe
           # fill up the missing whitespace when appending to translated_texts, as this looks like a table
           translated_texts.append(translator.translate_text(text, ...).text)
   output = delimiter.join(translated_texts)

   Due to the nature of ML models, we otherwise cannot guarantee that the output is stable/preserves these kinds of tokens. You can also take a look at ignore tags as another option.
2. EnricoPicci commented on Jun 26, 2024

   

   EnricoPicci

   on Jun 26, 2024

   Author

   More actions

   Jan, thanks for your prompt response. I will implement your suggestions. At the same time it is interesting the different behaviour between "EN-GB" and "EN-US".