Bidirectional language model Deep Learning
noun phrase
Definition: A language model that uses both left and right context to compute contextual representations of linguistic units, rather than relying only on preceding context [Devlin et al. 2019].
Example in context: “BERT is the very first bidirectional language model that is purely based on transformer architecture.” [ Wang et al. 2023]
Synonym: BiDi language model
Related term: masked language model (common implementation)