Transformer Uncertainty Estimation with Hierarchical Stochastic Attention
Date:
In this talk, I introduce the stochastic transformers, which have been published at AAAI 2022 conference and discuss about uncertain estimation in NLP/IR.Transformers are state-of-the-art in a wide range of NLP tasks and have also been applied to many real-world products. Understanding the reliability and certainty of transformer model predictions is crucial for building trustable machine learning applications, e.g., medical diagnosis. Although many recent transformer extensions have been proposed, the study of the uncertainty estimation of transformer models is under-explored. [Link]