Semi-Supervised Knowledge Amalgamation for Sequence Classification

Published in AAAI, 2021

In this paper, we define the open problem of Semi-Supervised Knowledge Amalgamation (SKA) and propose its very first solution, which we demonstrate using sequence classification tasks. The goal of SKA is to use a set of pre-trained teacher models and a tiny set of labeled data to train a student model that can predict any of the teachers’ classes accurately. In the paper, we demosntrate that using the tiny labeled data to learn which teachers to trust, we can train one student model that is more accurate than any of the teacher students.

  title={Semi-Supervised Knowledge Amalgamation for Sequence Classification},
  author={Thadajarassiri, Jidapa and Hartvigsen, Thomas and Kong, Xiangnan and Rundensteiner, Elke},
  booktitle={Proceedings of the 35th AAAI Conference on Artificial Intelligence},