Figure 3 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Download
View publication
Validation and training losses of KL-Norm for varying β and a fixed bottleneck size on GLUE.

Validation and training losses of KL-Norm for varying β and a fixed bottleneck size on GLUE.

Source publication
Figure 1. Architecture of proposed KL Regularized normalization framework
Figure 2. Validation and training losses of KL-Norm for varying β and a...
Figure 3. Validation and training losses of KL-Norm for varying β and a...
Average results and standard deviation in parentheses over 5 runs on...
+1 Test accuracy of models transferring to new target datasets. All models...
KL Regularized Normalization Framework for Low Resource Tasks
Preprint
Full-text available
  • Dec 2022
Large pre-trained models, such as Bert, GPT, and Wav2Vec, have demonstrated great potential for learning representations that are transferable to a wide variety of downstream tasks . It is difficult to obtain a large quantity of supervised data due to the limited availability of resources and time. In light of this, a significant amount of research...
Cite
Download full-text

Similar publications

Figure 2. Across 11 CelebA attributes, the FPR gender difference of...
Figure 4. The performance and bias with 95% confidence intervals of...
Figure 8. The performance and fairness with 95% confidence intervals of...
The top table represents when the finetuning number is 128, and the...
Overcoming Bias in Pretrained Models by Manipulating the Finetuning Dataset
Preprint
Full-text available
  • Mar 2023
Transfer learning is beneficial by allowing the expressive features of models pretrained on large-scale datasets to be finetuned for the target task of smaller, more domain-specific datasets. However, there is a concern that these pretrained models may come with their own biases which would propagate into the finetuned model. In this work, we inves...
View