Modular Domain Adaptation

Schematic diagram for modular domain adaptation
Average out-of-domain accuracy of various methods on four multi-domain datasets
Average performance drop of baseline models when going from in-domain to out-of-domain data
Comparing logistic regression (with and without DsNorm and DsBias) to various off-the-shelf lexicons
  • Although linear models may still be preferred for reasons of simplicity, interpretability, or minimizing bias, contextual embedding models had much higher out-of-domain accuracy on all the tasks we considered; thus, the computational social science community would benefit greatly if model developers would release both linear and contextual embedding models when they are unable to share their training data.
  • By using the kinds of modular domain adaptation techniques we propose in the paper, model developers can facilitate domain adaptation by model consumers, thereby enabling more accurate measurements in new domains, and we hope that this work will encourage the development of other techniques that work for this kind of modular setup.
  • Domain fine-tuning remains a highly effective way to do domain adaptation with a small amount of labeled data, but may not be a possibility when working with commercial models or cloud-based APIs.
  • The accuracies of a variety of sentiment lexicons (both historical and contemporary) were surprisingly similar on a variety of sentiment analysis datasets; even when tuning to the application domain, none did better in terms of out-of-domain accuracy than a basic bag-of-words logistic regression model deployed in combination with our lightweight adaptation techniques.
  • In the end, transparency is hugely beneficial, and we strongly encourage model developers to release both the code and data for their models whenever possible, both for reproducibility, and to enable more powerful domain adaptation techniques.

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Visual Attribute Transfer through Deep Image Analogy

Recommender Systems using Deep Learning in PyTorch from scratch

What is machine learning?

Advanced Vision Algorithm Helps Robots Learn to See in 3D

Sentiment analysis using NLTK.

Your brAIn’s all muddled up!

On Writing Custom Loss Functions in Keras

ReLU Activation Function

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dallas Card

Dallas Card

More from Medium

One Platform to Supercharge your Supply Chain: Glassbox Methodology

Transformer Map กรณีศึกษา Forth บุญเติม สู่ เต่าบิน

What Does the Future Hold for AI and Programming?