Leveraging Common Structure to Improve Prediction across Related Datasets

In many applications, training data is provided in the form of related datasets obtained from several sources, which typically affects the sample distribution. The learned classification models, which are expected to perform well on similar data coming from new sources, often suffer due to bias introduced by what we call ‘spurious’ samples – those due to source characteristics and not representative of any other part of the data. As standard outlier detection and robust classification usually fall short of determining groups of spurious samples, we propose a procedure which identifies the common structure across datasets by minimizing a multi-dataset divergence metric, increasing accuracy for new datasets. Problem statement Often, the data available for learning is collected from different sources, making it likely that the differences between these groups break typical assumptions such as the samples being independent and identically distributed. It is often the case that da...

• Type: Journal
• Year: 2015
• Where: AAAI
• Authors: Matt Barnes, Nick Gisolfi, Madalina Fiterau, Artur Dubrawski
• Keywords: AAAI 2015, Intelligent Agents