Multi-expert Methods Evaluation on Financial and Economic Data: Introducing Bag of Experts

Abstract

The use of machine learning into economics scenarios results appealing since it allows for automatically testing economic models and predict consumer/client behavior to support decision-making processes. The finance market typically uses a set of expert labelers or Bureau credit scores given by governmental or private agencies such as Experian, Equifax, and Creditinfo, among others. This work focuses on introducing a so-named Bag of Expert (BoE): a novel approach for creating multi-expert Learning (MEL) frameworks aimed to emulate real experts labeling (human-given labels) using neural networks. The MEL systems “learn” to perform decision-making tasks by considering a uniform number of labels per sample or individuals along with respective descriptive variables. The BoE is created similarly to Generative Adversarial Network (GANs), but rather than using noise or perturbation by a generator, we trained a feed-forward neural network to randomize sampling data, and either add or decrease hidden neurons. Additionally, this paper aims to investigate the performance on economics-related datasets of several state-of-the-art MEL methods, such as GPC, GPC-PLAT, KAAR, MA-LFC, MA-DGRL, and MA-MAE. To do so, we develop an experimental framework composed of four tests: the first one using novice experts, the second with proficient experts, the third is a mix of novices, intermediate and proficient experts, and the last one uses crowd-sourcing. Our BoE method presents promising results and can be suitable as an alternative to properly assess the reliability of both MEL methods and conventional labeler generators (i.e., virtual expert labelers). © 2020, Springer Nature Switzerland AG.