6.2 Understanding Fusions

BigML fusions combine multiple Machine Learning models, ensembles, logistic regressions, and deepnets and average their predictions to balance out the individual weaknesses of the single models and yield a better performance. Fusions are based on the assumption that the combination of multiple models often outperfoms the component single models. We can see fusions as an heterogeneous ensemble composed by different types of models instead of just decision trees.

For fusions to improve upon the individual model performance, the component models have to be as accurate and diverse as possible. If you use several identical models or models with sub-par performance, the fusion will not be able to improve the results of the models. Although for many cases the gains in performance may not be huge, fusions have other advantages such as the fact that they are usually more stable than single models and the model errors tend to be smoothed out across the entire input space. For problems in which every small gain in model performance measures counts, fusions can be a quick solution because they are so easy to execute on BigML.

Fusions can solve Classification and Regression problems on BigML. For classification problems, fusions average the per-class probabilities across all the component models. The class with the highest probability is predicted. For regression models, the final prediction is the result of averaging the per-model predicted values. These methods to combine single model predictions and return an output is equivalent to the ensembles “probability” method (see Combine single tree predictions: probability, confidence or votes ).

You need at least one existing Supervised learning model to create a fusion. The component models of a fusion must have a compatible categorical or numeric objective field (see subsection 6.2.1 ). The models can be built using different datasets and different input fields. If two fields from different models have the same name, they will be considered the same field to create the fusion. The per-field importances from the components models will be average to create the fusion field importances (see subsection 6.2.2 ).

6.2.1 Fusion objective field

All the models composing the fusion must have the same Objective Field. BigML checks that two models have the same objective field by ensuring the objectives have the same field optype (numeric or categorical) and the same name (if you create a fusion from the BigML API the field ID instead of the name will be used to validate the compatibility of two objective fields). You can select models with different objective field names (see subsection 6.4.2 ) if they are compatible.

6.2.2 Fusion field importances

The fusion field importances are calculated by averaging the per-field importances of the following component models: decision trees, ensembles and deepnets. These averages are normalized so the sum of all field importances is 100%. Logistic regressions are excluded from this calculation since the field importances cannot be calculated for them. If the fusion is only composed by logistic regression models, it will not have importances.