Viet-Anh on Software Logo

What is: Neural Oblivious Decision Ensembles?

SourceNeural Oblivious Decision Ensembles for Deep Learning on Tabular Data
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Neural Oblivious Decision Ensembles (NODE) is a tabular data architecture that consists of differentiable oblivious decision trees (ODT) that are trained end-to-end by backpropagation.

The core building block is a Neural Oblivious Decision Ensemble (NODE) layer. The layer is composed of mm differentiable oblivious decision trees (ODTs) of equal depth dd. As an input, all mm trees get a common vector xRnx \in \mathbb{R}^{n}, containing nn numeric features. Below we describe a design of a single differentiable ODT.

In its essence, an ODT is a decision table that splits the data along dd splitting features and compares each feature to a learned threshold. Then, the tree returns one of the 2d2^{d} possible responses, corresponding to the comparisons result. Therefore, each ODT is completely determined by its splitting features fRdf \in \mathbb{R}^{d}, splitting thresholds bRdb \in \mathbb{R}^{d} and a dd-dimensional tensor of responses RR2×2×2dR \in \mathbb{R} \underbrace{2 \times 2 \times 2}_{d}. In this notation, the tree output is defined as:

h(x)=R[1(f_1(x)b1),,1(f_d(x)b_d)]h(x)=R\left[\mathbb{1}\left(f\_{1}(x)-b_{1}\right), \ldots, \mathbb{1}\left(f\_{d}(x)-b\_{d}\right)\right]

where 1()\mathbb{1}(\cdot) denotes the Heaviside function.