What is: Neural Oblivious Decision Ensembles?

Neural Oblivious Decision Ensembles (NODE) is a tabular data architecture that consists of differentiable oblivious decision trees (ODT) that are trained end-to-end by backpropagation.

The core building block is a Neural Oblivious Decision Ensemble (NODE) layer. The layer is composed of $m$ differentiable oblivious decision trees (ODTs) of equal depth $d$. As an input, all $m$ trees get a common vector $x \in \mathbb{R}^{n}$, containing $n$ numeric features. Below we describe a design of a single differentiable ODT.

In its essence, an ODT is a decision table that splits the data along $d$ splitting features and compares each feature to a learned threshold. Then, the tree returns one of the $2^{d}$ possible responses, corresponding to the comparisons result. Therefore, each ODT is completely determined by its splitting features $f \in \mathbb{R}^{d}$, splitting thresholds $b \in \mathbb{R}^{d}$ and a $d$-dimensional tensor of responses $R \in \mathbb{R} \underbrace{2 \times 2 \times 2}_{d}$. In this notation, the tree output is defined as:

where $\mathbb{1}(\cdot)$ denotes the Heaviside function.