LinearTrees.jl

LinearTrees fits decision trees with linear models along their paths. A tree can represent both smooth trends and abrupt changes. Gradient boosting combines these trees into an ensemble. A separate continuous-tree model joins multivariate leaf polynomials and provides conditional predictive uncertainty.

Use the package for regression, classification, count prediction, and quantile regression on tabular data. It supports frequency weights, categorical features, exact and approximate split search, local coefficients, and SHAP values.

Installation

LinearTrees requires Julia 1.10 or later. Install the registered package in your project environment:

using Pkg
Pkg.add("LinearTrees")

Continuous trees, leaf refits, and pruning require version 0.0.2 or later. Until that version is registered, install the development version with Pkg.add(url = "https://github.com/BJMCox/LinearTrees.jl").

A first model

Rows are observations and columns are features. Fit with fit_tree and evaluate new rows with predict:

using LinearTrees, Random

rng = Xoshiro(42)
X = rand(rng, 300, 3)
y = 2 .* X[:, 1] .- X[:, 2] .+ 3 .* (X[:, 3] .> 0.6)

tree = fit_tree(X, y; max_depth = 4)
Xnew = rand(rng, 5, 3)
round.(predict(tree, Xnew); digits = 3)

5-element Vector{Float64}:
  1.679
  2.074
  4.448
  3.499
 -0.244

The default loss is squared error. Choose another loss for probabilities, counts, positive responses, or conditional quantiles. Use fit_boost for an ensemble.

Learn the package

Goal	Page
Fit and assess your first model	Getting started
Understand a tree and control its size	Tree fitting
Refit fixed regions with multivariate linear models	Ridge leaf refits
Choose splits using linear child models	Model-based splits
Prune a ridge-leaf tree using validation data	Validation pruning
Train an ensemble with validation	Boosting
Fit a globally continuous regression tree	Continuous trees
Average uncertainty across continuous geometries	Continuous model averaging
Match the loss to your target	Loss functions
Explain fitted predictions	Interpretation
Use tables, MLJ, or saved models	Interfaces and persistence
Choose split search and threading	Performance
Look up a function or type	API reference

Model scope

Tree growth follows the PILOT approach, extended here to several losses and boosted ensembles. Linear terms are fitted one feature at a time. A prediction can involve several features because it sums terms along a path.

PILOT trees and boosted ensembles return point predictions, class probabilities, or conditional quantiles according to the loss. They do not provide posterior distributions. ContinuousTree instead provides a conjugate Student-t predictive distribution conditional on its selected model and preprocessing. ContinuousEnsemble averages over a supplied finite set of geometries. Ridge leaf refits and model-based splits fit multivariate affine terminal models under squared error. prune_refit optionally replaces their subtrees using validation loss. The package does not provide missing-value imputation. Class probabilities do not carry a calibration guarantee.

The original algorithm is described by Raymaekers, Rousseeuw, Verdonck, and Yao (2024), Fast linear model trees by PILOT, Machine Learning 113, 6561–6610.

License and support

Report bugs and request features through the issue tracker.