LinearTrees.jl

LinearTrees fits decision trees with linear models along their paths. A tree can represent both smooth trends and abrupt changes. Gradient boosting combines these trees into an ensemble.

Use the package for regression, classification, count prediction, and quantile regression on tabular data. It supports frequency weights, categorical features, exact and approximate split search, local coefficients, and SHAP values.

Installation

LinearTrees requires Julia 1.10 or later. Install the development version from GitHub in your project environment:

using Pkg
Pkg.add(url = "https://github.com/BJMCox/LinearTrees.jl")

A first model

Rows are observations and columns are features. Fit with fit_tree and evaluate new rows with predict:

using LinearTrees, Random

rng = Xoshiro(42)
X = rand(rng, 300, 3)
y = 2 .* X[:, 1] .- X[:, 2] .+ 3 .* (X[:, 3] .> 0.6)

tree = fit_tree(X, y; max_depth = 4)
Xnew = rand(rng, 5, 3)
round.(predict(tree, Xnew); digits = 3)

5-element Vector{Float64}:
  1.679
  2.074
  4.448
  3.499
 -0.244

The default loss is squared error. Choose another loss for probabilities, counts, positive responses, or conditional quantiles. Use fit_boost for an ensemble.

Learn the package

Goal	Page
Fit and assess your first model	Getting started
Understand a tree and control its size	Tree fitting
Train an ensemble with validation	Boosting
Match the loss to your target	Loss functions
Explain fitted predictions	Interpretation
Use tables, MLJ, or saved models	Interfaces and persistence
Choose split search and threading	Performance
Look up a function or type	API reference

Model scope

Tree growth follows the PILOT approach, extended here to several losses and boosted ensembles. Linear terms are fitted one feature at a time. A prediction can involve several features because it sums terms along a path.

The package returns point predictions, class probabilities, or conditional quantiles according to the loss. It does not provide Bayesian posterior distributions, predictive intervals, missing-value imputation, or pruning. Class probabilities do not carry a calibration guarantee.

The original algorithm is described by Raymaekers, Rousseeuw, Verdonck, and Yao (2024), Fast linear model trees by PILOT, Machine Learning 113, 6561–6610.

License and support

Report bugs and request features through the issue tracker.