API reference

fit_tree(X, y, loss=MSE(); kwargs...) -> LinearTree

Fit a linear model tree to the rows of numeric matrix X and target vector y. Columns are features. Targets must satisfy the domain of loss; feature values must be finite. Use predict for response-scale predictions and score for scores on the loss scale.

Keywords

• weights=nothing: nonnegative frequency weights, with a positive total. The default gives every row weight one. Zero-weight rows are excluded.
• categorical=Int[]: columns containing positive integer category codes.
• rule=BIC(): node selection rule. See BIC and GainRule.
• split_search=ExactSearch(): numeric threshold search. BinnedSearch supports scalar or vector scores. HybridSearch supports scalar scores.
• max_depth=12: maximum branching depth. Unsplit LIN nodes add no depth.
• min_fit=10: minimum total row weight for another model-selection step.
• min_leaf=5: minimum total row weight on each side of a split.
• min_sum_hessian=1.0: stop below this total Hessian, summed over score coordinates.
• max_lin_chain=10: maximum number of consecutive unsplit linear nodes.
• truncate=true: bound score accumulation and each node's feature extrapolation.
• truncation_factor=3: parameter passed to scorebound. Must be at least one.
• features=1:size(X, 2): feature columns available for fitting. Prediction still expects the original matrix layout.
• nthreads=Threads.nthreads(): maximum number of available Julia threads to use.
• niter=5: node-refit iterations for non-smooth losses such as MAD and Quantile. Smooth losses ignore this keyword.
• presort=nothing: optional Int32 matrix of stable sorted row indices, one column per original feature. Used by exact and local-bin search.

Nodes add score increments along a path. Model selection uses a quadratic surrogate for nonquadratic losses. Huber, binary logistic, and log-link updates check true training loss before accepting a full step. No pruning pass follows growth.

For table inputs and stored encoders, use LinearTreeRegressorFit or LinearTreeClassifierFit through fit.

refit_leaves(tree, X, y; weights=nothing, features=:path,
             max_features=8, lambda=1.0, truncate=tree.truncate)

Freeze an MSE tree's partitions and fit one full-response affine model per terminal region. :path takes the first max_features distinct numeric features on each route; an explicit vector selects the same features in every region. Categorical split columns route rows but cannot be regressors.

The per-region objective is weighted squared error plus lambda times the squared slopes on weighted-RMS-standardized predictors. The intercept is not penalized. Every terminal region needs positive training weight. Zero-weight rows are excluded before checking predictors or computing feature bounds. The refit uses the source tree's final score bounds when truncate=true and clamps used leaf features to their positive-weight training extrema.

prune_refit(model, Xtrain, ytrain, Xval, yval;
            train_weights=nothing, val_weights=nothing, features=:path,
            max_features=8, lambda=1.0, tolerance=0.0)

Visit a RefitTree's reachable internal nodes bottom-up. A candidate replaces a whole subtree with a full-response affine ridge leaf fitted only on training rows reaching that node. Accept it only when its served, weighted validation squared loss plus the absolute tolerance is strictly below the current subtree's loss. Nodes without positive-weight validation rows are skipped. The returned model owns a compact topology snapshot. This operation can create discontinuities across region boundaries.

fit_model_tree(X, y; features=1:size(X, 2), max_features=8, lambda=1.0,
               max_depth=4, min_leaf=8, split_penalty=1.0,
               split_search=ExactSearch(), weights=nothing,
               truncate=true, nthreads=Threads.nthreads()) -> RefitTree

Fit a scalar-MSE tree with full-response ridge linear leaves. At each node, ExactSearch() evaluates every eligible distinct-value threshold by the combined weighted penalized ridge objective of its children. BinnedSearch() evaluates approximately equal-count bin edges and optionally refines near its best edge. features are candidate split columns; their first max_features also form the bounded leaf-regressor basis. The intercept is unpenalized. Each accepted split improves the augmented-QR-verified objective by more than split_penalty. This is greedy parametric partitioning, not a calibrated statistical significance test. nthreads limits parallel feature scans for binned searches on large nodes. Exact searches, dense binned searches, small nodes, and tree growth stay serial. Tied split selection follows the supplied feature order regardless of thread count.

fit_boost(X, y, loss = MSE(); nrounds = 100, eta = 0.1, max_depth = 5,
          min_fit = 10, min_leaf = 5, min_sum_hessian = 1.0,
          lambda_slope = 1.0, lambda_intercept = 1.0, gamma = 0.0,
          subsample = 1.0, colsample = 1.0, rng = Random.default_rng(),
          weights = nothing, categorical = Int[], truncate = true,
          Xval = nothing, yval = nothing, wval = nothing, patience = 10,
          split_search = ExactSearch(), nthreads = Threads.nthreads()) -> LinearBoost

Second-order gradient boosting (Guryanov 2019) with the linear model tree as base learner. Each round computes the gradient and Hessian of loss at the ensemble score, freezes them into a Frozen target, and fits one tree with GainRule(lambda_slope, lambda_intercept, gamma), depth max_depth, and the row and column samples drawn from rng. The tree's raw score times eta is added to the ensemble. With Xval/yval, the validation deviance is recorded per round and fitting stops after patience rounds without improvement, keeping the trees up to the best round. Exact and node-local binned searches presort X once. HybridSearch prepares global bin IDs once from all positive-weight training rows, then selects the sampled rows' IDs each round. It supports numeric features and scalar losses. Sampled-out rows have weight zero for that tree only. history holds the per-round deviance and the ensemble's validated flag says whether it is the validation deviance, and so whether early stopping was in play. It uses raw accumulated scores, before the ensemble's prediction clamp. For base losses with backtracks(loss), a round's retained tree is scaled until the training objective on those raw scores is finite and nonincreasing; validation receives the same scaled tree.

For Quantile and MAD the frozen Hessian is the IRLS weight at the ensemble score, so a round is one IRLS step, not an exact L1 fit.

fit_continuous_tree(X, y; pairs=:none, candidate_search=:full,
    max_splits=6, max_depth=4, min_leaf=8, n_thresholds=3,
    split_penalty=2.0, coefficient_precision=0.01,
    noise_shape=2.0, noise_rate=1.0, response_normalization=nothing)

Fit a ContinuousTree to finite numeric predictors, with observations in rows. A vector target gives vector predictions. An n × q target matrix gives matrix predictions with shared tree geometry and independent outputs.

Leaves use an intercept, all predictors, and the interactions selected by pairs: :none, :all, or a vector such as [(1, 2), (2, 3)]. Polynomial coefficients match across full leaf faces, including partial faces at T-junctions. Derivatives may jump.

Predictors map their training ranges to [-1, 1]. By default, responses use their training means and population standard deviations, with scale one for constant columns. Pass response_normalization=(center=..., scale=...) to fix the response transform. Use two real scalars for one output or two real vectors with one entry per output. Values are copied; scales must be positive. The coefficient prior is isotropic normal with precision coefficient_precision / σ² on an orthonormal basis of continuous raw leaf coefficients. Independently for each output, σ² has an inverse-gamma prior with noise_shape and noise_rate. All three hyperparameters must be positive. This prior depends on the tree geometry and the response units after normalization. The posterior treats centering and scaling as fixed. Choose explicit transforms independently of fitting responses to avoid estimating prior units from those responses. This does not account for adaptive tree or hyperparameter selection.

Greedy search compares single splits, paired sibling splits, and three-split crosses. It sums output log marginal likelihoods and subtracts split_penalty per split. max_splits bounds the total number of internal nodes. min_leaf bounds the observed samples in each new leaf. By default, each feature has n_thresholds equally spaced cuts in its training range. Use n_thresholds=nothing for every distinct-value midpoint. :full searches the whole candidate family at each step. :graph_pruned skips crosses outside pairs, retaining all single and sibling moves. Neither search finds a guaranteed global optimum.

Predictions extrapolate the routed leaf polynomial without clipping. predictive gives Student-t marginals conditional on the chosen tree, interaction graph and normalization, excluding model-selection uncertainty.

fit_continuous_ensemble(X, y, geometries; logprior=zeros(length(geometries)),
    coefficient_precision=0.01, noise_shape=2.0, noise_rate=1.0,
    response_normalization=nothing)

Refit each supplied continuous-tree geometry on common normalized X and y, then normalize prior-times-evidence weights across this finite model space. Entries with identical tree geometry and pair-term basis are rejected. Every geometry must use the normalization computed from X.

Responses use their fitting means and population standard deviations by default. response_normalization=(center=..., scale=...) fixes a common transform for every component, with the same contract as fit_continuous_tree. The conjugate calculation is exact conditional on fixed geometry, prior, and normalization choices. Fix these independently of fitting responses for a Bayesian model whose prior does not use those responses. Geometries selected using y make it a restricted, adaptive approximation to a posterior over all continuous trees.

predict(tree_or_ensemble, X; nthreads=Threads.nthreads())

Prediction on the response scale, linkinv(tree_or_ensemble.loss, score).

predict(tree_or_ensemble, X; nthreads=Threads.nthreads())

Softmax override: n × K matrix of class probabilities, linkinv applied row by row to the K-1-vector score.

predict(model::ContinuousTree, X; batch_size=1024)

Return posterior mean predictions. The output is a vector for a vector target and a matrix for a matrix target, with observations in rows. Outside the training range, evaluate the routed leaf polynomial without clipping. batch_size bounds temporary normalized predictor storage.

Return the posterior-model-weighted conditional means.

Response prediction for an MSE refit; equal to the clipped score.

In-place Softmax prediction into an n × K matrix.

predict!(out, model::ContinuousTree, X; batch_size=1024)

Write posterior mean predictions to out, which must match the target rank and the number of prediction rows. This method uses a temporary prediction array and supports aliasing between out and X. batch_size bounds temporary normalized predictor storage.

Write ensemble mean predictions after reading all of X, preserving aliasing.

predictive(model::ContinuousTree, X; observation=true, batch_size=1024)

Return (location, scale2, dof) for the conditional Student-t predictive marginals. observation=true includes observation noise. Use false for latent-function uncertainty. scale2 is the Student-t scale squared, not its variance. When dof > 2, variance is scale2 * dof / (dof - 2).

location and scale2 follow the target rank. dof is a scalar for a vector target and one value per output for a matrix target. Marginals condition on the selected tree, interaction graph and training normalization. They exclude model-selection uncertainty and do not describe independent draws across rows. batch_size bounds the temporary projected design and precision solve.

predictive(model::ContinuousEnsemble, X; observation=true, batch_size=1024)

Return mixture location, variance, within_variance, between_variance, posterior weights, and conditional Student-t components. The mixture is generally not Student-t. Its variance is infinite when a positive-weight component has at most two degrees of freedom.

score(tree_or_ensemble, X; clip=true, nthreads=Threads.nthreads())

Raw path sum per row on the link scale. clip=false returns the additive unclipped sum. Rows split into nthreads contiguous blocks when there are at least PARALLEL_MIN_ROWS of them; each row writes only its own output slot, so the result matches the serial loop exactly.

score(tree_or_ensemble, X; clip=true, nthreads=Threads.nthreads())

Softmax override: n × (K-1) matrix of raw reference-class logits.

Raw full-response score, optionally clipped to the source tree's score bounds.

Fitted tree. lo, hi are the stored first-truncation bounds on the score scale. base is the SHAP empty-coalition value. truncate == false disables both clamps at prediction.

RefitTree{T}

A snapshot of an MSE tree's routing geometry with one multivariate ridge predictor per terminal region. routing owns copies of the original nodes and categorical masks. Node score increments are ignored.

LinearBoost{T,V,L}

A gradient boosting ensemble of Frozen-loss linear model trees. The score is f0 + eta Σ_t score(trees[t], x), clamped to [lo, hi] (the base loss's scorebound on the training target) when truncate, then passed through the base loss's link. history[t] is the validation deviance after round t when validation data was given, else the training deviance; validated says which, so a reader can tell whether early stopping was in play. Build with fit_boost.

ContinuousTree

A continuous regression tree fitted by fit_continuous_tree. Its leaf polynomials agree on every shared face. Holding other predictors fixed therefore gives joined, piecewise linear coordinate slices for each output. Selected pair terms allow interactions, but not interactions of order three or higher. Outputs share geometry and have independent conditional posteriors.

ContinuousEnsemble

A finite mixture of continuous trees refitted on one common training set. components hold the conditional trees; weights are their posterior model probabilities under the supplied logprior.

Number of trees in the ensemble.

One node of a linear model tree. Coefficients have type V, which is the feature type T for scalar-score losses and SVector{K-1,T} for softmax. Categorical split nodes keep a packed left-level mask in the tree's mask pool at words catstart:catstart+catwords-1. Numeric nodes have catwords == 0.

Per-node model kind. LIN nodes have one child and add no depth.

Constant node: one intercept, no split.

Single line across the whole node: a x + b. One child, no added depth.

Two constants, one per side of a threshold.

Broken line, continuous at the threshold: basis [x, 1, max(x - t, 0)].

Two independent lines, one per side of a threshold.

PILOT's BIC rule: n log(dev/n) + dof(kind) log(n) with dof per kind (con, lin, pcon, blin, plin). The defaults are the paper's (1, 2, 5, 5, 7).

Test-only rule: pick the lowest surrogate deviance among kinds, never stop on con.

GainRule(; lambda_slope = 1.0, lambda_intercept = 1.0, gamma = 0.0)

Boosting's selection rule (Guryanov 2019, eq. 12 and 13). Offers con, pcon, and plin. The regularised deviance of a fit is `Σ h (z − a x − b)²

• lambdaslope a² + lambdaintercept b²`, obtained by adding the two ridges

to the Gram sums in fit_lin and fit_con. A split's score is its children's regularised deviance plus gamma per coefficient coordinate, so a split is taken exactly when its gain over the parent's constant fit exceeds gamma. BIC's dof penalty and dmin floor play no part.

Policy for choosing the numeric thresholds evaluated during tree fitting.

ExactSearch()

Evaluate every eligible numeric threshold. This is the default split search.

BinnedSearch(; nbins=64, refine=true)

Approximate numeric split search for scalar or vector scores. Build at most nbins equal-count bins within each node, keeping tied values together. Evaluate bin edges using sufficient statistics of the original feature values and responses. Weights retain their frequency meaning. Categorical search remains exact.

With refine=true, scan all eligible row boundaries in the two bins adjacent to the winning coarse split. No refinement runs when a constant or unsplit line wins. Nodes with at most nbins rows use exact search.

This can change the chosen model and predictions. It does not guarantee the exact optimum or an error bound. nbins must be at least two. For Softmax, the search uses the same diagonal-Hessian surrogate as ExactSearch() and selects one shared split by summing the coordinate objectives.

HybridSearch(; nbins=64)

Approximate numeric split search for scalar-score losses. Assign rows to at most nbins equal-count bins once per fit, keeping equal feature values in the same bin. Tree nodes scan these global bins and refine the best coarse boundary over its two adjacent occupied bins. Nodes with at most nbins rows use exact search.

The sufficient statistics use the original feature values and responses, and weights retain their frequency meaning. This can change the selected model and predictions; it does not guarantee the exact optimum or an error bound. Use ExactSearch() or BinnedSearch() for categorical features, and ExactSearch() for vector-score losses. nbins must lie in 2:65535. Boosting reuses bins from all positive-weight training rows across rounds.

A twice-differentiable or IRLS-approximated loss. Implement gradhess!, linkinv, initscore, deviance, scorebound, and validate_target.

Squared-error loss for a real-valued target. Identity link.

Huber loss with transition δ: quadratic within δ of the target, linear beyond it. Identity link.

Pinball (quantile) loss at quantile τ. Non-smooth: fit by IRLS. Identity link.

Mean absolute deviation loss. Non-smooth: fit by IRLS. Identity link.

Logistic (binomial cross-entropy) loss for a {0,1} target. Logit link.

Poisson deviance loss for a non-negative integer count target. Log link.

Negative binomial deviance loss with dispersion θ, for an overdispersed count target. Log link.

Gamma deviance loss for a positive real target. Log link.

Tweedie deviance loss with power ρ ∈ (1, 2), for a non-negative target with a point mass at zero. Log link.

Softmax(K)

K-class softmax with class K as the reference at logit zero. Diagonal Hessian. K is a type parameter so the coefficient type SVector{K-1,T} is known at compile time and every fit specialises on it.

Frozen{V}() <: Loss

Boosting's per-tree loss. Each target row is a pair (z0, h0) of working response and frozen Hessian at the ensemble score, both of type V, so one tree minimises Σ w h0 (f − z0)² / 2: the second-order objective of a boosting round with z0 = −g0 / h0. gradhess! returns g = (f − z0) h0 and h = h0, including exact zero components, the start score is zero, the score bounds are infinite so truncate = true keeps only the feature clamp, and the loss is smooth so no IRLS refit runs. A tree fitted with Frozen predicts its raw score; only a LinearBoost gives it a link.

gradhess!(g, h, loss, y, f)

Unweighted g = ∂ℓ/∂f and h = max(∂²ℓ/∂f², HMIN) per row. Frozen preserves its supplied Hessian, including zero. The caller applies frequency weights after any floor.

gradhess!(g, h, ::Softmax, y, f)

Per-row cross-entropy gradient and diagonal Hessian against the K-1 reference-class logits, with integer class labels y in 1:K.

linkinv(loss, s)

Map raw score s to the response scale: the identity for MSE, Huber, Quantile, and MAD, the logistic sigmoid for Logistic, exp for the count and rate losses, and class probabilities for Softmax.

initscore(::Softmax, y, w)

Log-odds of each non-reference class against the reference class K, from the weighted class frequencies.

deviance(loss, y, f, w)

Weighted 2 Σ w[i] pointloss(loss, y[i], f[i]), the model's reported deviance on the score scale f.

deviance(m::LinearTreeClassifierFit)

Training deviance against m's stored coded target: 0/1 (positive class classes[1]) for a two-class Logistic fit, the 1-based sorted-class index for a Softmax fit. score(m.tree, m.X) returns a plain Matrix for Softmax, so each row is repacked into the SVector pointloss needs.

scorebound(loss, y; truncation_factor=3)

(lo, hi) score-scale clamp bounds fit to the training target y, used when fit_tree's truncate is set. For identity-link losses, with half-width B = (max(y) - min(y)) / 2, truncation_factor pads each side by (truncation_factor - 1) * B, so the bounds are [min(y) - (truncation_factor - 1) B, max(y) + (truncation_factor - 1) B]. Log-link losses allow every representable positive mean below an upper bound of exp(3) * max(maximum(y), 1), capped below floating-point overflow. Logistic and softmax scores use [-10, 10]; frozen scores are unbounded.

validate_target(loss, y)

Throw ArgumentError if y is not finite everywhere or does not satisfy loss's domain (e.g. {0,1} for Logistic, non-negative integers for Poisson); otherwise return nothing.

issmooth(loss)

true when loss has a well-defined Hessian everywhere and fits by Newton steps; false for the L1-type losses (Quantile, MAD), which fit by IRLS.

Adapter around a LossFunctions.SupervisedLoss. Margin losses receive targets in {0,1} and are re-expressed with t = 2y - 1. The pointwise loss, gradient, and hessian are multiplied by scale.

Score-space link between the tree's raw score and the value the inner loss expects.

Score-space link marking the tree's raw score as a logit, for a margin loss.

Score-space link marking the tree's raw score as a log mean, for PoissonLoss.

Newton-step scale that lines an inner loss up with the matching native Loss.

feature_importance(tree)

Surrogate-deviance drop per feature from nonconstant node models, including unsplit LIN nodes, normalised to sum to one. Zeros when no node has positive gain.

feature_importance(boost)

Positive split gains summed over every tree, normalised to sum to one.

coeftable(tree, x)

Intercept and per-feature slopes of the unclipped score at x. A feature clamped at x contributes zero slope and its piece value goes to the intercept. Categorical pieces are constants.

Intercept and slopes of the locally active, unclipped affine predictor.

coeftable(boost, x)

Intercept and per-feature slopes of the unclipped ensemble score at x: f0 plus eta times the sum of each tree's coeftable.

shap(tree, X; nthreads=Threads.nthreads()) -> ShapResult

Path-dependent TreeSHAP values for every row of X against every feature of tree, on the unclipped score scale. Each row's SHAP values sum to score_row(tree, X, i, false) - result.base, the game's efficiency identity.

shap(boost, X; nthreads=Threads.nthreads()) -> ShapResult

Path-dependent TreeSHAP for a LinearBoost; see shap!.

shap!(values, clipped, tree, X; nthreads=Threads.nthreads())

In-place shap: write into values (n × p, or n × p × (K-1) for Softmax) and clipped (Vector{Bool}, length n), then return a ShapResult wrapping them.

shap!(values, clipped, boost, X; nthreads=Threads.nthreads())

In-place shap for an ensemble: eta times the sum of every tree's path-dependent SHAP values, base f0 + eta Σ expected_score(tree), so each row sums to score(boost, x; clip = false) − base. clipped[i] is true when the ensemble clamp changed row i. One PathPool per row block, reused across the ensemble's trees.

ShapResult{A,B,C}

values is n × p for scalar-score trees, n × p × (K-1) for Softmax. base is the game's empty-coalition value, expected_score(tree); fit_tree stores the same number in tree.base. clipped[i] is true when the score bound changed row i's prediction.

expected_score(tree) -> V

Cover-weighted game_value(tree, ·, ∅) over the whole tree: at each split the absent feature's piece is evaluated at the node's xmean on both children, weighted by cover(child) / cover(parent). Independent of x. fit_tree stores it in tree.base. shap/shap! recompute it from the nodes (O(nodes)) so a hand-built tree whose base field is stale still satisfies the efficiency identity.

LinearTreeRegressorFit{Tr}

A fit_tree result wrapped with its input encoder and training data as a StatsAPI.RegressionModel. Build with fit(LinearTreeRegressorFit, X, y; loss=MSE(), weights=nothing, unseen=:error, kwargs...), where X is a matrix or a Tables.jl table and kwargs forward to fit_tree.

LinearTreeClassifierFit{Tr,C}

A fit_tree result for a categorical target, wrapped as a StatsAPI.StatisticalModel. Build with fit(LinearTreeClassifierFit, X, y; weights=nothing, unseen=:error, kwargs...). classes = sort(unique(y)) fixes both the label-to-code map and the column order of predict's probability matrix. Two classes fit a single Logistic tree with classes[1] (the first sorted class) as the positive outcome; predict returns hcat(p1, 1 .- p1), so column k is still P(classes[k]). Three or more classes fit a Softmax tree with classes[k] at coordinate k.

LinearBoostRegressorFit{B}

A fit_boost result wrapped with its input encoder and training data as a StatsAPI.RegressionModel. Build with fit(LinearBoostRegressorFit, X, y; loss=MSE(), weights=nothing, unseen=:error, Xval=nothing, yval=nothing, wval=nothing, kwargs...); Xval may be a matrix or table and is encoded like X; kwargs forward to fit_boost.

LinearBoostClassifierFit{B,C}

A fit_boost result for a categorical target, with the same class conventions as LinearTreeClassifierFit: classes[1] is the Logistic positive outcome for two classes, and column k of predict is P(classes[k]). Build with fit(LinearBoostClassifierFit, X, y; weights=nothing, unseen=:error, Xval=nothing, yval=nothing, wval=nothing, kwargs...).

fit(LinearTreeRegressorFit, X, y; loss=MSE(), weights=nothing, unseen=:error, kwargs...)
fit(LinearTreeClassifierFit, X, y; weights=nothing, unseen=:error, kwargs...)

Fit a LinearTreeRegressorFit or LinearTreeClassifierFit from a matrix or Tables.jl table X and a target y. kwargs forward to fit_tree. unseen is :error (default) or :right, applied to a categorical level absent from the declared training pool at predict time.

Number of rows passed to fit, zero-weight rows included.

dof(m)

Count of stored coefficients across every node: 1 for CON, 2 for LIN or PCON, 3 for BLIN, 4 for PLIN, times K-1 for a Softmax fit. A parameter count, not an effective degrees of freedom, and unrelated to BIC's per-kind selection penalty of the same name.

Training row weights, as stored (ones when fit was called with no weights).

Training residuals on the response scale; rows with unencodable zero-weight predictors are NaN.

LinearTreeRegressor

A model type for constructing a linear tree regressor, based on LinearTrees.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LinearTreeRegressor = @load LinearTreeRegressor pkg=LinearTrees

Do model = LinearTreeRegressor() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in LinearTreeRegressor(loss=...).

LinearTreeRegressor fits a PILOT-style linear model tree for a continuous or count target, generalised to any twice-differentiable loss.

Hyperparameters

• loss::Loss = MSE(): the loss to fit; see MSE, Huber, Quantile, MAD, Poisson, NegBin, Gamma, Tweedie.
• rule::SelectionRule = BIC(): model-kind selection rule at each node.
• max_depth::Int = 12: maximum tree depth.
• min_samples_split::Int = 10: minimum total weight in a node to attempt a split.
• min_samples_leaf::Int = 5: minimum total weight required in each child.
• min_sum_hessian::Float64 = 1.0: minimum summed hessian in a node to attempt a split.
• max_lin_chain::Int = 10: maximum consecutive lin fits along one root-to-node path.
• truncate::Bool = true: clamp predictions to the training score range.
• truncation_factor::Float64 = 3.0: half-range multiplier for the truncation bounds.

Operations

• predict(mach, Xnew): return the predicted response for Xnew.

Fitted parameters

The fields of fitted_params(mach) are:

• tree: the fitted LinearTree.

Report

The fields of report(mach) are:

• feature_importances: node-model gain feature importances, as in feature_importance.
• kind_counts: a count of fitted nodes by ModelKind.

See also LinearTreeClassifier.

LinearTreeClassifier

A model type for constructing a linear tree classifier, based on LinearTrees.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LinearTreeClassifier = @load LinearTreeClassifier pkg=LinearTrees

Do model = LinearTreeClassifier() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in LinearTreeClassifier(rule=...).

LinearTreeClassifier fits a PILOT-style linear model tree for a finite (two- or many-class) target: a Logistic tree for two classes, a Softmax tree for three or more.

Hyperparameters

• rule::SelectionRule = BIC(): model-kind selection rule at each node.
• max_depth::Int = 12: maximum tree depth.
• min_samples_split::Int = 10: minimum total weight in a node to attempt a split.
• min_samples_leaf::Int = 5: minimum total weight required in each child.
• min_sum_hessian::Float64 = 1.0: minimum summed hessian in a node to attempt a split.
• max_lin_chain::Int = 10: maximum consecutive lin fits along one root-to-node path.
• truncate::Bool = true: clamp predictions to the training score range.
• truncation_factor::Float64 = 3.0: half-range multiplier for the truncation bounds.

Operations

• predict(mach, Xnew): return the predicted class distribution for Xnew, as a UnivariateFinite. Use predict_mode(mach, Xnew) for point predictions.

Fitted parameters

The fields of fitted_params(mach) are:

• tree: the fitted LinearTree.

Report

The fields of report(mach) are:

• feature_importances: node-model gain feature importances, as in feature_importance.
• kind_counts: a count of fitted nodes by ModelKind.

See also LinearTreeRegressor.

LinearBoostRegressor

A model type for constructing a linear boost regressor, based on LinearTrees.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LinearBoostRegressor = @load LinearBoostRegressor pkg=LinearTrees

Do model = LinearBoostRegressor() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in LinearBoostRegressor(loss=...).

LinearBoostRegressor fits a gradient-boosted ensemble of linear model trees for a continuous or count target. MLJ iteration controls nrounds; this model does not expose validation early stopping, for which use IteratedModel.

Hyperparameters

• loss::Loss = MSE(): scalar-response loss, such as MSE, Huber, Quantile, MAD, Poisson, NegBin, Gamma, or Tweedie.
• nrounds::Int = 100: number of boosting rounds.
• eta::Float64 = 0.1: shrinkage per round.
• max_depth::Int = 5: maximum base-tree depth.
• min_samples_split::Int = 10: minimum node weight for a split.
• min_samples_leaf::Int = 5: minimum child weight.
• min_sum_hessian::Float64 = 1.0: minimum node Hessian sum.
• lambda_slope::Float64 = 1.0: slope penalty.
• lambda_intercept::Float64 = 1.0: intercept penalty.
• gamma::Float64 = 0.0: split-gain penalty.
• subsample::Float64 = 1.0: row-sampling fraction.
• colsample::Float64 = 1.0: feature-sampling fraction.
• truncate::Bool = true: clamp predictions to the training score range.
• rng::Union{AbstractRNG,Integer} = default_rng(): random source; an integer starts a fresh Xoshiro on each fit.

Fitted parameters and report

fitted_params(mach).boost is the fitted LinearBoost. report(mach) holds feature_importances, nrounds, and per-round history; the history is also available through training_losses(mach).

See also LinearBoostClassifier.

LinearBoostClassifier

A model type for constructing a linear boost classifier, based on LinearTrees.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LinearBoostClassifier = @load LinearBoostClassifier pkg=LinearTrees

Do model = LinearBoostClassifier() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in LinearBoostClassifier(split_search=...).

LinearBoostClassifier fits a gradient-boosted ensemble of linear model trees for a finite target. It uses Logistic loss for two classes and Softmax for three or more. MLJ iteration controls nrounds; this model does not expose validation early stopping, for which use IteratedModel.

Hyperparameters

• nrounds::Int = 100: number of boosting rounds.
• eta::Float64 = 0.1: shrinkage per round.
• max_depth::Int = 5: maximum base-tree depth.
• min_samples_split::Int = 10: minimum node weight for a split.
• min_samples_leaf::Int = 5: minimum child weight.
• min_sum_hessian::Float64 = 1.0: minimum node Hessian sum.
• lambda_slope::Float64 = 1.0: slope penalty.
• lambda_intercept::Float64 = 1.0: intercept penalty.
• gamma::Float64 = 0.0: split-gain penalty.
• subsample::Float64 = 1.0: row-sampling fraction.
• colsample::Float64 = 1.0: feature-sampling fraction.
• truncate::Bool = true: clamp predictions to the training score range.
• rng::Union{AbstractRNG,Integer} = default_rng(): random source; an integer starts a fresh Xoshiro on each fit.

Operations

predict(mach, Xnew) returns a UnivariateFinite; use predict_mode for point predictions.

Fitted parameters and report

fitted_params(mach).boost is the fitted LinearBoost. report(mach) holds feature_importances, nrounds, and per-round history; the history is also available through training_losses(mach).

See also LinearBoostRegressor.

A named view of a tree for AbstractTrees traversal and printing.

to_dict(tree)

JSON-friendly form: nodes as vectors of field dictionaries, loss as a name plus parameters. Three encodings keep every field JSON-safe:

• Every Real-valued field (threshold, lcoef, lintercept, rcoef, rintercept, xmin, xmax, cover, xmean, gain, lo, hi, base) goes through jsonnum: finite values become Float64, non-finite values become the string "Inf", "-Inf", or "NaN", and an SVector field maps elementwise into a Vector{Any}.
• Integer fields (feature, left, right, catstart, catwords, model) store as Int, restored to their struct field type (Int32 or ModelKind) by from_dict.
• catmasks (UInt64, may use the top bit) store as lowercase hex strings via string(m; base = 16), since a plain integer would overflow Int64 and lose exactness in a JSON parser; restored with parse(UInt64, s; base = 16).

from_dict(d) -> LinearTree

Inverse of to_dict. Throws ArgumentError for an unrecognised format version, element type, or loss name.