Standardisation support

[gm89uk]

Hi @MilesCranmer / Team,

I have a QoL request for running standardised data. Often standardisation (e.g. mean 0, sd 1) comes with massive speed ups and better convergence. When using SR across a lot of different projects, it becomes quite tedious to make the resulting pareto-front work with raw values.

What would be great is to have an option for standardisation (e.g. standardize = true/false), then a "Raw_Equation" column in HoF? If standardize = true, then SR could run with standardised X, then the Raw_Equation column could be the affine transformations substituted back to the expression tree. Perhaps I am working inefficiently and there is a better way!

George

[IlyaKhalafi]

You can already build the raw-unit column from the exported SymPy expressions. For a single-output model with feature selection disabled, fitted on x_scaler.transform(X) using a default StandardScaler, this keeps the original hall of fame intact

import sympy as sp

model.refresh()
hof = model.equations_.copy()
substitutions = {
    sp.Symbol(name): (sp.Symbol(name) - mean) / scale
    for name, mean, scale in zip(
        model.feature_names_in_, x_scaler.mean_, x_scaler.scale_
    )
}
hof["Raw_Equation"] = hof["sympy_format"].map(
    lambda expr: expr.subs(substitutions, simultaneous=True)
)

Keep the scaler fitted on the training data, with the same feature order. Use scale_ rather than recomputing standard deviations, since StandardScaler uses a scale of one for constant features. If you standardized the target too, apply y_scaler.scale_[0] * expr + y_scaler.mean_[0] to each resulting expression.

PySR's export code supplies sympy_format, and its symbol creation uses those feature names. Custom operators still need their SymPy mappings.

I checked this substitution on synthetic expressions with fitted scikit-learn scalers, including a constant feature and target scaling. I did not run a PySR search. The existing complexity and loss columns still describe the standardized search, so I would keep this as a separate export rather than replace model.equations_.

[gm89uk]

Thank you. I'm using SR.jl so it is relatively straight forward. I am running for a very high niterations and stop after a few days, so the search never finishes, so I will struggle to do this. It's not a big deal, and the cost efficiency of adding another column to the HoF is probably not worth it.

[IlyaKhalafi]

For SR.jl directly, you can use the saved frontier without waiting for all the iterations. With the default save_to_file=true, the current search loop saves updated Pareto frontiers during the run.

For a single output, the writer uses outputs/<run_id>/hall_of_fame.csv by default and also writes a .bak. After stopping, you can convert the last intact CSV with your training means and scales. Keep the original CSV and write the raw-unit column to a separate export. If the process was killed mid-write, check the backup for an intact copy.

For future long runs, timeout_in_seconds supplies a normal stopping condition, so equation_search can clean up and return its result before exhausting the iteration budget.

This is from the current source, not a Julia search I ran.