How to Fit a NonLinear Regression Model

Intro

Often data does not follow a direct line. That is why we have polynomials (i.e. X^2, X^3). These relationships are still direct, but the rate is squared or cubed. We can still use linear regression with some modifications to fit this relationship. In this article, we will learn how to fit a Non Linear Regression Model in Sklearn.

Creating a NonLinear Regression Model

To create a non linear regression model, we use the PolynomialFeatures class. This is similar to working with interaction effects. We create an instance of PolynomialFeatures and specify the number of degrees. In our example below, we want to fit a model with x2 and x3. Then, you use the fit_transform method on your feature matrix and pass this to a linear model.

from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_boston
from sklearn.preprocessing import PolynomialFeatures

boston = load_boston()
features = boston.data
target = boston.target

# poly with x^2 and x^3
polynomial = PolynomialFeatures(degree=3, include_bias=False)
feat_poly = polynomial.fit_transform(features)

regression = LinearRegression()
model = regression.fit(feat_poly, target)
print(modescore())