The Wayback Machine - https://web.archive.org/web/20220320215120/https://www.geeksforgeeks.org/python-decision-tree-regression-using-sklearn/
Skip to content
Related Articles

Related Articles

Table of Contents

Improve Article
Save Article
Like Article

Python | Decision Tree Regression using sklearn

  • Difficulty Level : Easy
  • Last Updated : 20 Oct, 2021

Decision Tree is a decision-making tool that uses a flowchart-like tree structure or is a model of decisions and all of their possible results, including outcomes, input costs, and utility.
Decision-tree algorithm falls under the category of supervised learning algorithms. It works for both continuous as well as categorical output variables.

The branches/edges represent the result of the node and the nodes have either: 

  1. Conditions [Decision Nodes]
  2. Result [End Nodes]

The branches/edges represent the truth/falsity of the statement and take makes a decision based on that in the example below which shows a decision tree that evaluates the smallest of three numbers:  


Image

Decision Tree Regression: 
Decision tree regression observes features of an object and trains a model in the structure of a tree to predict data in the future to produce meaningful continuous output. Continuous output means that the output/result is not discrete, i.e., it is not represented just by a discrete, known set of numbers or values.

Discrete output example: A weather prediction model that predicts whether or not there’ll be rain on a particular day. 
Continuous output example: A profit prediction model that states the probable profit that can be generated from the sale of a product.
Here, continuous values are predicted with the help of a decision tree regression model.

Let’s see the Step-by-Step implementation – 

  • Step 1: Import the required libraries. 

Python3




# import numpy package for arrays and stuff
import numpy as np 
  
# import matplotlib.pyplot for plotting our result
import matplotlib.pyplot as plt
  
# import pandas for importing csv files 
import pandas as pd 
  • Step 2: Initialize and print the Dataset.
Python3



# import dataset
# dataset = pd.read_csv('Data.csv') 
# alternatively open up .csv file to read data
  
dataset = np.array(
[['Asset Flip', 100, 1000],
['Text Based', 500, 3000],
['Visual Novel', 1500, 5000],
['2D Pixel Art', 3500, 8000],
['2D Vector Art', 5000, 6500],
['Strategy', 6000, 7000],
['First Person Shooter', 8000, 15000],
['Simulator', 9500, 20000],
['Racing', 12000, 21000],
['RPG', 14000, 25000],
['Sandbox', 15500, 27000],
['Open-World', 16500, 30000],
['MMOFPS', 25000, 52000],
['MMORPG', 30000, 80000]
])
  
# print the dataset
print(dataset) 
Output:
[['Asset Flip' '100' '1000']
 ['Text Based' '500' '3000']
 ['Visual Novel' '1500' '5000']
 ['2D Pixel Art' '3500' '8000']
 ['2D Vector Art' '5000' '6500']
 ['Strategy' '6000' '7000']
 ['First Person Shooter' '8000' '15000']
 ['Simulator' '9500' '20000']
 ['Racing' '12000' '21000']
 ['RPG' '14000' '25000']
 ['Sandbox' '15500' '27000']
 ['Open-World' '16500' '30000']
 ['MMOFPS' '25000' '52000']
 ['MMORPG' '30000' '80000']]
  • Step 3: Select all the rows and column 1 from the dataset to “X”.
Python3



# select all rows by : and column 1
# by 1:2 representing features
X = dataset[:, 1:2].astype(int) 
  
# print X
print(X)
Output:
[[  100]
 [  500]
 [ 1500]
 [ 3500]
 [ 5000]
 [ 6000]
 [ 8000]
 [ 9500]
 [12000]
 [14000]
 [15500]
 [16500]
 [25000]
 [30000]]

  • Step 4: Select all of the rows and column 2 from the dataset to “y”.
Python3



# select all rows by : and column 2
# by 2 to Y representing labels
y = dataset[:, 2].astype(int) 
  
# print y
print(y)
Output:
[ 1000  3000  5000  8000  6500  7000 15000 20000 21000 25000 27000 30000 52000 80000]
  • Step 5: Fit decision tree regressor to the dataset
Python3



# import the regressor
from sklearn.tree import DecisionTreeRegressor 
  
# create a regressor object
regressor = DecisionTreeRegressor(random_state = 0) 
  
# fit the regressor with X and Y data
regressor.fit(X, y)
Output:
DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=None,
                      max_features=None, max_leaf_nodes=None,
                      min_impurity_decrease=0.0, min_impurity_split=None,
                      min_samples_leaf=1, min_samples_split=2,
                      min_weight_fraction_leaf=0.0, presort='deprecated',
                      random_state=0, splitter='best')
  • Step 6: Predicting a new value
Python3



# predicting a new value
  
# test the output by changing values, like 3750
y_pred = regressor.predict([[3750]])
  
# print the predicted price
print("Predicted price: % d\n"% y_pred) 
Output:
Predicted price:  8000
  • Step 7: Visualising the result
Python3



# arange for creating a range of values 
# from min value of X to max value of X 
# with a difference of 0.01 between two
# consecutive values
X_grid = np.arange(min(X), max(X), 0.01)
  
# reshape for reshaping the data into 
# a len(X_grid)*1 array, i.e. to make
# a column out of the X_grid values
X_grid = X_grid.reshape((len(X_grid), 1)) 
  
# scatter plot for original data
plt.scatter(X, y, color = 'red')
  
# plot predicted data
plt.plot(X_grid, regressor.predict(X_grid), color = 'blue') 
  
# specify title
plt.title('Profit to Production Cost (Decision Tree Regression)') 
  
# specify X axis label
plt.xlabel('Production Cost')
  
# specify Y axis label
plt.ylabel('Profit')
  
# show the plot
plt.show()
Image
  • Step 8: The tree is finally exported and shown in the TREE STRUCTURE below, visualized using http://www.webgraphviz.com/ by copying the data from the ‘tree.dot’ file.
Python3



# import export_graphviz
from sklearn.tree import export_graphviz 
  
# export the decision tree to a tree.dot file
# for visualizing the plot easily anywhere
export_graphviz(regressor, out_file ='tree.dot',
               feature_names =['Production Cost']) 
Output (Decision Tree): 

Image
 

My Personal Notes
arrow_drop_up
Previous
Python | Decision tree implementation
Next

Random Forest Regression in Python
Recommended Articles
Page :
Python | Linear Regression using sklearn
23, May 19
ML | Logistic Regression v/s Decision Tree Classification
23, May 19
ML | Linear Regression vs Logistic Regression
05, Feb 20
Calculating the completeness score using sklearn in Python
25, Sep 20
homogeneity_score using sklearn in Python
22, Sep 20
Python | Create Test DataSets using Sklearn
24, Jan 19
ML | Implementation of KNN classifier using Sklearn
23, May 19
ML | Dummy classifiers using sklearn
23, May 19
Implementing DBSCAN algorithm using Sklearn
06, Jun 19
Implementing Agglomerative Clustering using Sklearn
07, Jun 19
ML | OPTICS Clustering Implementing using Sklearn
12, Jul 19
ML | Ridge Regressor using sklearn
06, Sep 19
ML | Voting Classifier using Sklearn
23, Nov 19
ML | Implementing L1 and L2 regularization using Sklearn
11, Jun 19
Data Pre-Processing with Sklearn using Standard and Minmax scaler
21, Feb 21
How to import datasets using sklearn in PyBrain
26, Feb 22
sklearn.Binarizer() in Python
03, Jul 18
ML | sklearn.linear_model.LinearRegression() in Python
26, Sep 18
How to use datasets.fetch_mldata() in sklearn - Python?
17, Aug 20
sklearn.cross_decomposition.PLSRegression() function in Python
15, Oct 20
sklearn.metrics.max_error() function in Python
25, Oct 20
Sklearn.StratifiedShuffleSplit() function in Python
06, Nov 20
Sklearn | Feature Extraction with TF-IDF
20, Sep 19
Python | Decision tree implementation
30, Nov 17
Article Contributed By :
https://media.geeksforgeeks.org/auth/avatar.png
AnkanDas22
@AnkanDas22
Vote for difficulty
Current difficulty :
Easy





Improved By :
Article Tags :
Practice Tags :
Report Issue

We use cookies to ensure you have the best browsing experience on our website. By using our site, you
acknowledge that you have read and understood our
Cookie Policy &
        Privacy Policy

Lightbox

Start Your Coding Journey Now!