The Wayback Machine - https://web.archive.org/web/20240906210914/https://www.geeksforgeeks.org/robust-correlation/
Open In App

Robust Correlation

Last Updated : 19 Mar, 2024
Comments
Improve
Suggest changes
Like Article
Like
Save
Share
Report
News Follow

Correlation is a statistical tool that is used to analyze and measure the degree of relationship or degree of association between two or more variables. There are generally three types of correlation:

  • Positive correlation: When we increase the value of one variable, the value of another variable increased respectively, this is called Positive Correlation.
  • Negative correlation: When we increase the value of one variable, the value of another variable decreases respectively, this is called negative Correlation.
  • Zero correlation: When the change in the value of one variable does not impact another substantially, then it is called zero correlation.
Image

Pearson Correlation:

Pearson correlation is the most common way of calculating the correlation. It is denoted by r. Consider for two variables x and y, it is represented by the following formula:

[Tex]r = \frac{\sum_{i=1}^{n} (x_i – \overline{x})(y_i – \overline{y})} {\sqrt{\sum_{i=1}^{n} (x_i – \overline{x})^2(y_i – \overline{y})^2}}[/Tex]

A value closer to -1 represents a perfectly negative correlation, whereas  0 represents no correlation and 1 represents a strong positive correlation.

The Pearson correlation coefficient is a good estimator of correlation between two variables for normal distribution. However, it does not fill the criteria of the robust estimator because it is not:

  • Resistant: This means changing a small fraction of data even by a huge amount does not considerably affect the value of the estimate.
  • Robustness of Efficiency: the statistic has high efficiency in a variety of situations rather than in any one situation. Efficiency means that the estimate is close to the optimal estimate given that we know what distribution that the data comes from.

Efficiency can be measure using the following formula:

[Tex]Efficiency  = \frac{lowest-variance-possible}{actual-variance}[/Tex]

Percentage Bend Correlation:

Percent bend correlation was proposed by shoemaker and Hettmanspergr in 1982 and also mentioned by Wilcox in his book. This correlation is both resistant and robust to efficiency. 

Following are the steps to perform Percentage Bend correlation on two variables X and Y:

  • Set m = (1-\beta) *m  + 0.5, Round m to nearest integer. Here, \beta is between 0 and 0.5
  • Take W_{i} = |X_{i} – M_{x}| for i = 1, 2, …n, where M_x is the median of X.
  • Sort W_i in the ascending order.
  • [Tex]\hat{W_x} = W(m)        [/Tex], where W(m) is the estimate of the (1-\beta) quantile of W.
  • Sort the X values.
  • Computer the number of values \frac{(X_{i} – M_{x})}{\hat{W}_{x}(\beta)} that are <-1 and store in i_1 and the number that are > +1 and store in [Tex]i_2        [/Tex] respectively. Then compute the following:

[Tex]S_{x} = \sum_{i=i1+1}^{n-i2}{X_{i}}[/Tex]

[Tex]\hat{\phi}_{x} = \frac{\hat{W}_{x}(i2 – i1) + S_{x}}{n – i1 – i2}[/Tex]

[Tex]U_{i} = \frac{X_{i} – \hat{\phi}_{x}}{\hat{W}_{x}}[/Tex]

  • Repeat the above steps for the Y estimator to get \hat{W_y}, \hat{\phi}_{y} and V_i.
  • Define the function:

[Tex]\Psi(x) = \max[-1, \min(1,x)][/Tex]

therefore compute,

[Tex]A_i = \Psi (U_i), B_i = \Psi (V_i)[/Tex]

  • Calculate the percent bend correlation:

[Tex]\rho_{pb} = \frac{\sum_{i=1}^{n}{A_{i}B_{i}}}           {\sqrt{\sum_{i=1}^{n}{A_{i}^2}\sum_{i=1}^{n}{B_{i}^2}}}[/Tex]

Winsorized Correlation:

The standard correlation like Pearson is sometimes heavily influenced by extreme values. The Winsorized correlation solves this by setting the tail values equal to a certain percentile value.

 For example, for a 90% Winsorized correlation, the bottom 5% of the values are set equal to the value corresponding to the 5th percentile while the upper 5% of the values are set equal to the value corresponding to the 95th percentile. Then the standard correlation is applied.

Implementation:

  • In this implementation, we will be using the motor trend car Road Tests dataset available in the graphics library in R. It is very popular and easily available. This dataset contains 32 observations of 11 different variables related to cars. We will be performing the correlation analysis between these variables (Pearson, percentage bend and winsorized) and plot them.
R

# Install the required packages install.packages("dplyr") install.packages("correlation") install.packages("see") # import required packages library(dplyr) library(correlation) library(see) # Load data data("mtcars") # check help for mtcars data ?mtcars ## Description # The data was extracted from the 1974 Motor Trend US magazine, # and comprises fuel consumption and 10 aspects of automobile # design and performance for 32 automobiles #(1973–74 models). ## Usage # mtcars ## Format # A data frame with 32 observations on 11 (numeric) variables. # # [, 1] mpg Miles/(US) gallon # [, 2] cyl Number of cylinders # [, 3] disp Displacement (cu.in.) # [, 4] hp Gross horsepower # [, 5] dart Rear axle ratio # [, 6] wt Weight (1000 lbs) # [, 7] qsec 1/4 mile time # [, 8] vs Engine (0 = V-shaped, 1 = straight) # [, 9] am Transmission (0 = automatic, 1 = manual) # [,10] gear Number of forward gears # [,11] carb Number of carburetors ## Source # Henderson and Velleman (1981), Building multiple regression # models interactively. Biometrics, 37, 391–411. # perform different correlation and print summary # pearson correlation pearson_corr = correlation(mtcars) pearson_summary = summary(pearson_corr) print(pearson_summary) # percentage bend correlation pbc_corr = correlation(mtcars,method='percentage') pbc_summary= summary(pbc_corr) print(pbc_summary) # winsorized correlation wins_corr = correlation(mtcars, winsorize = 0.2) winsor_summary = summary(wins_corr) print(winsor_summary) # plot different correlation analysis pearson_summary%>%plot() pbc_summary%>%plot() winsor_summary%>%plot()

# Correlation Matrix (pearson-method) Parameter | carb | gear | am | vs | qsec | wt | dart | hp | disp | cyl --------------------------------------------------------------------------------------------------------------------- mpg | -0.55* | 0.48 | 0.60** | 0.66** | 0.42 | -0.87*** | 0.68*** | -0.78*** | -0.85*** | -0.85*** cyl | 0.53* | -0.49 | -0.52* | -0.81*** | -0.59* | 0.78*** | -0.70*** | 0.83*** | 0.90*** | disp | 0.39 | -0.56* | -0.59* | -0.71*** | -0.43 | 0.89*** | -0.71*** | 0.79*** | | hp | 0.75*** | -0.13 | -0.24 | -0.72*** | -0.71*** | 0.66** | -0.45 | | | dart | -0.09 | 0.70*** | 0.71*** | 0.44 | 0.09 | -0.71*** | | | | wt | 0.43 | -0.58* | -0.69*** | -0.55* | -0.17 | | | | | qsec | -0.66** | -0.21 | -0.23 | 0.74*** | | | | | | vs | -0.57* | 0.21 | 0.17 | | | | | | | am | 0.06 | 0.79*** | | | | | | | | gear | 0.27 | | | | | | | | | p-value adjustment method: Holm (1979)>

# Correlation Matrix (percentage-method) Parameter | carb | gear | am | vs | qsec | wt | dart | hp | disp | cyl ---------------------------------------------------------------------------------------------------------------------- mpg | -0.64** | 0.55* | 0.58** | 0.68*** | 0.48 | -0.90*** | 0.68*** | -0.90*** | -0.88*** | -0.91*** cyl | 0.58* | -0.55* | -0.52* | -0.81*** | -0.60** | 0.85*** | -0.72*** | 0.91*** | 0.94*** | disp | 0.47 | -0.61** | -0.60** | -0.73*** | -0.50 | 0.88*** | -0.74*** | 0.89*** | | hp | 0.70*** | -0.37 | -0.40 | -0.79*** | -0.69*** | 0.80*** | -0.59** | | | dart | -0.11 | 0.78*** | 0.73*** | 0.47 | 0.13 | -0.76*** | | | | wt | 0.53* | -0.64** | -0.76*** | -0.57* | -0.26 | | | | | qsec | -0.68*** | -0.13 | -0.17 | 0.80*** | | | | | | vs | -0.62** | 0.27 | 0.17 | | | | | | | am | -0.07 | 0.80*** | | | | | | | | gear | 0.11 | | | | | | | | | p-value adjustment method: Holm (1979)>

# Winsorized Correlation Matrix Parameter | carb | gear | am | vs | qsec | wt | dart | hp | disp | cyl --------------------------------------------------------------------------------------------------------------------- mpg | -0.63** | 0.65** | 0.55* | 0.70*** | 0.49 | -0.86*** | 0.67*** | -0.88*** | -0.87*** | -0.93*** cyl | 0.60** | -0.68*** | -0.52* | -0.81*** | -0.60** | 0.87*** | -0.74*** | 0.90*** | 0.94*** | disp | 0.45 | -0.74*** | -0.57* | -0.72*** | -0.51* | 0.85*** | -0.74*** | 0.89*** | | hp | 0.69*** | -0.56* | -0.37 | -0.79*** | -0.63** | 0.77*** | -0.60** | | | dart | -0.12 | 0.88*** | 0.72*** | 0.50* | 0.22 | -0.76*** | | | | wt | 0.53* | -0.69*** | -0.78*** | -0.56* | -0.29 | | | | | qsec | -0.61** | 0.15 | -0.12 | 0.84*** | | | | | | vs | -0.62** | 0.45 | 0.17 | | | | | | | am | -0.11 | 0.78*** | | | | | | | | gear | -0.03 | | | | | | | | | p-value adjustment method: Holm (1979)

Image

Pearson correlation

Image

Percentage Bend Correlation

Image

Winsor correlation

References:



P
pawangfg
News
Improve
Previous Article
Program to find correlation coefficient
Next Article
Normal Probability Plot

Similar Reads

MATLAB for Signal Analysis: Demystifying Cross-Correlation and Correlation Coefficients
When analyzing relationships between signals or datasets, two commonly used techniques are cross-correlation and the correlation coefficient. Both methods have their unique applications and interpretations. In this article, we will delve into the technical details of these methods, their implementations in MATLAB, and the interpretation of their ou
5 min read
Robust Regression for Machine Learning in Python
Simple linear regression aims to find the best fit line that describes the linear relationship between some input variables(denoted by X) and the target variable(denoted by y). This has some limitations as in real-world problems, there is a high probability that the dataset may have outliers. This results in biased model fitting. To overcome this l
4 min read
Probabilistic Neural Networks: A Statistical Approach to Robust and Interpretable Classification
Probabilistic Neural Networks (PNNs) are a class of artificial neural networks that leverage statistical principles to perform classification tasks. Introduced by Donald Specht in 1990, PNNs have gained popularity due to their robustness, simplicity, and ability to handle noisy data. This article delves into the intricacies of PNNs, providing a det
10 min read
Different Robust Standard Errors of Logit Regression in Stata and R
Logistic regression is widely used in statistics and machine learning for modeling binary outcome variables. However, standard errors in logistic regression can be sensitive to violations of model assumptions, such as heteroscedasticity or clustering of observations. Robust standard errors provide a way to mitigate these issues and produce more rel
4 min read
Redundancy and Correlation in Data Mining
Prerequisites:Chi-square test, covariance-and-correlation What is Data Redundancy ? During data integration in data mining, various data stores are used. This can lead to the problem of redundancy in data. An attribute (column or feature of data set) is called redundant if it can be derived from any other attribute or set of attributes. Inconsisten
2 min read
Exploring Correlation in Python
This article aims to give a better understanding of a very important technique of multivariate exploration. A correlation Matrix is basically a covariance matrix. Also known as the auto-covariance matrix, dispersion matrix, variance matrix, or variance-covariance matrix. It is a matrix in which the i-j position defines the correlation between the i
4 min read
Probability plot correlation coefficient
The probability plot correlation coefficient (PPCC) is a graphical technique for identifying the shape parameter that best describes the dataset. Most of the statistical analysis has been done assuming the shape of the distribution in mind. However, these assumptions may be challenged because sometimes the distributions can have very different shap
4 min read
Correlation and Regression with R
Correlation and regression analysis are two fundamental statistical techniques used to examine the relationships between variables. R Programming Language is a powerful programming language and environment for statistical computing and graphics, making it an excellent choice for conducting these analyses. In this response, I'll provide an overview
8 min read
Difference between Correlation and Regression
In the realm of data analysis and statistics, these two techniques play an important role- Correlation and Regression techniques understand the relationship between variables, make predictions and generate useful data insights from data. For data analysts and researchers, these tools are essential across various fields. Let's study the concepts of
7 min read
Canonical Correlation Analysis (CCA) using Sklearn
Canonical Correlation Analysis (CCA) is a statistical method used in data analysis to identify and quantify the relationships between two sets of variables. When working with multivariate data—that is, when there are several variables in each of the two sets and we want to know how they connect—it is very helpful. This post will explain CCA, go ove
10 min read
Spearman's Rank Correlation
Correlation measures the strength of the association between two variables. For instance, if we are interested in knowing whether there is a relationship between the heights of fathers and sons, a correlation coefficient can be calculated to answer this question. To learn more about correlation, please refer to this.  Methods for correlation analys
8 min read
Convolution and Cross-Correlation in CNN
Answer: Convolution in CNN involves flipping both the rows and columns of the kernel before sliding it over the input, while cross-correlation skips this flipping step.These operations are foundational in extracting features and detecting patterns within the data, despite their technical differences. AspectConvolutionCross-CorrelationKernel Flippin
2 min read
What is Diffrence between Correlation and Multicollinearity?
Answer: Correlation measures the linear relationship between two variables, while multicollinearity indicates a high correlation among predictor variables in a regression model, potentially causing issues like unstable estimates and inflated standard errors.Here's a table comparing correlation and multicollinearity: This table outlines the key diff
2 min read
How to Calculate Correlation By Group in R
Calculating correlation by group in R Programming Language involves finding the correlation coefficient between two variables within each subgroup defined by another variable. In R, correlation by group can be achieved by using the cor() function along with other functions like group_by() from the 'dplyr' package or aggregate() function. Syntax:lib
5 min read
How to Calculate Matthews Correlation Coefficient in R
The correlation coefficient is a statistical measure used to quantify the relationship between two variables. It indicates the strength and direction of the linear association between them. The range of coefficient values is from -1 to 1. It is denoted as '?' ? = 1 indicates a perfect positive linear relationship.? = −1 indicates a perfect negative
7 min read
Pearson Product Moment Correlation
The Pearson product-moment correlation coefficient (or Pearson correlation coefficient) is a measure of the strength of a linear association between two variables and is denoted by r. Basically, a Pearson product-moment correlation attempts to draw a line of best fit through the data of two variables, and the Pearson correlation coefficient, r, ind
3 min read
What is Correlation Analysis?
Most of the data in the world is interrelated by various factors. Data Science deals with understanding the relationships between different variables. This helps us learn the underlying patterns and connections that can give us valuable insights. "Correlation Analysis" is an important tool used to understand the type of relation between variables.
6 min read
Reasons Why Correlation Does NOT Imply Causation
Understanding the difference between correlation and causation is crucial in data science to avoid misinterpretations that can lead to incorrect conclusions and ineffective solutions. In this article, we will explore why a correlation between two variables does not necessarily imply that one causes the other. Why doesn't correlation imply causation
5 min read
What is Canonical Correlation Analysis?
Canonical Correlation Analysis (CCA) is an advanced statistical technique used to probe the relationships between two sets of multivariate variables on the same subjects. It is particularly applicable in circumstances where multiple regression would be appropriate, but there are multiple intercorrelated outcome variables. CCA identifies and quantif
7 min read
Cross-correlation Analysis in Python
Cross-correlation analysis is a powerful technique in signal processing and time series analysis used to measure the similarity between two series at different time lags. It reveals how one series (reference) is correlated with the other (target) when shifted by a specific amount. This information is valuable in various domains, including finance (
5 min read
How to Generate Two Variables with Precise Pre-Specified Correlation in R?
Generating two variables with a precise pre-specified correlation in R involves creating two sets of data that exhibit a specific correlation coefficient between them. This can be achieved using various methods, such as generating random data or manipulating existing data to achieve the desired correlation. Below is a step-by-step guide on how to g
3 min read
Why the Result of Cross-Correlation Coefficient is 1?
Cross-correlation is a statistical measure used to assess the similarity or relationship between two signals or time series data. It is particularly useful in signal processing, econometrics, and many other fields where understanding the relationship between datasets is crucial. When calculating the cross-correlation coefficient, a value of 1 indic
4 min read
Covariance vs Correlation: Understanding Differences and Applications
Understanding the relation between variables is seen as an essential component of Machine Learning. With covariance and correlation serving as two key concepts for quantifying this relationship. Despite being often used interchangeably, covariance and correlation have unique meanings and uses. In this guide, we will understand the concepts of Covar
9 min read
Pearson Correlation Testing in R Programming
Correlation is a statistical measure that indicates how strongly two variables are related. It involves the relationship between multiple variables as well. For instance, if one is interested to know whether there is a relationship between the heights of fathers and sons, a correlation coefficient can be calculated to answer this question. Generall
5 min read
Python | Kendall Rank Correlation Coefficient
What is correlation test? The strength of the association between two variables is known as the correlation test. For instance, if we are interested to know whether there is a relationship between the heights of fathers and sons, a correlation coefficient can be calculated to answer this question. For know more about correlation please refer this.M
3 min read
Create a correlation Matrix using Python
In the field of data science and machine learning, a correlation matrix aids in understanding relationships between variables. Correlation matrix represents how different variables interact with each other. For someone who is navigating the complex landscape of data, understanding and harnessing the potential of correlation matrices is a skill that
8 min read
Machine Learning Tutorial
Machine Learning tutorial covers basic and advanced concepts, specially designed to cater to both students and experienced working professionals. This machine learning tutorial helps you gain a solid introduction to the fundamentals of machine learning and explore a wide range of techniques, including supervised, unsupervised, and reinforcement lea
8 min read
Adding new column to existing DataFrame in Pandas
Adding new columns to an existing DataFrame is a fundamental task in data analysis using Pandas . It allows you to enrich your data with additional information and facilitate further analysis and manipulation. This article will explore various methods for adding new columns, including simple assignment, the insert() method, the assign() method. Let
7 min read
Linear Regression in Machine learning
Machine Learning is a branch of Artificial intelligence that focuses on the development of algorithms and statistical models that can learn from and make predictions on data. Linear regression is also a type of machine-learning algorithm more specifically a supervised machine-learning algorithm that learns from the labelled datasets and maps the da
15+ min read
Supervised and Unsupervised learning
Machine learning is a field of computer science that gives computers the ability to learn without being explicitly programmed. Supervised learning and unsupervised learning are two main types of machine learning. In supervised learning, the machine is trained on a set of labeled data, which means that the input data is paired with the desired outpu
15 min read
Article Tags :
Practice Tags :
We use cookies to ensure you have the best browsing experience on our website. By using our site, you acknowledge that you have read and understood our Cookie Policy & Privacy Policy
Lightbox