Kernel Density Estimated Linear Regression

Abstract

Regression analysis is a cornerstone of predictive modeling, with linear regression and kernel regression standing as two of its most prominent paradigms. However, each approach has inherent limitations: linear regression is highly susceptible to outliers in noisy and unevenly distributed datasets, while kernel regression often suffers from overfitting. When data exhibits linear trends, linear regression tends to generalize better, whereas kernel regression may fail to capture the broader patterns effectively. To address these challenges, we propose a novel methodology that integrates linear regression principles with kernel density estimation, termed Kernel Density Estimated Linear Regression (KDLR). This approach leverages kernel density estimation to assign higher weights to data points in dense regions, simultaneously de-emphasizing the influence of sparse or noisy points. This dynamic weighting mechanism enhances robustness and improves the model’s ability to recognize meaningful patterns. Extensive evaluations on datasets, including the California housing dataset with varying levels of outliers, demonstrate the efficacy of KDLR. Using the mean squared error metric, KDLR consistently achieves superior accuracy compared to traditional regression methods. By prioritizing dense data regions, it captures significant trends while mitigating the effects of noise. Applications of KDLR span diverse fields, including stock price prediction in finance, patient data modeling in healthcare, and climate modeling in environmental studies - domains where robust anomaly and noise handling are critical. This research establishes KDLR as a transformative tool for predictive analytics, offering a compelling blend of precision, resilience, and adaptability. Its ability to manage noisy data effectively while identifying critical patterns positions KDLR as a versatile and innovative solution for both academic research and industrial practice, with the potential to redefine best practices in regression modeling.