Like most linear models, Ordinary Least Squares is a fast, efficient algorithm. You can implement it with a dusty old machine and still get pretty good results.
Similar to Logistic Regression (which came soon after OLS in history), Linear Regression has been a breakthrough in statistical applications.
It has been used to identify countless patterns and predict countless values in countless domains all over the world in last couple of centuries.
With its computationally efficient and usually accurate nature, Ordinary Least Squares and other Linear Regression extensions remain popular both in academia and the industry.
If you have outliers that you'd like to observe. Or if you want to conclude unexpected black-swan like scenarios this is not the model for you.
Like most Regression models, OLS Linear Regression is a generalist algorithm that will produce trend conforming results.
With Linear Models such as OLS (also similar in Logistic Regression scenario), you can get rich statistical insights that some other advanced or advantageous models can't provide.
If you are after sophisticated discoveries for direct interpretation or to create inputs for other systems and models Ordinary Linear Squares algorithm can generate a plethora of insightful results ranging from, variance, covariance, partial regression, residual plots and influence measures.
For this feature OLS can be viewed as a perfect supportive Machine Learning Algorithm that will complete and compete with most modern algorithms. Just keep the limitations in mind and keep on exploring!
Linear Regression in general is nothing like k Nearest Neighbors. It can be considered very distant relatives with Naive Bayes for its mathematical roots however, there are so many technical aspects to learn in the regression world.
This is more like an opportunity to learn about statistics and intricacies of datasets however, it's also definitely something that takes away from practicality and will discourage some of the time conscious, result oriented folks.
Ordinary Least Squares won't work well with non-linear data. If you are not sure about the linearity or if you know your data has non-linear relations then this is a giveaway that most likely Ordinary Least Squares won't perform well for you at this time.
If you'd like to predict outliers or if you want to conclude unexpected black-swan like scenarios this is not the model for you.
Like most Regression models, OLS Linear Regression is a generalist algorithm that will produce trend conforming results.
Just because OLS is not likely to predict outlier scenarios doesn't mean OLS won't tend to overfit on outliers. Ordinary Least Squares is an inherently sensitive model which requires careful tweaking of regularization parameters.
When you enter the world of regularization you might realize that this requires an intense knowledge of data and getting really hands-on.
There is no one regularization method that fits it all and it's not that intuitive to grasp very quickly. So, not to say there is no merit in these efforts and discussions, it might discourage someone seeking a more practical application or the general crowd.
It's also worth noting that perfect regularization can be difficult to validate and time consuming. On the other hand it's quite important to get it right because if you under do it you will risk overfitting on irrelevant features and if you over do it the risk is to miss out on important features that might be valuable/relevant for future predictions.
Linear Regression is fast and scalable. It's not very resource-hungry.
Scalability also means you can work on big data problems.
Statistical output you are able to produce with a Ordinary Least Squares far outweighs the trouble of data preparation (given that you are after the statistical output and deep exploration of your data and all its relation/causalities.)
You don't survive 200 something years of heavy academia and industry utilization and happen not to have any modifications. Once you open the box of Linear Regression, you discover a world of optimization, modification and extensions (OLS, WLS, ALS, Lasso, Ridge, Logistic Regression just to name a few).
If your problem has non-linear tendencies Linear Regression is instantly irrelevant.
Another problem is when data has noise or outlier and Linear Regression tends to overfit. Discovering and getting rid of overfitting can be another pain point for the unwilling practitioner. And even if you are willing, at times it can be difficult to reach optimal setup.
Regularization, handling missing values, scaling, normalization and data preparation can be tedious.
Even interpreting the results of Linear Regression as they are intended in a meaningful way can take some education which makes it a bit less appealing to non-statistical audience.