### Probabilistic and Statistical Methods in Computer Science

However, saying machine learning is all about accurate predictions whereas statistical models are designed for inference is almost a meaningless statement unless you are well versed in these concepts. Firstly, we must understand that statistics and statistical models are not the same.

Statistics is the mathematical study of data. You cannot do statistics unless you have data. A statistical model is a model for the data that is used either to infer something about the relationships within the data or to create a model that is able to predict future values. Often, these two go hand-in-hand. So there are actually two things we need to discuss: firstly, how is statistics different from machine learning, and secondly, how are statistical models different from machine learning. To make this slightly more explicit, there are lots of statistical models that can make predictions, but predictive accuracy is not their strength.

Likewise, machine learning models provide various degrees of interpretability, from the highly interpretable lasso regression to impenetrable neural networks , but they generally sacrifice interpretability for predictive power.

## Applied Mathematics and Statistics | Peer Reviewed Journal

From a high-level perspective, this is a good answer. Good enough for most people.

However, there are cases where this explanation leaves us with a misunderstanding about the differences between machine learning and statistical modeling. Let us look at the example of linear regression. It seems to me that the similarity of methods that are used in statistical modeling and in machine learning has caused people to assume that they are the same thing.

This is understandable, but simply not true. The most obvious example is the case of linear regression, which is probably the major cause of this misunderstanding. Linear regression is a statistical method, we can train a linear regressor and obtain the same outcome as a statistical regression model aiming to minimize the squared error between data points.

### Organisation

The purpose of machine learning, in this case, is to obtain the best performance on the test set. For the statistical model, we find a line that minimizes the mean squared error across all of the data, assuming the data to be a linear regressor with some random noise added, which is typically Gaussian in nature.

No training and no test set are necessary. For many cases, especially in research such as the sensor example below , the point of our model is to characterize the relationship between the data and our outcome variable, not to make predictions about future data. We call this procedure statistical inference, as opposed to prediction.

However, we can still use this model to make predictions, and this may be your primary purpose, but the way the model is evaluated will not involve a test set and will instead involve evaluating the significance and robustness of the model parameters. The purpose of supervised machine learning is obtaining a model that can make repeatable predictions. We typically do not care if the model is interpretable, although I would personally recommend always testing to ensure that model predictions do make sense.

Machine learning is all about results, it is likely working in a company where your worth is characterized solely by your performance. Whereas, statistical modeling is more about finding relationships between variables and the significance of those relationships, whilst also catering for prediction.

## Probability and Statistics in Data Science using Python

To give a concrete example of the difference between these two procedures, I will give a personal example. By day, I am an environmental scientist and I work primarily with sensor data. If I am trying to prove that a sensor is able to respond to a certain kind of stimuli such as a concentration of a gas , then I would use a statistical model to determine whether the signal response is statistically significant. I would try to understand this relationship and test for its repeatability so that I can accurately characterize the sensor response and make inferences based on this data.

Some things I might test are whether the response is, in fact, linear, whether the response can be attributed to the gas concentration and not random noise in the sensor, etc. In contrast, I can also get an array of 20 different sensors, and I can use this to try and predict the response of my newly characterized sensor.

This may seem a bit strange if you do not know much about sensors, but this is currently an important area of environmental science. A model with 20 different variables predicting the outcome of my sensor is clearly all about prediction, and I do not expect it to be particularly interpretable. This model would likely be something a bit more esoteric like a neural network due to non-linearities arising from chemical kinetics and the relationship between physical variables and gas concentrations.

I would like the model to make sense, but as long as I can make accurate predictions I would be pretty happy.

## Statistical Methods for Data Science

If I am trying to prove the relationship between my data variables to a degree of statistical significance so that I can publish it in a scientific paper, I would use a statistical model and not machine learning. This is because I care more about the relationship between the variables as opposed to making a prediction. Making predictions may still be important, but the lack of interpretability afforded by most machine learning algorithms makes it difficult to prove relationships within the data this is actually a big problem in academic research now, with researchers using algorithms that they do not understand and obtaining specious inferences.

It should be clear that these two approaches are different in their goal, despite using similar means to get there. The assessment of the machine learning algorithm uses a test set to validate its accuracy.

1. Unsustainable Transport: City Transport in the New Century (Transport, Development and Sustainability Series).
2. A Piece of Me!
3. Welcome to the Illinois Computer Science Department homepage!!

Since these methods produce the same result, it is easy to see why one might assume that they are the same. I think this misconception is quite well encapsulated in this ostensibly witty year challenge comparing statistics and machine learning. However, conflating these two terms based solely on the fact that they both leverage the same fundamental notions of probability is unjustified.

For example, if we make the statement that machine learning is simply glorified statistics based on this fact, we could also make the following statements. Physics is just glorified mathematics. Zoology is just glorified stamp collection. Architecture is just glorified sand-castle construction. These statements especially the last one are pretty ridiculous and all based on this idea of conflating terms that are built upon similar ideas pun intended for the architecture example. In actuality, physics is built upon mathematics, it is the application of mathematics to understand physical phenomena present in reality.

Physics also includes aspects of statistics, and the modern form of statistics is typically built from a framework consisting of Zermelo-Frankel set theory combined with measure theory to produce probability spaces. They both have a lot in common because they come from a similar origin and apply similar ideas to reach a logical conclusion.

Similarly, architecture and sand-castle construction probably have a lot in common — although I am not an architect so I cannot give an informed explanation — but they are clearly not the same. To give you a scope of how far this debate goes, there is actually a paper published in Nature Methods which outlines the difference between statistics and machine learning.

This idea might seem laughable, but it is kind of sad that this level of discussion is necessary. Before we go on further, I will quickly clear up two other common misconceptions that are related to machine learning and statistics. These are that AI is different from machine learning and that data science is different from statistics.

These are fairly uncontested issues so it will be quick. JavaScript is currently disabled, this site works much better if you enable JavaScript in your browser. Free Preview. Buy eBook. Buy Hardcover. Buy Softcover. FAQ Policy. Undergraduate Programs for a Major or Concentration in Statistics Statistics is the art and science of study design and data analysis. Advising Requirements. Statistics Programs M.