Career Insights: All You Need to Know About Machine Learning ...

Career Insights: All You Need to Know About Machine Learning ...

Posted on November 30, 2017

What does Machine Learning mean? Where does it come from? What are its applications? Why are the employers in particular adopting it? What are its pros and cons? All Answers to All Your Questions in this blog post.

I have been fascinated by machine learning since I was a boy. My mother has some anecdotes about me being fascinated by an idea or theory which I didn’t understand completely, but had to know about. This fascination with learning about something new has stayed with me to this day. So much that even my love of travelling and visiting places has been fueled by these inquiring spirit that drove me to learn. A constant drive to learn and to get to understand the bigger picture of everything that is around us. To me, it just makes sense.

Over the years since childhood, I have been fortunate enough to build a career in the field of technology (primarily Information Technology though I have been in the realm of the data driven for over a decade now). And while technology has always fascinated me, so has machine learning (ML). Now you might be wondering – what is ML and how does it work? So let’s delve deep into the concept of Machine Learning in this post.

The most fundamental definition of Machine Learning (ML) is that it involves algorithms that are designed to learn on its own based on data. So it is about generating programs which can help or assist you. That can be done without human intervention in most cases. That is the fundamental and most common definition of an ML system. However, there are several more advanced definitions based on complexity and capability of the program. In simple terms, it means that while an ML program has a limited ability at being creative in a similar way as a human brain can be creative to suit its needs. And a machine learning model is also referred to as an AI. I personally don’t prefer calling them this but that’s just me.

So for today, we will explore the basic definition of an ML system and take a look at different types of machine learning algorithms that exist. Let’s set some context for it – it’s not a super-tech subject; in fact, all you need to know will be covered in this post.

Machine Learning is about computers being programmed to make predictions based on a set of known rules. That set of rules is called the training set and is fed into the system to be trained. By feeding the data into the system, the program learns on its own, depending on the complexity and the capability of the program. Now based on that data fed into the system, it generates or comes up with some set of rules which help identify unknown data that is fed into it. In that sense, it is quite akin to a basic logic in a computer.

What happens with machine learning, though, is that it can learn over time and over many different situations based on the data available and can generate some insight from it. In that sense, it’s a bit like a human brain that is more analytical and is creative to make sure that it delivers the right things to the user. That’s what it is all about.

Now let’s take a look at the different types of algorithms used in machine learning. We could broadly categorize them as follows –

Statistical ML models such as Regression, Classification and Clustering.

These algorithms look at the data at hand and try to draw conclusions related to the data.

Neural network based ML algorithms such as Neural Networks, Reinforcement Learning, Genetic Algorithms and Deep Network Learners such as Convolutional Networks and Recurrent Neural Networks

These are models in which the data sets are processed by applying a layer of layers of filters to them

This is the most commonly use machine learning algorithms. However, they are based on data in a traditional model. And what that means is that the data needs to be available in a very conventional format. For example, the traditional data formats usually have data in rows and columns and each row is called a vector. Every column is called a feature which can have a range of different values. And each feature value is assigned some statistical value. This is very specific format, and as it usually involves using the same statistical value for each feature value. For example, every value for the “Age” feature will have the same range and statistical value. So ML does not need to learn or figure out by itself. It learns when the data is fed into it. So it’s pretty much a manual process.

Then it has advanced machine learning models such as those involving Neural Networks. And when using such models, we try to give them our data in an already normal or understandable format. So that the machine, if not the programmer, understands what we are talking about. This is done by converting our data into that format first before feeding it into them. The neural networks, thus, would take the data in that format and automatically calculate some meaning out of it. Neural networks are one such advanced model which is very data and feature driven. That is to say – the output is determined by the nature of the data inputted into it. Based on the format (which is also a data type in itself) of your data, it determines what your output will be. That is to say, it looks at all the features or data inputted into it and determines what is the value which needs to be associated with the given feature. So we feed the data in such feature-value format, and depending on there nature or properties, it will determine what the output will be in the format that we understand. It’s very data driven.

Then there are some other statistical based machine learning models such as Regression, Classification and Clustering that we mentioned earlier. These are the algorithms which are designed to learn on their own based on data. This is where data is fed into the system to be trained. The trained data is referred to as the testing set and is used to give insights. Like – the trained data helps determine what is the right way to perform the task. And it’s usually given a rating or a rank for the data. The model is then able to repeat this process on other data that may be given to it. It usually does this over the number of data points or iterations. So a pattern is repeated with the data over and over. The system also helps create rules for the data set. However, the training process is not only about getting the data to follow a rule or make a prediction rather it is also about learning and figuring out what the best model structure for the data is. Like -it is about trying different combinations to train the data sets and figure out the one that works best.

We already discussed the statistical-based models. Those are the most common ones for today. But we still have the neural network models. There are a few more. These will be further discussed in the rest of this post. Let’s take a quick look at some of the types of algorithms used in Machine Learning. This is to have a better idea of what is Machine Learning and all that it entails.

Before starting on ML algorithms, let’s look at what Machine Learning is as a concept. We have seen that Machine Learning is based on data fed in. Like – we provide the data set, and the machine learns on its own, and generates a rule, a model(regression, classification etc), which it can share with us. That way, we can use the model to generate some prediction or give us some recommendations.

But we can also take a more qualitative side to this. Think of it this way – Machine learning is all about the data. The data is the fuel which gives the machine (in this case -an algorithm or ML model) life and drives it to generate an insight (which may be a prediction given a particular data set or a model which can be used to predict future outcomes given similar data). That is to say, the data that you feed in to the algorithm or model is the basis for the output of the algorithm or model’s activity. So data (the input that the model or algorithm is trained on) is what is the fuel for the model. What fuels machine learning is the data. And that data is then used by the Model to create something that is useful in a given context.

The next step would be to look at the training set/data that is fed into the ML model. Typically, the data set or training set will be provided to the model by the user. However, this step is where the training data is generated. This training data is then fed in to the model and the model is trained on it, and this is where the learning actually takes place.

Next step is to see how each model can be classified. We have seen that a model can be classified based on the type of output. For example, classification of models is mostly based on the output that the model can generate. Most commonly, the output is a yes or a no or, based on the number of the input data points, the model can also predict how many points it is certain will follow the rules or will fit the model. So, for example, you can use a simple logistic regression. We use that to predict what a customer will do. In this case, it will predict how many clicks/sales a customer would be making on your website. Other popular categories of models for classification based on output is that a model can be a multi-class classification. Here we will have different categories of items or objects which the model can be used to assign. We also have case based models – these work on the principle of using the data set to create the model.