This post is also available in: Español (Spanish)
The increasing diversification of the type and volume of data, the lowering of computational processing costs and storage costs have opened a window of opportunity for the resurgence of a discipline that already existed on paper and among equations: Machine Learning.
What is Machine Learning?
Machine Learning is an artificial intelligence technique used within data analysis that automates the construction of analytical models through the iterative discovery of patterns within a data set. Using algorithms that automatically and iteratively learn from the data, the resulting analytical models are then used to solve specific problems associated with that data set.
When we speak of analytical models we refer to the whole set of logical, analytical and/or empirical rules that best describe the data set. It is how to talk about the best formulas by which the data set is governed, although they are not necessarily constant and stay the same as more data is collected.
The fact that an algorithm learns automatically means that for a given problem it is capable of learning patterns that have not been previously defined or programmed. Due to this automatic form of pattern detection, the algorithm can also automatically improve its learning with growing data sets.
To learn iteratively, algorithms simply resort to the same set of data repeatedly, with a specific objective or problem, modifying the set of internal rules (analytical model) with each iteration to adjust it as best as possible to the data set seen. The combination of these two characteristics results in algorithms that are able to update themselves with new data, always producing the best analytical model that describes the data set.
What problems can be solved with Machine Learning?
To avoid errors in the use of Machine Learning, we must, despite the automatic and iterated nature of Machine Learning, raise problems that are “known unknown”: problems that we know exist, but whose answers we do not know. Known unknown problems are often questions and curiosities that arise naturally from knowledge of a specific area of the business. They can generally be divided into three large groups:
- Descriptive (data mining): They are related to classic questions and problems that are related to BI, how there are correlations between some sales and some customers, what business patterns have existed over time or which and how many different group of customers exist that buy a certain product/service.
- Predictive (forecasts): Predictive problems arise when we want to know what and with how much certainty a particular result will take place in our business process. Usually we talk about predictions when we try to project a certain process over time, but we also talk about predictions when we try to know what kind of customers will buy our products/services, what kind of response a particular market will have or what will be the impact of certain marketing campaign.
- Prescriptive (optimization): Once the prediction has been developed for a given problem, we are then interested in knowing how to act. For relatively simple problems and predictions with low impact, decision-making is usually easy. However, with the increase of complexities and interdependence between agents, processes and results, making the best decisions turns complicated. Machine Learning allows us to tackle this type of problem by iterating and learning to always make the most optimal decision (given a desired result) in highly complex environments and contexts.
Some of the most widely publicized examples of the use of Machine Learning are:
- Recommendation of Amazon and Netflix products based on the patterns of existing customer groups.
- Follow up of the opinion of users and customers on Twitter and recognition of the generalized feeling of the opinions expressed.
- Detection of fraud in financial systems based on the search for anomalous patterns of behavior.
- Autonomous cars whose models are trained in many dimensions and concepts: recognition of images and elements of roads, identification of driving patterns or the prediction of movements of other cars.
Should I implement Machine Learning in my company?
Yes! But as long as you have passed the following general rules:
• Do I have a process or business area that generates large volumes of data?
• Is the process or area repetitive and do I have a feeling that there is a pattern in the how the process develops/behaves?
• Is there a problem or concise question that I want to answer from the data? Is there any business situation that I am interested in predicting?
What do I do to implement Machine Learning in my company?
1. Formulate and limit the problem to be solved.
2. Identify the data source and if necessary expand the collection
3. Immerse yourself in existing knowledge and use the help of professionals who have already solved similar problems.
4. Choose the correct tool to solve the chosen problem.
5. Assess the solution and its results, and iterate the previous steps until finding better solutions to the problem.
6. Deploy the best solution as a service in your organization to help in decision-making decisions or add functionality to a specific area.
Due to the high availability of data and the reduction in computer processing and storage costs, Machine Learning has been opened as a tool for the detection of patterns in data sets, with the consequent application to decision making and forecasts in a business. Machine Learning automatically allows us to identify the best analytical models in data sets to solve a given problem, either describe what happened, predict future behaviors or help in decision making in situations of high complexity.
The implementation of Machine Learning in a business must occur whenever the need to solve a problem is identified within a context with large volumes of data, a repetitive process and with some identified patterns, and if the resolution of this problem adds value through the description, prediction, or optimization of some process that helps decision-making. To implement Machine Learning, the most important thing is to identify the problem to solve. Once the problem is identified, the process of testing – solution – evaluation must be iterated until an algorithm is found that is of value.
Machine Learning in SolidQ
This post is the beginning of a series of introductory articles on Machine Learning. In our following articles, we will delve a bit more into:
- Common problems that Machine Learning solves.
- Implementation and solutions of Machine Learning with Microsoft tools.
- Examples of use in solutions applied to business problems.
At SolidQ, we have extensive experience in the Machine Learning application with Microsoft, R and Python technologies to real business problems. We are available to guide you and, if necessary, develop your Machine Learning solution for your data mining problems, predictions or optimizations. Additionally, we offer a 5-day introductory workshop on Machine Learning. More information on solidq.com.