According to the similarity of the function and form of the algorithm, we can classify the algorithm, such as tree-based algorithm, neural network-based algorithm, and so on. Of course, the scope of machine learning is very large, and it is difficult for some algorithms to be clearly classified into a certain category.
13 Common Algorithms of Machine Learning
1) Regression algorithm
Regression algorithm is a type of algorithm that tries to explore the relationship between variables by using a measure of error. Regression algorithm is a powerful tool for statistical machine learning. In the field of machine learning, when people talk about regression, sometimes they refer to a type of problem and sometimes a type of algorithm. This often confuses beginners.
Common regression algorithms include: Ordinary Least Square, Logistic Regression, Stepwise Regression, Multivariate Adaptive Regression Splines, and Locally Estimated Scatterplot Smoothing).
2) Regularization method
The regularization method is an extension of other algorithms (usually regression algorithms), and the algorithm is adjusted according to the complexity of the algorithm. Regularization methods usually reward simple models and penalize complex algorithms.
Common algorithms include: Ridge Regression, Least Absolute Shrinkage and Selection Operator (LASSO), and Elastic Net.
3) Decision tree learning
The decision tree algorithm uses a tree structure to establish a decision model based on the attributes of the data. The decision tree model is often used to solve classification and regression problems.
Common algorithms include: Classification and Regression Tree (CART), ID3 (Iterative Dichotomiser 3), C4.5, Chi-squared Automatic Interaction Detection (CHAID), Decision Stump, Random Forest (Random Forest), multivariate Adaptive regression spline (MARS) and gradient boosting machine (Gradient Boosting Machine, GBM)
4) Example-based algorithm
Case-based algorithms are often used to model decision-making problems. Such models often first select a batch of sample data, and then compare the new data with the sample data based on some similarity. In this way, the best match is found. Therefore, instance-based algorithms are often referred to as “winner-takes-all” learning or “memory-based learning”.
Common algorithms include k-Nearest Neighbor (KNN), Learning Vector Quantization (LVQ), and Self-Organizing Map (SOM).
5) Bayesian method
Bayesian method algorithm is a kind of algorithm based on Bayes theorem, mainly used to solve classification and regression problems.
Common algorithms include: Naive Bayes algorithm, Averaged One-Dependence Estimators (AODE), and Bayesian Belief Network (BBN).
6) Clustering algorithm
Clustering, like regression, sometimes people describe a type of problem, and sometimes a type of algorithm. Clustering algorithms usually merge the input data in a central point or hierarchical manner. All clustering algorithms try to find the internal structure of the data in order to classify the data according to the biggest common point.
Common clustering algorithms include k-Means algorithm and Expectation Maximization (EM).
7) Reduce dimensionality algorithm
Like clustering algorithms, dimensionality reduction algorithms try to analyze the internal structure of the data, but dimensionality reduction algorithms try to use less information to summarize or interpret data in an unsupervised learning manner. This type of algorithm can be used to visualize high-dimensional data or to simplify data for supervised learning.
Common algorithms include: Principle Component Analysis (PCA), Partial Least Square Regression (PLS), Sammon mapping, Multidimensional Scaling (MDS), Projection Pursuit (Projection Pursuit) Wait.
8) Association rule learning
Association rule learning finds useful association rules in a large number of multivariate data sets by finding the rules that best explain the relationship between data variables.
Common algorithms include Apriori algorithm and Eclat algorithm.
9) Genetic algorithm (genetic algorithm)
The genetic algorithm simulates the mutation, exchange and Darwinian natural selection of biological reproduction (the survival of the fittest in every ecological environment).
It encodes the possible solutions of the problem into a vector, called an individual, each element of the vector is called a gene, and uses an objective function (corresponding to the natural selection criteria) to evaluate each individual in the group (a collection of individuals).
According to the evaluation value (fitness), genetic operations such as selection, exchange and mutation are performed on individuals to obtain a new population.
Genetic algorithms are suitable for very complex and difficult environments, such as with a lot of noise and irrelevant data, things are constantly updated, problem goals cannot be clearly and accurately defined, and the value of current behavior can be determined through a long execution process.
10) Artificial Neural Network
Artificial neural network algorithm simulates biological neural network and are a type of pattern matching algorithm. Usually used to solve classification and regression problems. Artificial neural networks are a huge branch of machine learning, with hundreds of different algorithms.
(Deep learning is one of the algorithms, we will discuss it separately), important artificial neural network algorithms include: Perceptron Neural Network, Back Propagation, Hopfield Network, Self-Organizing Map ( Self-Organizing Map, SOM).
11) Deep learning
Deep learning algorithms are the development of artificial intelligence . It has won a lot of attention recently, especially after Baidu has also begun to work hard on deep learning, which has attracted a lot of attention. With computing power becoming increasingly cheap today, deep learning is trying to build a much larger and more complex neural network.
Many deep learning algorithms are semi-supervised learning algorithms, which are used to process large data sets with a small amount of unidentified data.
Common deep learning algorithms include: Restricted Boltzmann Machine (RBN), Deep Belief Networks (DBN), Convolutional Network (Convolutional Network), and Stacked Auto-encoders.
12) Core-based algorithm
The most famous of kernel-based algorithms is the support vector machine (SVM). The kernel-based algorithm maps the input data to a high-order vector space. In these high-order vector spaces, some classification or regression problems can be solved more easily.
Common kernel-based algorithms include: Support Vector Machine (SVM), Radial Basis Function (RBF), and Linear Discriminate Analysis (LDA), etc.
13) Integrated algorithm
The ensemble algorithm uses some relatively weak learning models to independently train the same samples, and then integrates the results for overall prediction. The main difficulty of the integrated algorithm is which independent weaker learning models are integrated and how to integrate the learning results. This is a very powerful algorithm and also very popular.
Common algorithms include: Boosting, Bootstrapped Aggregation (Bagging), AdaBoost, Stacked Generalization (Stacked Generalization, Blending), Gradient Boosting Machine (GBM), Random Forest (Random Forest), GBDT (Gradient Boosting Decision Tree) .
4 Main Learning Methods of Machine Learning
There are many algorithms for machine learning. Many times, people are confused. Many algorithms are a type of algorithm, and some algorithms are extended from other algorithms. Here, we will introduce to you from two aspects. The first aspect is the way of learning, and the second aspect is the classification of algorithms.
1) Supervised learning
Under supervised learning, the input data is called “training data”, and each set of training data has a clear identification or result, such as “spam” and “non-spam” in the anti-spam system, and recognition of handwritten numbers “1”, “2”, “3”, “4” and so on.
When building a predictive model, supervised learning establishes a learning process that compares the predictive results with the actual results of the “training data”, and continuously adjusts the predictive model until the predictive result of the model reaches an expected accuracy rate.
Common application scenarios of supervised learning are classification problems and regression problems. Common algorithms are Logistic Regression and Back Propagation Neural Network.
2) Reinforcement learning
In this learning mode, the input data is used as feedback to the model. Unlike the supervision model, the input data is only used as a way to check whether the model is right or wrong. Under reinforcement learning, the input data is directly fed back to the model. Make adjustments immediately.
Common application scenarios include dynamic systems and robot control. Common algorithms include Q-Learning and Temporal difference learning.
3) Unsupervised learning
In unsupervised learning, the data is not specifically identified, and the learning model is to infer some internal structure of the data.Popular application scenarios involve association rules and clustering learning. Common algorithms include Apriori algorithm and k-Means algorithm.
4) Semi-supervised learning
In this learning mode, part of the input data is identified and part is not. This learning model can be used to make predictions, but the model first needs to learn the internal structure of the data in order to organize the data reasonably to make predictions.
Application scenarios include classification and regression. Algorithms include some extensions to commonly used supervised learning algorithms. These algorithms first try to model unidentified data, and then predict the identified data on this basis. Graph inference algorithm (Graph Inference) or Laplacian support vector machine (Laplacian SVM.) etc.
