- Support Vector Machine History

- 29 July 2020
- by HolyPython

# Contributions of a prolific mind

From steppes of Central Asia to Murray Hill, New Jersey

## Statistical progress continues

Support Vector Machine algorithm is yet another *flag race* in the history.

There are chain of events that lead to the invention of support vector machines generally dating back to the middle of 20th century:

**In 1950 Aronszajn** publishes the “Theory of Reproducing Kernels”. **In 1957 Frank Rosenblatt** took this idea and invented perceptron, a simple linear classifier.

**6 years later Vapnik and Lerner** comes and announces “Generalized Portrait Algorithm” (1963). This was the true inspiration for Boser, Guyon and Vapnik’s 1992 paper at the COLT conference introducing **Support Vector Machines**.

The original paper, **“A training algorithm for optimal margin classifiers” (1992)**, can be found here (*ISBN* *978-0897914970)*.

Vapnik’s work were originally in Russian then and have been translated to English and German in the following years.

## Vladimir Vapnik

Vladimir Vapnik, born in 1936, studied Mathematics at Uzbek State University (then in Soviet Union) and moved to the US in 1990 to work with AT&T Bell Labs (now Nokia Bell Labs) shortly before publishing his major work in SVM.

His book Statistical Learning Theory (1998) has been outstandingly cited more than 60.000 times. Book here.

Since supervised machine learning techniques cannot be used with unlabeled data, **Vapnik with Hava Siegelmann **** also developed SVC (Support Vector Clustering) an unsupervised extension of Support Vector Machines in November 2001. **You can see the research paper here.

Vapnik also founded VC theory with Chervonenkis (**Vapnik–Chervonenkis theory** ), a theory in computational learning.

## Further Advancements

The Nature of Statistical Learning Theory (1995)

In 1998 Shawe, Taylor et al. made a significant contribution to the generalization of hard margin Support Vector Machines. Shawe, Taylor, Cristianini then gave statistical bounds to the generalization of soft margin Support Vector Machines in 2000.

## Finishing Thoughts

**I think when it comes to Support Vector Machines, Vapnik stands out as a true legend** that contributed to the statistical learning for decades, fathered SVMs along with Boser and Guyon, continued to advance the models and then founded SVC (Support Vector Clustering) with Siegelmann.

Vapnik is still alive and quite active. He resides in New Jersey, USA.