Kernel methods

Kernel methods

Kernel Methods (KMs) are a class of algorithms for pattern analysis, whose best known elementis the Support Vector Machine (SVM). The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in general types of data (such as sequences, text documents, sets of points, vectors, images, etc).

KMs approach the problem by mapping the data into a high dimensional feature space, where each co-ordinate corresponds to one feature of the data items, transforming the data into a set of points in a Euclidean space. In that space, a variety of methods can be used to find relations in the data. Since the mapping can be quite general (not necessarily linear, for example), the relations found in this way are accordingly very general. This approach is called the kernel trick.

KMs owe their name to the use of kernel functions, that enable them to operate in the feature space without ever computing the coordinates of the data in that space, but rather by simply computing the inner products between the images of all pairs of data in the feature space. This operation is often computationally cheaper than the explicit computation of the coordinates. Kernel functions have been introduced for sequence data, text, images, as well as vectors.

Algorithms capable of operating with kernels include SVM, Fisher's linear discriminant analysis (LDA), principal components analysis (PCA), canonical correlation analysis, ridge regression, spectral clustering, and many others.

Because of the particular culture of the research community that has been developing this approach since the mid-1990s, most kernel algorithms are based on convex optimization or eigenproblems, are computationally efficient and statistically well-founded. Typically, their statistical properties are analyzed using statistical learning theory.

Applications

At the moment, the main application areas are in geostatistics, kriging, Inverse distance weighting, bioinformatics, text categorization, and handwriting recognition.

Since any kernel can be used with any kernel-algorithm, it is possible to construct exotic combinations such as: regression over biosequences; classification of documents; clustering of images; and so on.

External links

* [http://www.kernel-machines.org Kernel-Machines Org] -- community website
* [http://www.support-vector-machines.org www.support-vector-machines.org] "(Literature, Review, Software, Links related to Support Vector Machines - Academic Site)"


Wikimedia Foundation. 2010.

Игры ⚽ Поможем решить контрольную работу

Look at other dictionaries:

  • Kernel principal component analysis — (kernel PCA) is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are done in a reproducing kernel Hilbert space with a non linear mapping.ExampleThe two …   Wikipedia

  • Kernel(Maschinelles Lernen) — Im Bereich des Maschinellen Lernens wurden in den letzten Jahren eine Klasse von Algorithmen entwickelt, die sich eines Kernels (dt. Kern) bedienen, um ihre Berechnungen implizit in einem hochdimensionalen Raum auszuführen. Bekannte Algorithmen,… …   Deutsch Wikipedia

  • Kernel regression — Not to be confused with Kernel principal component analysis. The kernel regression is a non parametric technique in statistics to estimate the conditional expectation of a random variable. The objective is to find a non linear relation between a… …   Wikipedia

  • Kernel (Maschinelles Lernen) — Im Bereich des Maschinellen Lernens wurden in den letzten Jahren eine Klasse von Algorithmen entwickelt, die sich eines Kernels (dt. Kern) bedienen, um ihre Berechnungen implizit in einem hochdimensionalen Raum auszuführen. Bekannte Algorithmen,… …   Deutsch Wikipedia

  • Kernel smoother — A kernel smoother is a statistical technique for estimating a real valued function f(X),,left( Xin mathbb{R}^{p} ight) by using its noisy observations, when no parametric model for this function is known. The estimated function is smooth, and the …   Wikipedia

  • Kernel trick — In machine learning, the kernel trick is a method for using a linear classifier algorithm to solve a non linear problem by mapping the original non linear observations into a higher dimensional space, where the linear classifier is subsequently… …   Wikipedia

  • Kernel density estimation — of 100 normally distributed random numbers using different smoothing bandwidths. In statistics, kernel density estimation is a non parametric way of estimating the probability density function of a random variable. Kernel density estimation is a… …   Wikipedia

  • Kernel Patch Protection — (KPP), informally known as PatchGuard, is a feature of x64 editions of Microsoft Windows that prevents patching the kernel. It was first introduced in 2005 with the x64 editions of Windows XP and Windows Server 2003 Service Pack 1.cite web… …   Wikipedia

  • Kernel (computer science) — In computer science, the kernel is the central component of most computer operating systems (OS). Its responsibilities include managing the system s resources (the communication between hardware and software components). As a basic component of… …   Wikipedia

  • Kernel (computing) — A kernel connects the application software to the hardware of a computer In computing, the kernel is the main component of most computer operating systems; it is a bridge between applications and the actual data processing done at the hardware… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”