NTD in AI: Kernel (Trick)

Non-technical definitions in AI

Kernel (in Kernel Trick): A Kernel is a group of algorithms that are used in certain machine learning problems, principally, but not exclusively, in classification problems. The kernel algorithms map the dataset into higher dimensions in the hope of separating the data into groups by this action.

As an analogy imagine you have a large number of red and blue balloons on the floor of a basketball court placed at random. These balloons are held in place by string and a weight. In classification problems you have to find the boundaries (technically Decision Boundaries) between groups of red and blue balloons, which is very difficult in this case. However, if we pump all red balloons with helium, they will reveal themselves in the 3rd dimension as they float up (but are restrained by the string and weight). We can now draw a boundary between the 2 groups, which is an imaginary plane above the blue and below the red balloons.

Adding helium is analogous to mapping the datapoints into a higher dimensional space. The kernel trick is so called because a mathematical method circumvents the need to transform all of the data. Hence in machine learning, the kernel trick reduces the need to expend time and resources to find the correct mapping functions.

Support Vector Machines (SVMs) are based on the use of Kernels.

Machine learning is a technical subject and the use of technical terms by engineers have the potential of coming between clear communication with non-engineers, especially in the business setting. In spare moments I started to put together simple, non-technical definitions of nouns and verbs used in the field of machine learning as a kind of Rosetta Stone for non-engineers.This is a work-in-progress which I may collect into a book one day. This is one of those definitions.

NTD in AI: Kernel (Trick)

Other non-technical definitions: