What is a Decision Boundary?

A decision boundary is a line (in the case of two features), where all (or most) samples of one class are on one side of that line, and all samples of the other class are on the opposite side of the line. The line separates one class from the other. If you have more than two features, the decision boundary is not a line, but a (hyper)-plane in the dimension of your feature space.

It is useful to note that different algorithms are capable of drawing different types of decision boundaries. There are some algorithms, for example, that can only draw straight lines (or flat hyper-planes). When your features make weird shapes (imagine, for example, a feature for one class creating a banana shape), it can be quite hard to draw lines or planes that separate all samples of one class from all samples of the other class. In this case, you need a more complex algorithm that can draw curved lines or hyperplanes. There are disadvantages to using more complex algorithms