How to understand reproducing kernel hilbert space (RKHS)?
RKHS is a vector space with specific properties, e.g., inner product, normed. A kernel function uniquely determine a RKHS. A vector space is a space with a linear structure, meaning, we can add elements of the vector space or add multiple elements with scalars to obtain another element.
Essentially, RKHS lets us apply concepts from finite-dimensional linear algebra to infinite-dimensional space of functions. The fact that Hilbert space is complete guarantees that certain algorithm will converge. More importantly, the presence of inner product allows us to use orthogonality and projection.
A Hilbert space is a RKHS if the evaluation functional is bounded. This condition is the weakest condition that ensures both the existence of inner product and the ability to evaluate each function at every point in the domain.