On a Lie group, exp is a map from the Lie algebra to its Lie group. If you think of the Lie algebra as the tangent space to the identity of the Lie group, exp(v) is defined to be h(1), where h is the unique Lie group homeomorphism from the real numbers to the Lie group such that its velocity at time 0 is v. On a Riemannian manifold, exp is a map from the tangent bundle of the manifold to the manifold, and exp(v) is defined to be h(1), where h is the unique geodesic traveling through the base-point of v such that its velocity at time 0 is v.

exponential function | matrix exponential