Shahin send the below on 19 Oct:
I was just thinking again about what we discussed, and I noticed that constant functions
are in the null of J(f) = \int |f’’|^2, but this couldn’t be seen with the formula J(f)
= \int | \tilde (\omega) |^2 \omega^4 d\omega because it’s only valid for square
integrable functions. If f(x) is square-integrable then f(x) + ax + b is square
integrable iff a=b=0, so the formula is valid only for the representative with 0
coefficient in the null space. This is why it seemed that the null space is zero for the
frequency domain formula of J(f) given above.
I guess this makes things more complicated, as we *want* to have use basis functions in
the kernel of J(.) for expressing f (we want to allow for vertical shifts).
I think we can come to the following remarks:
1- If the penalty J(.) is given in the L2 domain, the kernel of J() is important. But we
can use the frequency domain representation of J() to understand what it does.
2- If the penalty J(.) is given in the frequency domain, we have to remember that it is
only valid on L2 integrable functions f. It is not clear what we do with intercept
shifts, etc (or do we?)
3- If a kernel is specified, then it is implicitly assumed that we only look at functions
orthogonal to the null space of the induced J(.) — that’s what the representer theorem
says, essentially.
Does this make sense?
Cheers,
Shahin