Skip to main content Skip to navigation


Forum kernel of penalty functional

You need to be logged in to post in this topic.
  1. Shahin send the below on 19 Oct:

    I was just thinking again about what we discussed, and I noticed that constant functions
    are in the null of J(f) = \int |f’’|^2, but this couldn’t be seen with the formula J(f)
    =  \int | \tilde (\omega) |^2 \omega^4 d\omega because it’s only valid for square
    integrable functions. If f(x) is square-integrable then  f(x) + ax + b is square
    integrable iff a=b=0, so the formula is valid only for the representative with 0
    coefficient in the null space. This is why it seemed that the null space is zero for the
    frequency domain formula of J(f) given above.

    I guess this makes things more complicated, as we *want* to have use basis functions in
    the kernel of J(.) for expressing f (we want to allow for vertical shifts).

    I think we can come to the following remarks:

    1- If the penalty J(.) is given in the L2 domain, the kernel of J() is important. But we
    can use the frequency domain representation of J() to understand what it does.
    2- If the penalty J(.) is given in the frequency domain, we have to remember that it is
    only valid on L2 integrable functions f. It is not clear what we do with intercept
    shifts, etc (or do we?)
    3- If a kernel is specified, then it is implicitly assumed that we only look at functions
    orthogonal to the null space of the induced J(.) — that’s what the representer theorem
    says, essentially.

    Does this make sense?


  2. hi shahin,

    this makes sense to me, indeed. and thanks for bringing this up. was great you came

    one way to bring in intercepts could be to use a cut-off for R^d. the corresponding
    kernels would then be based on eigen-expansions where the eigenfunctions would play the role of "waves" with certain frequencies. have to think about this.