Event

Humans have a remarkable ability to infer shape from shading (SFS) information. In computer vision this is often formulated with a Lambertian reflectance function, but it remains under-posed and incompletely solved.  Abstractly, the intensity in an image is a single valued function and the goal is to uncover the vector valued normal function. This ill-posedness has resulted in many proposed techniques that are either regularizations or propagations from known values. Our goal is to understand, mathematically and computationally, how we solve this problem.

First, it has been shown psychophysically that our perception (via gauge figure estimates) is remarkably accurate even when the boundary is masked.  Thus classical propagating approaches requiring a known values along a boundary, such as that of characteristic curves or fast marching methods,  are unlikely to model the visual system's solution.

An alternative approach requires regularization priors (in a Bayesian framework) or energy terms (in a variational framework).  However, many of the proposed priors are ad-hoc and chosen by researchers to optimize performance for a particular test dataset.  It is hard to conclude (from solely performance metrics) whether these priors are useful or accurate, e.g.  good results are functions of these priors, resolution, the optimization techniques, the test set, and so on.

In this talk, we describe a different approach. We consider the SFS problem on image patches modeled as Taylor polynomials of any order and seek to recover a solution for that patch.  We build a boot-strapping tensor framework that allows us to relate a smooth image patch to all of the polynomial surface solutions (under any light source).  We then use a generic constraint on the light source to restrict these solutions to a 2-D subspace, plus an unknown rotation matrix.  We then investigate several special cases where the ambiguity reduces and the solution can be anchored. Interestingly, these anchor solutions relate to those situations in which human performance is also veridical.