Bayesian Inference with Constraints Through Projection and Randomized Linear Least Squares Problems

In many applications, the signal that is being reconstructed must satisfy some constraints, e.g., material densities being positive, pixels of an image lying between 0 and 1, or the energy of a signal being bounded. Therefore, adding such prior information to a Bayesian model improves the meaningfulness of the resulting posterior distribution.

A common method to model constraints in a prior distribution is truncation, i.e., choosing an unconstrained prior distribution and remove all probability outside of the constraint set. Another method is to reparametrize the signal, e.g., if the signal x is assumed to be positive, then it can be written as x = exp(z) and one can choose an unconstrained prior distribution for z. These two methods focus on the interior of the constraint set, but very often, signals of interest lie on the boundary of the constraint set.

As natural alternative, one can first sample from an unconstrained posterior distribution and then apply a suitable projection onto a convex constraint set C. The resulting implicit posterior will consist of the original density in the interior of the constraint set, and a “lower dimensional” distribution on the boundary, see the illustration below.

If the forward model is linear and both the likelihood and prior are Gaussian distributions, then the unconstrained sampling and projection can be done in a single step by repeatedly solving randomized constrained linear least square problems of the form

min_x∈C || A x - b_s ||₂² + α || x - c_s ||₂² ,

where b_s and c_s are samples from suitable random distributions.  The solutions of these optimization problems will be samples of a projected posterior.

Consider the problem of deblurring a noisy signal with prior knowledge that the signal lies in the interval [0,1]. The figure above shows the pointwise median and 99% credibility interval for such a problem using no constraints (left), only the lower bound (middle) and both upper and lower bounds (right). The projection framework significantly improves the quality of the median near the boundary.

This example was created by PhD student Jasper Everink.

UQ with Nonnegativity Constraints

Nonnegativity constraints appear in many applications where the underlying physics dictates that the solution cannot be negative. This is the case, e.g., for absorption coefficients in computed tomography, image intensities in astronomical imaging, and wave velocities in seismic travel-time tomography. Hence, nonnegativity constraints are very important for producing meaningful reconstructions.

We considers computational aspects of UQ with nonnegativity constraints x ≥ 0. If we replace the nonnegativity constraint with a positivity constraint, x > 0, then we can handle this constraint within the "standard" UQ framework with a transformation of variables x = exp(z). In this work we specifically address situations where we expect the existence of zero elements in the solution – in fact, in some imaging problems a substantial fraction of the solution's elements/pixels may be zero. Therefore we use a nonnegative Gaussian distribution.

The above figure shows a simple Gauss distribution (left), a truncated Gauss distribution with zero probability for x = 0 (middle), and a nonnegative Gaussian distribution with nonzero probability for x = 0 (right).

We use a Markov chain Monte Carlo (MCMC) method to sample the posterior with constraints, and we refer to our method as a Nonnegative Hierarchical Gibbs Sampler.

We applied our computational method to positron emission tomography (PET), where a radioactive tracer element is injected into a body and exhibits radioactive decay, resulting in photon emission. The emitted photons that leave the body are recorded by a photon detector, and the resulting mathematical reconstruction problem is equivalent to a classical X-ray CT problem. The above figure shows our simulation results.

• Top left: the mean of the computed image samples from our nonnegative hierarchical Gibbs sampler.
• Bottom left: histogram of the regularization parameters produced by the sampler.
• Top right: the MAP estimate using the mean of the regularization parameters.
• The pixel-wise standard deviations from the sampler. As expected, image pixels with higher intensity also have higher variance.

For details see the paper by Bardsley & Hansen in Publications below.

The Horseshoe Prior for Edge-Preserving Reconstruction

If we want to reconstruct an image with sharp edges, we can use a Cauchy or Laplace distribution of the difference between neighbor pixels as the prior (this is related to total variation regularization). Unfortunately, these priors are computationally demanding. As as a computationally attractive alternative, we can use a so-called horseshoe prior that resembles the Cauchy and Laplace priors - see the figure below where the horseshoe prior's density is guaranteed to lie in the shaded band.

The horseshoe prior originated in the statistical community, and here we introduce it in the setting of Bayesian inverse problem - see the paper by Uribe, Dong & Hansen (2022).

The main advantage of the horseshoe prior is that it imposes a conditionally Gaussian distribution on the differences, which allows more efficient computations because Gaussians can be handled with efficient least-squares methods. The difficulty is that the horseshoe prior has hyperpriors with heavy tails, and to alleviate this issue we propose an extended horseshoe prior that uses a scale-mixture representation of the heavy-tailed hyperpriors.

For the posterior, we exploit the extended horseshoe prior to compute conditional distributions for the associated parameters that can be sampled in closed form due to conjugacy. This allows the application of a Gibbs sampler.

Here we illustrate our method with a 1D deconvolution problem for two different noise levels, and we compare with posterior statistics obtained with a Laplace-difference prior proposed by Uribe, Bardsley, Dong, Hansen & Riis (2022).  Compared to the Laplace-difference prior, we see that the horseshoe prior gives a sharper posterior (dotted lines) and a lower posterior uncertainty (the shaded area shows the 95% credibility interval). The solid line shows the ground truth.

Boundary Detection in X-Ray CT Applications

When an X-ray passes through an object, its intensity is attenuated depending on the material's attenuation coefficient. The accumulated attenuation is the integral of the attenuation coefficient along the ray. We consider the case of 2D parallel-beam systems for which the measured data is called a sinogram.

Any X-ray going through the image is characterized by its slope θ with the horizontal axis. In practice, a full-angle geometry, i.e. when θ covers the entire interval [0,π), is not always possible. This could be due to obstacles in the CT measurement setup or to avoid unhealthy levels of radiation exposure. In such cases, a limited-angle geometry is needed, i.e., θ belongs to the reduced interval [0,θ_max) with θ_max ≪ π. In such a limited-angle setup, the quality of reconstructed image is compromised when conventional reconstruction methods, such as filtered back projection (FBP), are applied.

In many X-ray CT applications, the boundaries of objects contain valuable information, e.g., to differentiate between benign vs malignant tumors. A common approach in detecting boundaries is via an image segmentation post-processing step, but the quality of the estimated boundaries is compromised in a limited-angel imaging case.

We provide a novel goal-oriented, Bayesian framework for the limited-angel CT problem, in which we reconstruct the boundaries directly and quantify their uncertainty, and without the need for reconstructing the image. This approach avoids the error propagation and reduces the dimensionality of the problem from finding a 2D image to a 1D boundary of a region. In addition, our method estimates and performs UQ of the roughness/smoothness of the boundaries that carry important information for, e.g., medical imaging applications.

In the above plots, we see the performance of our method in estimating the boundaries of an object in the limited-angle CT setup with different θ_max. The arrows indicate the angles of the X-rays and thus region of the limited-angle interval [0, θ_max ). The estimated boundaries are shown in dark blue color, and the light blue region indicates the uncertainty in this estimation. The smaller the θ_max the larger the uncertainty region - which is consistent with the results from microlocal analysis. A paper by Afkham and Dong is in preparation for IEEE Trans. Image Processing.

Defect Detection in X-Ray CT of Subsea Pipes

X-ray computed tomography (CT) is used to monitor the condition of subsea oil or gas pipes in operation, in order to detect defects that might cause leaks. In this example, we use data obtained in the test facilities at FORCE Technology. Computations are done with our CUQIpy software which draws upon the Core Imaging Library (CIL) for the CT models.

We formulate a Bayesian inverse problem with built-in defect detection. The goal is to detect defects and quantify their uncertainties. We express the CT problem as y = A (x+ε) + e, where y is the measured X-ray absorption, e is data noise, and A is the linear forward model representing the physics and geometry of the measurements. We use a novel representation x + ε of the unknown image to be reconstructed, expressed as a sum of two images; x contains the pipe structure and ε contains potential defects.

In the Bayesian setting we formulate the joint posterior distribution p(x,ε | y) ∝ p(y | x,ε) p(x) p(ε), where p(x,ε | y) is the joint posterior distribution representing the solution to the CT problem, p(y | x,ε) is the likelihood that represents the data misfit, while p(x) and p(ε) are prior distributions representing any knowledge we have about the unknown images.

We impose priors that promote the structures we are looking for. For the pipe structure x, we use the structural Gaussian prior proposed by Christensen, Riis, Uribe & Jørgensen (submitted). The above figure shows a sample from this prior which promotes the known layered pipe structure. We expect small and few defects, and therefore we impose a sparsity promoting prior on the defect image ε (to be described in a forthcoming paper). We explore the conditional posterior distributions p(x | y,ε) and p(ε | y,x) using Gibbs sampling.

The figure above shows the means of the x-samples (left), the ε-samples (middle), and their sum with annotation (right). These results indicate that our methodology has successfully separated the defects from the overall pipe structure. With the defects separated from the pipe image we use simple image analysis methods to identify the defects. This analysis results in binary images that indicate if a pixel belongs to a defect. To quantify the uncertainty related to the defect detection, the image analysis is performed for each sample of the defect posterior.

Based on this, we calculate the probability of each pixel belonging to a defect; this result is shown in the figure below where red and blue indicate negative and positive values of ε, and the numbers refer to the annotation in the above figure.

This is joint work with Assoc. Prof. Marcelo Pereyra from Heriot-Watt University. The example was created by PhD student Silja L. Christensen.

How to Quantify Model Error in Inverse Problems

Using an ideal and complex mathematical model will produce a good reconstruction, but at a high computational cost. Instead, one can use a simplified approximate model that requires less computational work, but this results in a poorer reconstruction. We use the Bayesian framework to study the model error between the ideal and approximate models and find ways to reduce its influence when solving a Bayesian inverse problem.

We consider the model y = F(x,η) + e , where y is the data, x is the reconstruction, F is the exact forward operator with model parameters η, and e is the noise. Given an approximate operator f and an initial estimate η₀ of η, we obtain the model y = f(x,η₀) + δ + e   with   δ = F(x,η) - f(x,η₀) , where δ denotes the model error. The corresponding likelihood then takes the form p(y | x) = p_v|x(y - f(x,η₀) | x)     with     v = δ + e .

Since p(v | x) is difficult to characterize, we may assume that δ and x are independent. Samples of δ are then generated by pushing forward samples of x from the prior through the relation δ = F(x,η) - f(x,η0). Unfortunately, such samples of δ are inferior because they lack information related to the likelihood p. We can obtain better samples of δ by pushing forward samples obtained via the posterior - and this process can be repeated in an iterative fashion.

We develop methods based on the above ideas and apply them to reconstruction problems in magnetic resonance electrical impedance tomography (MREIT) where the solution x represents the electric conductivity inside a circular domain.

The plots below show that our method improves on results obtained by either neglecting the model error or representing it approximately. Specifically, we show the exact solution x (top left) and estimates of x obtained without the model error (top right), with the model error obtained by pushing forward samples from the prior (bottom left), and with the model error obtained by pushing forward samples from the posterior, repeated three times (bottom right).

This is joint work with Prof. Bangti Jin from Chinese University of Hong Kong. The example was created by PhD student Puyuan Mi.