CUQI is a research project at DTU Compute.
It is financed by Villum Fonden (the Villum Foundation)
and headed by Professor Per Christian Hansen.
What is CUQI
CUQI is a research project where we develop a mathematical, statistical and computational framework for applying uncertainty quantification to inverse problems. We also develop a software package for UQ modeling and computations, see below.
Uncertainty Quantification, or simply UQ, characterizes the sensitivity of a solution taking into account errors and inaccuracies in the data, models, etc.
Inverse Problems determine hidden information from measurements in, e.g., deconvolution, image deblurring, tomographic imaging, source reconstruction, and fauld inspection.
As an example, engineers who use X-ray imaging to inspect an object for defects can characterize the reliability of every details in the image - thus aiding the interpretation and decision whether a crack is actually present.
Villum Fonden (the Villum Foundation) is one of Denmark's largest research funding foundations with annual grants of approx. DKK 500 million. The aim is to foster research at the highest international level platformed at Danish universities and research institutions.
The Villum InvestIgator
The project is headed by Prof. Per Christian Hansen who specializes in numerical analysis, matrix computations, iterative methods, and computational methods for inverse problems.
His recent research projects were:
> High-Definition Tomography (HD-Tomo) funded by an ERC Advanced Research grant
> Improved Impedance Tomography with Hybrid Data, together with Prof. Kim Knudsen, funded by the Danish Council for Independent Research.
Per Christian Hansen is the (co-)author of five books, 100+ papers and seven software packages. His research focuses on computational methods, e.g., for regularization methods, imaging, tomography, and subspace methods for signal processing.
All his activities comprise a combination of theoretical insights and computational methods.
The Goal of the CUQI Project
Our research allows end-users of inverse problems, in science and engineering, to quantify the accuracy in their solutions, and in this way lower the risks and make more correct decisions.
We reach this goal by building general theoretical and computational foundations to quantify how model uncertainties, measurement noise, and other errors influence the solutions to inverse problems.
To do this we build on the solid mathematical and statistical foundation of uncertainty quantification. We also build a software package CUQIpy that allows experts and non-experts to apply UQ to their inverse problems, see below.
Based on our work, our aim is that UQ will become a natural part of solving inverse problems, in science and engineering.
In inverse problems we operate with a very large number of unknowns (e.g., pixels, voxels) and the computational burden can be overwhelming. Also, UQ for inverse problems raises new fundamental scientific questions that we must identify, formulate and answer properly. Hence, we need research in both theory and computational methods.
> The theoretical outcome is a mathematical and statistical basis for UQ studies of a range of inverse problems with different priors and noise models.
> The practical outcome is a modeling and computational platform – including an abstraction layer aimed at non-experts.
> The overall outcome is a framework to rigorously quantify the uncertainty of the solution’s details and identify the dominating types of errors.
To enable end-users to make use of our research, we release free open-source software that implements the theoretical and computational foundations developed during the project.
This is all released as part of the Python software package CUQIpy, which implements a computational abstraction layer for UQ studies of inverse problems aimed at non-experts.
In addition to the main software we also host individual software from papers and projects under the CUQI GitHub organization, including a growing collection of plugins for CUQIpy.
Examples of Research Activities
Here we illustrate our work on computational UQ for inverse problems with a few examples from ongoing research projects in CUQI. These examples illustrate various methodologies for implementing, applying and using UQ.
Bayesian Inference with Constraints Through Projection and Randomized Linear Least Squares Problems
In many applications, the signal that is being reconstructed must satisfy some constraints, e.g., material densities being positive, pixels of an image lying between 0 and 1, or the energy of a signal being bounded. Therefore, adding such prior information to a Bayesian model improves the meaningfulness of the resulting posterior distribution.
A common method to model constraints in a prior distribution is truncation, i.e., choosing an unconstrained prior distribution and remove all probability outside of the constraint set. Another method is to reparametrize the signal, e.g., if the signal x is assumed to be positive, then it can be written as x = exp(z) and one can choose an unconstrained prior distribution for z. These two methods focus on the interior of the constraint set, but very often, signals of interest lie on the boundary of the constraint set.
As natural alternative, one can first sample from an unconstrained posterior distribution and then apply a suitable projection onto a convex constraint set C. The resulting implicit posterior will consist of the original density in the interior of the constraint set, and a “lower dimensional” distribution on the boundary, see the illustration below.
If the forward model is linear and both the likelihood and prior are Gaussian distributions, then the unconstrained sampling and projection can be done in a single step by repeatedly solving randomized constrained linear least square problems of the form
minx∈C || A x - bs ||22 + γ || x - cs ||22 ,
where bs and cs are samples from suitable random distributions. The solutions of these optimization problems will be samples of a projected posterior.
Consider the problem of deblurring a noisy signal with prior knowledge that the signal lies in the interval [0,1]. The figure above shows the pointwise median and 99% credibility interval for such a problem using no constraints (left), only the lower bound (middle) and both upper and lower bounds (right). The projection framework significantly improves the quality of the median near the boundary.
This example was created by PhD student Jasper Everink.
UQ with Nonnegativity Constraints
Nonnegativity constraints appear in many applications where the underlying physics dictates that the solution cannot be negative. This is the case, e.g., for absorption coefficients in computed tomography, image intensities in astronomical imaging, and wave velocities in seismic travel-time tomography. Hence, nonnegativity constraints are very important for producing meaningful reconstructions.
We considers computational aspects of UQ with nonnegativity constraints x ≥ 0. If we replace the nonnegativity constraint with a positivity constraint, x > 0, then we can handle this constraint within the "standard" UQ framework with a transformation of variables x = exp(z). In this work we specifically address situations where we expect the existence of zero elements in the solution – in fact, in some imaging problems a substantial fraction of the solution's elements/pixels may be zero. Therefore we use a nonnegative Gaussian distribution.
The above figure shows a simple Gauss distribution (left), a truncated Gauss distribution with zero probability for x = 0 (middle), and a nonnegative Gaussian distribution with nonzero probability for x = 0 (right).
We use a Markov chain Monte Carlo (MCMC) method to sample the posterior with constraints, and we refer to our method as a Nonnegative Hierarchical Gibbs Sampler.
We applied our coputational method to positron emission tomography (PET), where a radioactive tracer element is injected into a body and exhibits radioactive decay, resulting in photon emission. The emitted photons that leave the body are recorded by a photon detector, and the resulting mathematical reconstruction problem is equivalent to a classical X-ray CT problem. The above figure shows our simulation results.
- Top left: the mean of the computed image samples from our nonnegative hierarchical Gibbs sampler.
- Bottom left: histogram of the regularization parameters produced by the sampler.
- Top right: the MAP estimate using the mean of the regularization parameters.
- The pixel-wise standard deviations from the sampler. As expected, image pixels with higher intensity also have higher variance.
For details see the paper by Bardsley & Hansen in Publications below.
Handling Uncertain Projection Angles in X-Ray CT
We consider an X-ray CT problem where the projection angles are not known precisely - but we have a statistical model for them. Our goal is to compute the unknown attenuation coefficient image, represented by the vector x, from measured and noisy data represented by the vector b. This is done according to the model
b = R(θ) x + e , θ ∼ πangles(⋅) , e ∼ πnoise(⋅) ,
where the matrix R(θ) represents the discretized Radon transform for the projection angles θ, and the uncertainty in the angles and the data are characterized by the probability distributions πangles and πnoise, respectively. We fix the forward model with the nominal angles θo of . We then obtain the model
b = R(θo) x + d + e , e ∼ πnoise(⋅) ,
where the model discrepancy term d acquires its "push-forward distribution" from
d = d(θ,x) = R(θ) x − R(θo) x .
Inspired by work by Kaipio & Somersalo we marginalize both d and e, to arrive at the following likelihood for b:
π(b|x) = πν|x(b − R(θo) x | x) , ν = d + e .
We now assume that both d|x and e follow Gaussian distributions, and hence πν|x = 𝒩(μ,C). Then the likelihood has a closed-form expression, and taking the negative logarithm we obtain
− log π(b|x) ∝ || L (b − R(θo) x − μ) ||22 ,
where LTL = C−1 is the Cholesky factorization of the inverse covariance matrix. Adding Total Variation (TV) regularization and a nonnegativity constraint we arrive at the variational reconstruction model
minx || L (b − R(θo) x − μ) ||22 + λTV(x) subject to x ≥ 0.
We developed a new computational method that alternates between updating the solution x and the pair (μ,C); the latter is achieved by computing samples of d|x.
The above figure compares reconstructions for two choices of the regularization parameter λ. From left to right:
- TV denotes a standard TV method that ignores the uncertainty in the projection angles and uses the nominalangles θo.
- MV-TV denotes our new method which produces improved reconstructions with less noise.
- TV-opt denotes TV reconstructions based on the true - but unknown - projection angles; this defines the unattainable optimal reconstruction.
For details see the paper by Riis, Dong & Hansen in Publications below.
All members of the CUQI team are associated with the Section for Scientific Computing at DTU Compute.
Permanent Team Members
Specialist in numerical analysis, iterative solvers, and computational methods for inverse problems.
Specialist in coupled-physics modeling, PDE-constrained optimization, functional analysis and electrical impedance tomography.
Project: goal-oriented uncertainty quantification.
Project: computational UQ for PDE problems.
Project: efficient and flexible computational methods.
Project: non-Gaussian priors.
, Department of Mathematical Sciences, University of Montana - specialist in computational methods for inverse problems and uncertainty quantification.
, Department of Mathematics, Chinese University of Hong Kong - specialist in theory and algorithms for inverse and ill-posed problems.
, Department of Applied Physics, University of Eastern Finland - specialist in computational methods for inverse problems and uncertainty quantification in imaging applications
, Laboratoire Jean Kuntzman,
Courses, Workshops, Training and Other Activities
Workshop: Imaging with Uncertainty Quantification (IUQ22)
September 27-29, 2022
This workshop brought together specialists in UQ for imaging, and the talks covered various aspects related to the development of theory, methodology and software, as well as applications of UQ in imaging. The goal was to stimulate networking and collaboration between researchers and students in these areas.
Before the workshop, we arranged a 1-day short course devoted to the Python software CUQIpy that we are currently developing for modeling and computations related to UQ for imaging.
For more details about the workshop, go to the IUQ22 Workshop homepage.
Felipe Uribe - Bayesian Inverse Problems
Postdoc Felipe Uribe is invited to talk about Bayesian inverse problems at the "Summer School on Recent Advancements in Computational and Learning Methods for Inverse Problems" (CLIP22), July 11–15: https://bugs.unica.it/cana/clip22/
Jakob Sauer Jørgensen - CUQIpy
Senior researcher Jakob Sauer Jørgensen is invited to talk about the software package CUQIpy at the "CIMPA Summer School 2022 Mathematical Methods in Data Analysis", July 18–29: https://sites.google.com/view/mathschoolinalbania/
PhD course: Introduction to Uncertainty Quantification for Inverse Problems
This course introduces state-of-the-art numerical methods for quantification and reduction of uncertainties in computational models. UQ is paramount to enhance analysis and prediction tasks in multiple applications such as tomography, material science, spatial statistics, reliability, etc. Therefore, the course can be of interest to students from any discipline in applied mathematics and engineering. The course provides the mathematical background for theory and methods of UQ, which are illustrated via Python exercises. Examples covered in the course include elemental models of deconvolution, diffusion and structural engineering.
Time: January 3–22, 2022 (3 week period at DTU). Next time: January 2024.
Course responsible: Associate Professor Yiqiu Dong, DTU Compute.
More details: Link to DTU's course description, course no. 02975.
PhD course: Bayesian Scientific Computing
The lectures focus on basic techniques in Bayesian methods, including probability distributions, Bayes' formula, conditioning, hierarchical models, estimation problems arising in this context, as well as certain numerical techniques for inverse problems, including regularization and iterative methods for solving large systems. The course is based on the book: D. Calvetti and E. Somersalo, Introduction to Bayesian Scientific Computing, Springer, 2007, as well as a new edition of it.
Time: December 9-13, 2019 (one full week).
Teachers: Professor Daniela Calvetti and Professor Erkki Somersalo, both from Case Western Reserve University, Cleveland, Ohio.
Course responsible: Professor Per Christian Hansen, DTU.
Throughout the rest of the project we will announce postdoc positions and other vacancies associated with CUQI here.
We expect to announce two postdoc positions in the fall of 2022.
We do not have funding for any more PhD positions.
Our summer internships allow master students a unique chance to build on their acquired skills, and to obtain experience with research in computational methods and applications of uncertain quantification to inverse problems in science and engineering.
We sponsors travel to Denmark, travel insurance, and reasonable accommodation expenses near DTU.
The summer internships in 2023 will be announced here primo 2023.
Here we list submitted and published papers produced in the CUQI project.
- B. M. Afkham, J. Chung, and M. Chung, Learning regularization parameters of inverse problems via deep neural networks, Inverse Problems, 37 (2021), 105017, doi 10.1088/1361-6420/ac245d.
- B. M. Afkham, Y. Dong, and P. C. Hansen, Uncertainty quantification of inclusion boundaries in the context of X-ray tomography, SIAM/ASA J. Uncertainty Quantification, to appear; arxiv.org/abs/2107.06607.
- A. M. A. Alghamdi, P. Chen, and M. Karamehmedović, Optimal design of photonic nanojets under uncertainty, submitted; arxiv.org/abs/2209.02454.
- E. Ametova, G. Burca, S. Chilingaryan, G. Fardell, J. S. Jørgensen, E. Papoutsellis, E. Pasca, R. Warr, M. Turner, W. R. B. Lionheart, and P. J. Withers, Crystalline phase discriminating neutron tomography using advanced reconstruction methods, J. Phys. D: Appl. Phys., 54 (2021), 325502, doi 10.1088/1361-6463/ac02f9 (open access).
- E. Ametova, G. Burca, G. Fardell, J. S. Jørgensen, E. Papoutsellis, E. Pasca, R. Warr, M. Turner, W. R. B. Lionheart, and P. J. Withers, Joint reconstruction with a correlative regularisation technique for multi-channel neutron tomography, Proc.16th International Meeting on Fully 3D Image Reconstruction in Radiology and Nuclear Medicine, pp. 431–434, 2021, no doi; arxiv.org/abs/2110.04143.
- K. O. Bangsgaard and M. S. Andersen, A statistical reconstruction model for absorption CT with source uncertainty, Inverse Problems, 37 (2021), 085009, doi 10.1088/1361-6420/ac11c7.
- J. M. Bardsley and P. C. Hansen, MCMC algorithms for computational UQ of nonnegativity constrained linear inverse problems, SIAM J. Sci. Comput., 41 (2020), pp. A1269–1288, doi 10.1137/18M1234588.
- D.Caviedes-Nozal, F. M. Heuchel, J. Brunskog, N. A. B. Riis, and E. Fernandez-Grande, A Bayesian spherical harmonics source radiation model for sound field control, J. Acoust. Soc. Am., 146 (2019), pp. 3425–3435, doi 10.1121/1.5133384.
- D. Caviedes-Nozal, N. A. B. Riis, F. M. Heuchel, J. Brunskog, P. Gerstoft, and E. Fernandez-Grande, Acoustic Gaussian processes, J. Acoust. Soc. Am., 149 (2021), pp. 1107–1119, doi 10.1121/10.0003497.
- S. L. Christensen, N. A. B. Riis, F. Uribe, and J. S. Jørgensen, Structural Gaussian priors for Baysian CT reconstruction of subsea pipes, submitted to Appl. Math. in Sci. and Eng.; arxiv.org/abs/2203.01030.
- Y. Dong and M. Pragliola, Including sparsity via horseshoe prior in imaging problems (2022), submitted.
- Y. Dong, C. Wu, and S. Yan, A fast method for simultaneous reconstruction and segmentation in X-ray CT application, Inverse Problems in Science and Engineering, 29 (2021), 3342–3359, doi 10.1080.17415977.2021.1999941.
- J. M. Everink, Y. Dong, and M. S. Andersen, Bayesian inference with projected densities, submitted to SIAM/ASA J. UQ; arxiv.org/abs/2209.12481.
- P. C. Hansen, K. Hayami, and K. Morikuni, GMRES methods for tomographic reconstruction with an unmatched back projector, J. Comp. Appl. Math.,413 (2022), 114352, doi 10.1016/j.cam.2022.114352 (open access).
- P. C. Hansen, J. S. Jørgensen, and P. W. Rasmussen, Stopping rules for algebraic iterative reconstruction methods in computed tomography; in 21st International Conference on Computational Science and Its Applications (ICCSA), IEEE, 2021, pp. 60-70, doi 10.1109/ICCSA54496.2021.00019.
- B. C. S. Jensen and K. Knudsen, Sound speed uncertainty in acousto-electric tomography, Inverse Problems, 37 (2021), Article ID: 125011, doi 10.1088/1361-6420/ac37f8.
- J. S. Jørgensen, E. Ametova, G. Burca, G. Fardell, E. Papoutsellis, E. Pasca, K. Thielemans, M. Turner, R. Warr, W. R. B. Lionheart, and P. J. Withers, Core Imaging Library - Part I: a versatile Python framework for tomographic imaging, Phil. Trans. Royal Soc. A, 379 (2021), Article ID: 20200193, 10.1098/rsta.2020.0192 (open access).
- M. Karamehmedović and K. Linder-Steinlein, Spectral properties of radiation for the Helmholtz equation with random coefficients, submitted to SIAM J. Appl. Math
- M. Karamehmedović, K. Scheel, F. L.-S. Pedersen, and P.-E. Hansen, Imaging with a steerable photonic nanojet probe, submitted to Proceedings of SPIE.
- M. Karamehmedović, K. Scheel, F. L.-S. Pedersen, A. Villegas, and P.-E. Hansen, Steerable photonic jet for super-resolution microscopy, submitted.
- M. Karamehmedović and D. Winterrose, On the transfer of information in multiplier equations, submitted, arxiv.org/abs/1912.10760v2.
- A. Kirkeby, M. T. R. Henriksen, and M. Karamehmedović, Stability of the inverse source problem for the Helmholtz equation in R3, Inverse Problems, 36 (2020), 055007, doi 10.1088/1361-6420/ab762d.
- K. Knudsen and A. K. Rasmussen, Direct regularized reconstruction for the three-dimensional Calderón problem, Inverse Problems and Imaging, 16 (2022), pp. 871–894, doi 10.3934/ipi.2022002.
- E. Papoutsellis, E. Ametova, C. Delplancke, G. Fardell, J. S. Jørgensen, E. Pasca, M. Turner, R. Warr, W. R. B. Lionheart and P. J. Withers, Core Imaging Library - Part II: Multi-channel reconstruction for dynamic and spectral tomography, Phil. Trans. Royal Soc. A, 379 (2021), Article ID: 20200193. doi 10.1098/rsta.2020.0193 (open access).
- N. A. B. Riis and Y. Dong, A new iterative method for CT reconstruction with uncertain view angles. In: J. Lellmann, M. Burger, and J. Modersitzki (eds), Scale Space and Variational Methods in Computer Vision. SSVM 2019. Lecture Notes in Computer Science, vol 11603, pp. 156–167, 2019. Springer, doi 10.1007/978-3-030-22368-7_13.
- N. A. B. Riis, Y. Dong, and P. C. Hansen, Computed tomography reconstruction with uncertain view angles by iteratively updated model discrepancy, J. Math. Imag., 63 (2021), pp. 133–143, doi 10.1007/s10851-020-00972-7. Also available: view-only version.
- N. A. B. Riis, Y. Dong, and P. C. Hansen, Computed tomography with view angle estimation using uncertainty quantification, Inverse Problems, 37 (2021), Articld ID 065007, doi 10.1088/1361-6420/abf5ba.
- K. Šehić, H. Bredmose, J. D. Sørensen, and M. Karamehmedović, Active-subspace analysis of exceedance probability for shallow-water waves, J. Eng. Math., 126 (2021), article no. 1, doi 10.1007/s10665-020-10080-5.
- K. Šehić, H. Bredmose, J. D. Sørensen, and M. Karamehmedović, Low-dimensional offshore wave input for extreme event quantification, J. Eng. Math., 126 (2021), article no. 13, doi 10.1007/s10665-021-10091-w.
- K. Šehić and M. Karamehmedović, Estimation of failure probabilities via local subset approximations, submitted, arxiv.org/abs/2003.05994.
- E. Y. Sidky, P. C. Hansen, J. S. Jørgensen, and X. Pan, Iterative image reconstruction for CT with unmatched projection matrices using the generalized minimal residual algorithm; submitted to 7th International Conference on Image Formation in X-Ray Computed Tomography, June 12-16, 2022, Baltimore, USA; arxiv.org/abs/2201.07408.
- F. Uribe, J. M. Bardsley,Y. Dong, P. C. Hansen, and N. A. B. Riis, A hybrid Gibbs sampler for edge-preserving tomographic reconstruction with uncertain angles, SIAM/ASA J. Uncertain. Quantif., 10 (2022), pp. 1293–1320, doi 10.1137/21M1412268.
- F. Uribe, Y. Dong, and P. C. Hansen, Horseshoe priors for edge-preserving linear Bayesian inversion, submitted; arXiv.org/abs/2207.09147.
- S. Wang, M. Karamehmedović, and F. Triki,Localization of moving sources: uniqueness, stability, and Bayesian inference, submitted to SIAM J. Appl. Math., arxiv.org/abs/2204.04465.
- R. Warr, E. Ametova, R. J. Cernik, G. Fardell, S. Handschuh, J. S. Jørgensen, E. Papoutsellis, E. Pasca, and P. J. Withers, Enhanced hyperspectral tomography for bioimaging by spatiospectral reconstruction, Sci. Rep. 11 (2021), 20818, doi 10.1038/s41598-021-00146-4.
- S. Yan and Y. Dong, GMM based simultaneous reconstruction and segmentation in X-ray CT application; in A. Elmoataz, J. Fadil, Y. Quéau, J. Rabin, and L. Simon (Eds), Scale Space and Variational Methods in Computer Vision. SSVM 2021, Lecture Notes in Computer Science, vol 12679, pp. 503–515, 2019, Springer, doi 10.1007/978-3-030-75549-2_40.