Wiki source code of Inversio-ongelmat, kevät 2011

Last modified by smsiltan@helsinki_fi on 2024/03/27 10:05

Show last authors
1 == Inversio-ongelmat, kevät 2011 ==
2
3 == Inverse problems, spring 2011 ==
4
5 [[image:attach:tomo.png||border="true"]]
6
7 == Finnish summary ==
8
9 Inversio-ongelmien tutkimus on sekä puhtaan että sovelletun matematiikan aktiivinen
10 osa-alue, joka on Matematiikan ja tilastotieteen laitoksella edustettuna kolmen professorin,
11 lukuisten tutkijoiden ja jatko-opiskelijoiden sekä Suomen Akatemian huippuyksikön voimin.
12
13 Alan tutkimuksessa tavoitteena on saada tuntemattomasta kohteesta tietoa epäsuorien mittausten avulla.
14
15 Inversio-ongelmilla on sovelluksia monilla eri tieteen ja tekniikan aloilla.
16 Näitä ovat mm. signaalinkäsittely, rakenteita tuhoamaton testaus, fysiikka ja astrofysiikka,
17 muodon optimointi, lääketieteellinen kuvantaminen ja potilaan elintoimintojen seuraaminen,
18 solubiologia, geofysikaalinen kuvantaminen sekä optiohinnottelu osakemarkkinoilla.
19
20 Esimerkkejä inversio-ongelmista:
21 -Epätarkan valokuvan terävöittäminen
22 -Kolmiulotteinen röntgentomografia
23 -Maan rakenteiden kuvantaminen maanjäristysaaltojen kulkuaikojen avulla
24 -Halkeamien etsiminen rakenteista
25 -Öljyn ja mineraalien etsintä
26 -Saasteiden maanalaisen leviämisen havainnointi
27 -Asteroidien muodon selvittäminen kirkkauden muutosten avulla
28 Yhteistä kaikille näille ongelmille on niiden herkkyys
29 mittausvirheille sekä se, että tutkittavasta kohteesta saadaan
30 vain epäsuoria ja kohinaisia mittauksia.
31
32 Keväällä 2011 järjestetään laaja inversio-ongelmien kurssi, jonka
33 voi sisällyttää sekä yleisen että sovelletun matematiikan syventäviin opintoihin.
34
35 Kurssin luennot koostuvat kahdesta osasta:
36 -**Analyyttiset inversio-ongelmat** (prof. Matti Lassas)
37 -**Laskennalliset inversio-ongelmat** (prof. Samuli Siltanen).
38
39 Luentojen laajuus on 5+5=10 op.
40
41 Luentojen lisäksi kurssi sisältää aiempaa "Matematiikan sovellusprojektit"-kurssia
42 vastaavan **projektityön**. Tämä tehdään ohjattuna parityönä ja sen laajuus on 5 op.
43
44 Yhteensä kurssin laajuus on siis 15 op.
45
46 == English summary ==
47
48 Inverse problems research is an active area of both pure and applied mathematics.
49 At the Department of Mathematics and Statistics the field is represented by three professors,
50 several researchers and graduate students and a Centre of Excellence of Academy of Finland.
51
52 Inverse problems are about interpreting indirect measurements, and they have
53 applications to many areas of science and technology. Examples include signal processing,
54 non-destructive testing, astrophysics, medical imaging and geophysical prospecting.\\
55
56 The lectures of the course consist of two parts:
57 -**Analytic inverse problems** (Prof. Matti Lassas)
58 -**Computational inverse problems** (Prof. Samuli Siltanen).\\
59
60 Together the lecture parts make up 5+5=10 credit units.\\
61
62 In addition to lectures the course involves a **project work**. It is done in teams of two and
63 gives 5 credit units to each student.\\
64
65 The course is in total 15 credit units.
66
67 == Results ==
68
69 [[attach:Tulokset2011web.pdf]]
70
71 == Lecturers ==
72
73 [[Matti Lassas>>doc:mathstatHenkilokunta.Lassas, Matti]] and [[Samuli Siltanen>>doc:mathstatHenkilokunta.Siltanen, Samuli]]
74
75 == Credit units ==
76
77 15 op.
78
79 == Type ==
80
81 Advanced studies (syventävä opintojakso)
82
83 == Prerequisites ==
84
85 Recommended: basics of Fourier transform, measure theory, integration and linear algebra
86 (for example courses Fourier-muunnoksen perusteet, Mitta- ja integrointiteoria, Lineaarialgebra 1&2).
87
88 Experience with Matlab is helpful in the computational part of the course, but not necessary.
89 Tutorials for learning basics of Matlab are available for example at the following locations:
90
91 * [[MathWorks tutorials>>url:http://www.mathworks.com/academia/student_center/tutorials/launchpad.html||shape="rect"]]
92 * [[University of Utah tutorials>>url:http://www.math.utah.edu/lab/ms/matlab/matlab.html||shape="rect"]]
93 * [[Clarkson University tutorials>>url:http://www.cyclismo.org/tutorial/matlab/||shape="rect"]]
94
95 == Lectures ==
96
97 **Period III**: Lectures and exercises, weekly schedule can be found below.
98 Tuesday 10-12, hall B120,
99 Wednesday 10-12, hall B120,
100 Friday 12-14, hall D123.
101
102 **Period IV**: Project work.
103
104 == Exam ==
105
106 Friday 15.4, 12-14, hall BK107.
107
108 The exam will consists of three problems related to the analytical part of the course
109 and three problems related to the computational part of the course.
110 Students need to answer four questions of their choice.
111
112 == Material ==
113
114 Material for the **analytical part** of the course will be uploaded here. The material
115 contains:
116
117 [[Background material 1 >>url:http://rrschmidt.wordpress.com/inverse-problems-course-notes/||shape="rect"]](Gunther Uhlmann's lecture notes)
118
119 [[Background material 2 >>url:http://www.columbia.edu/~~gb2030/COURSES/E6901/LectureNotesIP.pdf||shape="rect"]](Guillaume Bal's lecture notes)
120
121 Fourier series and Fourier transform Chapters 4 and 9 of Walter Rudin's book Real and Complex analysis
122
123 [[Lecture notes 1>>attach:matin_2011_01m_25d.PDF]],
124 [[Lecture notes 2>>attach:matin_2011_03m_01d.PDF]].
125 [[Lecture notes 3>>attach:matin_2011_03m_25d.PDF]].
126
127 The **computational part** follows the upcoming book
128 //Linear and nonlinear inverse problems with practical applications//
129 by Jennifer Mueller and Samuli Siltanen. The book will be referred to below as //LNIP//.
130 Relevant parts of the book will be copied to the students.
131 If you need a copy, please come to the lecture or contact samuli.siltanen (at) helsinki.fi.
132
133 Also, you can use the old computational [[lecture notes>>attach:IPnotes14.pdf]]. Apart from notation, that document
134 contains the same concepts and definitions than //LNIP//.
135
136 == Sister course ==
137
138 Professor **Jennifer Mueller** is lecturing a course on inverse problems at Colorado State University (CSU).
139 She also follows the forthcoming book //Linear and nonlinear inverse problems with practical applications,//
140 and the two courses will share software, project topics and other material. Here is the homepage of the
141 [[CSU Inverse Problems course.>>url:http://www.math.colostate.edu/~~mueller/m676.html||shape="rect"]]
142
143 == Matlab files ==
144
145 All Matlab routines given here should be run in a working directory containing a sub-directory called "data".
146 More precisely: //before running the files below, you need to create a subfolder "data" yourself.//
147
148 Files named as "xxx_comp.m" compute results and save them to sub-directory "data" as .mat -files.
149 Files named as "xxx_plot.m" read those results from the .mat file and create a plot. This way you can
150 modify plots easily without repeating possibly time-consuming computations.
151
152 ===== **One-dimensional deconvolution example: ** =====
153
154 In the naming convention for Matlab files here, DC stands for "DeConvolution".
155
156 Basic routines: [[attach:DC_PSF.m]], [[attach:DC_PSF_plot.m]], [[attach:DC_convmtx.m]], [[attach:DC_target.m]]
157
158 Simulate continuum data: [[attach:DC1_cont_data_comp.m]],[[attach:DC1_cont_data_plot.m]]
159
160 Create discrete measurement model and test naive inversion including //inverse crime//.
161 Experiment with values n=32, n=64 and n=128 in DC_discretedata_comp.m.
162 [[attach:DC2_discretedata_comp.m]], [[attach:DC2_discretedata_plot.m]],[[attach:DC2_naive_plot.m]]
163
164 Simulate measurement data avoiding inverse crime. The data is first computed
165 using a very fine grid to approximate the convolution integral, and then sampled
166 at k points. Naive inversion is attempted again.
167 [[attach:DC3_nocrimedata_comp.m]], [[attach:DC3_nocrimedata_plot.m]],[[attach:DC3_naive_plot.m]]
168
169 Truncated SVD (singular value decomposition) is the first noise-robust inversion
170 method introduced in this course. Here it is applied to the crime-free data computed
171 in routine DC3_nocrimedata_comp.m. Try the inversion with different resolutions
172 by running first DC3_nocrimedata_comp.m with n=32, n=64, ..., n=512 and then
173 running DC4_truncSVD_comp.m.
174 [[attach:DC4_truncSVD_comp.m]]
175
176 Tikhonov regularization is a versatile and powerful inversion method. Here are files
177 for trying out classical Tikhonov regularization with identity matrix in the penalty term,
178 and also the generalized version with a difference matrix in the penalty term.
179 [[attach:DC5_Tikhonov_comp.m]]
180 [[attach:DC6_TikhonovD_comp.m]]
181
182 Total variation (TV) regularization is the method of choice when edge-preserving reconstructions
183 are called for. TV-regularized solutions can have jump discontinuities. The following routine makes
184 use of Matlab's Optimization Toolbox (in particular quadprog.m).
185 [[attach:DC7_TotalVariation_comp.m]]
186
187 ===== **X-ray tomography example (with explicit measurement matrix A):** =====
188
189 In the naming convention of Matlab files here, XRM stands for X-Ray tomography with Matrices.
190
191 **Step A.** Create measurement matrices A at different resolutions. The routine XRMA_matrix_comp.m
192 is not very effective computationally and requires Matlab's Image processing toolbox;
193 that's why there are a few sample results (.mat files) available below as well. The sample matrices
194 correspond to measuring NxN images with N evenly spaced angles with N=16,32,50,64.
195 Copy the .mat files to subdirectory "data" for use.
196 [[attach:XRMA_matrix_comp.m]][[attach:RadonMatrix16.mat]][[attach:RadonMatrix32.mat]][[attach:RadonMatrix50.mat]][[attach:RadonMatrix64.mat]]
197
198 **Step B.** Test naive tomographic reconstruction //including inverse crime//.
199 [[attach:XRMB_naive_comp.m]][[attach:XRMB_naive_plot.m]]
200
201 **Step C.** Create tomographic data avoiding inverse crime. This is done by simulating data with
202 a higher-resolution Shepp-Logan phantom and interpolating the sinogram back to the working resolution.
203 Due to the special structure of Matlab's radon.m routine, the construction of crime-free data is pretty
204 complicated. However, you do not need to understand all details of the routine for using it successfully.
205 This step requires the radon.m function from Matlab's Image processing toolbox, so for
206 convenience we include some .mat files for those of you who cannot access radon.m.
207 [[attach:XRMC_NoCrimeData_comp.m]][[attach:XRMC_NoCrimeData_plot.m]]
208 [[attach:XRMC_NoCrime16.mat]][[attach:XRMC_NoCrime32.mat]][[attach:XRMC_NoCrime50.mat]]
209
210 **Step D.** Try out naive reconstruction with data from Step C involving no inverse crime.
211 [[attach:XRMD_naive_comp.m]][[attach:XRMD_naive_plot.m]]
212
213 **Step E.** Compute singular value decomposition (SVD) of the measurement matrix A. This becomes very
214 demanding computationally when N is greater than 32.
215 [[attach:XRME_SVD_comp.m]][[attach:XRME_SVD_plot.m]]
216
217 **Step F.** Compute regularized reconstructions using truncated SVD. The routine here both computes
218 the reconstructions and plots them.
219 [[attach:XRMF_truncSVD_comp.m]]
220
221 **Step G.** Compute regularized reconstructions with Tikhonov regularization. There is no automatic choice
222 of regularization parameter here; you can modify the code and experiment with different parameter choices.
223 [[attach:XRMG_Tikhonov_comp.m]][[attach:XRMG_Tikhonov_plot.m]]
224
225 **Step H.** Compute regularized edge-preserving reconstructions using approximate total variation regularization.
226 The approximation comes from rounding the absolute value function so that the objective functional becomes
227 differentiable. We use Barzilai-Borwein gradient-based minimization.
228 The algorithm here includes non-negativity constraint by replacing all negative elements of the iterate by zero
229 in each iteration. Such box-constrained Barzilai-Borwein method has been analysed by Dai and Fletcher
230 Numer. Math. (2005) 100. We note that the choice of the initial steplength is done here in a quick-and-dirty way;
231 it would be more proper to use line search for the first step. These files run and plot the reconstruction:
232 [[attach:XRMH_aTV_comp.m]][[attach:XRMH_aTV_plot.m]]
233 These files are needed in the working directory when running XRMH_aTV_comp.m:
234 [[attach:XRMH_aTV_feval.m]][[attach:XRMH_aTV_fgrad.m]][[attach:XRMH_aTV_grad.m]][[attach:XRMH_aTV.m]][[attach:XRMH_misfit_grad.m]][[attach:XRMH_misfit.m]]
235
236 ----
237
238 == Schedule ==
239
240 **18.1.2011 (Tuesday)** **Lassas and Siltanen**
241 Introduction to inverse problems, explanation of practicalities of the course.
242 Introduction to the computational part:[[attach:Johdanto.pdf]]
243
244 **19.1.2011 (Wednesday) Siltanen**
245 Measurement model for one-dimensional convolution and deconvolution,
246 including continuum integral model and discrete model. See section 2.1 of the book //LNIP//.
247
248 **21.1.2011 (Friday) Siltanen**
249 Measurement model for X-ray imaging. Discussion of instability of naive inversion for both
250 deconvolution and tomography. See sections 2.1 and 2.3 of the book //LNIP//.
251
252 **25.1.2011 (Tuesday) Lassas**
253 Models used in analytic inverse problems. We review results on Fourier series.
254
255 **26.1.2011 (Wednesday) Lassas**
256
257 **28.1.2011 (Friday) Oksanen**
258 Exercise 1, hall D123. Please complete the following 5 problems before the exercise session.
259 Lauri Oksanen will record which problems the participants have done, and the number of
260 completed exercises will be taken into account when grading the course.
261
262 Problems:
263 -Exercises 2.4.1-2.4.4 of the book //LNIP//,
264 -Study the deconvolution example with Matlab. Find out how low the noise level must be
265 in routine DC_discretedata_comp.m to give a small error in naive reconstruction.
266 -Solve the problems in the in the attached [[file>>attach:Laskarit1.pdf]]
267
268 **1.2.2011 (Tuesday) Lassas**
269 Lecture
270
271 **2.2.2011 (Wednesday) Siltanen**
272 Ill-posedness. Singular value decomposition for matrices. Robust inversion using truncated SVD.
273
274 **4.2.2011 (Friday) Siltanen**
275 Simulation of noisy data. Inverse crime.
276
277 **8.2.2011 (Tuesday) Lassas**
278 Lecture
279
280 **9.2.2011 (Wednesday) Siltanen**
281 Tikhonov regularization.
282
283 **11.2.2011 (Friday) No instruction**
284 If you are not familiar with Matlab, now it's a good time to work through one of the tutorials above.
285 Also, you might want to familiarize yourself with the X-ray computations given above as they will be
286 used in the Matlab sessions below.
287
288 **15.2.2011 (Tuesday) Niemi**
289 Matlab exercise session. It is important to take part; presence in the session affects the course grade positively.
290
291 Location: C128.
292
293 Create tomographic data with no inverse crime at resolution 16x16 using the routine [[attach:XR3_NoCrimeData_comp.m]].
294 Try naive reconstruction with noisy and non-noisy data.
295
296 Compute reconstructions using truncated SVD (you need to write your own m file for this).
297 Plot the singular vectors and reconstructions as images. Calculate relative errors of reconstructions.
298
299 Use filtered backprojection and Tikhonov regularization to recover the 16x16 Shepp-Logan phantom using
300 [[attach:XR4_FBP_comp.m]]and [[attach:XR6_Tikhonov_comp.m]]. Compute relative errors.
301
302 Which of the above methods gives you the lowest relative error?
303
304 Repeat the above computations at resolutions 32x32 and 64x64.
305
306 **16.2.2011 (Wednesday) Niemi**
307 Exercise session, hall B120.
308
309 Problems:
310 - Exercises 1, 3 and 4 in Section 5.7 of //LNIP//,
311 - Solve the problems in the attached [[file>>attach:Laskarit2.pdf]]
312
313 **18.2.2011 (Friday) Oksanen**
314 Matlab workshop. It is important to take part; presence in the workshop affects the course grade positively.
315
316 Location: C128.
317
318 Take the file [[attach:XR6_Tikhonov_comp.m]]and make the reconstruction process matrix-free. In other words,
319 replace all occurrences of matrix A by an application of Matlab's radon.m function and all occurrences
320 of A^T (transpose of A) by Matlab's iradon.m function with filter choice 'none' (corresponding to unfiltered
321 backprojection, which is the adjoint of the Radon transform). Push the resolution as high as possible.
322
323 Can you reconstruct at 512x512 pixel resolution and 512 angles?
324
325 What is the best relative error you can achieve?
326
327 **22.2.2011 (Tuesday) Lassas**
328
329 **23.2.2011 (Wednesday) Lassas**
330
331 **25.2.2011 (Friday) Siltanen**
332
333 Idea and definition of total variation regularization. Reformulation as a constrained quadratic optimization
334 problem. See //LNIP//, Chapter 6.\\
335
336 **1.3.2011 (Tuesday) Siltanen**
337
338 Large-scale total variation regularization using the rounding trick and gradient-based optimization.
339 Barzilai-Borwein gradient descent method. See //LNIP//, Chapter 6.
340
341 **2.3.2011 (Wednesday) Lassas**
342
343 **4.3.2011 (Friday) Lassas**
344
345 **15.3.2011 (Tuesday) Lassas**
346
347 **16.3.2011 (Wednesday) Lassas** Lectures will be held at C123.
348
349 **18.3.2011 (Friday) There will be no lectures**
350
351 **22.3.2011 (Tuesday) Lassas and Siltanen**
352
353 Project work topics are chosen. Information about the project work is given. The hall is C123.
354
355 == Project work ==
356
357 The idea is to study an inverse problem both theoretically and computationally in teams of two students. The end product is a written report consisting of 10-20 pages. We recommend the classical table of contents for structuring the document:
358 1 Introduction
359 2 Materials and methods
360 3 Results
361 4 Discussion
362
363 Section 2 is for describing the data and the inversion methods used. In section 3 those methods are applied to the data and the results are reported with no interpretation; just facts and outcomes of computations are described. Section 4 is the place for discussing the results and drawing conclusions.
364
365 First goal: two first sections should be preliminary written. Student A describes the simulation or collection of data, and Student B describes the reconstruction method that will be used. At this point no numerical computations are not necessary. The draft of project work presented at the first deadline will be graded, and the grade represents 30% of the final grade of the project work.
366
367 Second and final goal: report is complete. The idea is that Student B simulates or collects the data, and Student B implements the reconstruction.
368
369 **Topics for project work:**
370
371 **~1. Stability and continuity of Radon transform.**
372 The work concentrates on understanding correct function spaces to study Radon transform and its inverse and possibly considering numerical examples. References: Bal's lecture notes, Natterer's book.
373
374 **2. Acoustic scattering from an obstacle.**
375 The work concentrates on direct and inverse scattering problems and integral equations on the boundary of an obstacle. References: 1. Kress, Linear integral operators, 2. Books of Colton and Kress.
376
377 **3. Understanding digital filters.**
378 The work concentrates on studying discrete or continuous filters.
379
380 **4. Inverse problems and Riemannian geometry.**
381 Understanding how geodesic and travel times are related or understanding geometric formulation of inverse conductivity problem. The work is related to invisibility cloaking.
382
383 **5. Inverse spectral problem or non-linear inverse problem for wave equation.**
384 The work is related to a classical question "Can you hear the shape of a drum?". In the project work one studies one-dimensional spectral problem or one dimensional wave equation and the goal is to understand some of the basic methods how the wave speed can be reconstructed using boundary measurements.
385 References: 1: Inverse boundary spectral problems (Katchalov-Kurylev-Lassas (Chapter 1), 2: An introduction to inverse scattering and inverse spectral problems (Chadan et al), Chadan's chapter, and 3: An introduction to the mathematical theory of inverse problems (Kirsch, Andreas, Applied Mathematical Sciences, 120. Springer-Verlag, New York, 1996), Chapter 4.
386 References 1-2 can be found from Kumpulan campus library and reference is provided by the lecturers. More material can be found from American Mathematical Societys web page [[Mathscinet>>url:http://www.ams.org/mathscinet/index.html||shape="rect"]] by searching with words "inverse spectral". In the work one aims to write a report which deals either on the Boundary Control method (Reference 1) or the Gelfand-Levitan method (References 2 and 3).
387
388 **6. Deblurring a misfocused photograph.**
389 Take a printed paper with some text and an image; a newspaper page will do. Make sure that the page contains a small isolated black dot. It's a good idea to take a sharp picture for comparison and a few misfocused versions with different amount of blur. Also, take photographs with different sensitivities (ISO settings in the camera), resulting in various amounts of measurement noise. Read the photograph into Matlab using the "imread" command and pick out only one of the three color components (red, green or blue). Then you have a grayscale pixel image that can be viewed as a discrete approximation of a real-valued light intensity distribution on the paper. You can read off the shape of the point spread function from the image of the isolated black dot. Apply Tikhonov regularization or (approximate) total variation regularization and find out if you can make the blurred text readable. Read the photograph into Matlab using the "imread" command and pick out only one of the three color components (red, green or blue). Then you have a grayscale pixel image that can be viewed as a discrete approximation of a real-valued light intensity distribution on the paper. You can read off the shape of the point spread function from the image of the isolated black dot. Apply Tikhonov regularization or (approximate) total variation regularization and find out if you can make the blurred text readable.
390
391 **7. Statistical inversion and Monte Carlo Markov Chain methods**
392 Statistical (Bayesian) inversion provides efficient means for incorporating //a priori// information into reconstructions. In this project work, one-dimensional deconvolution data is inverted using smoothness requirement (as in generalized Tikhonov regularization with derivative penalty) together with enforcing the information that the signal takes values between zero and one.
393
394 **8. Wavelets and Besov space regularization**
395 Implement the Besov space regularization method with parameters s=p=q=1 for the one-dimensional deconvolution problem using approximate absolute value function and Barzilai-Borwein minimization.
396
397 **9. Total variation regularization for tomography**
398 In the lecture material, total variation regularization was implemented for the one-dimensional deconvolution problem using the quadratic programming approach. The aim of this project work is to implement the same approach for the two-dimensional X-ray tomography application.