Files in this item



application/pdfSHEN-DISSERTATION-2021.pdf (18MB)
(no description provided)PDF


Title:Learning to map between domains
Author(s):Shen, Zengming
Director of Research:Huang, Thomas S.
Doctoral Committee Chair(s):Uddin, Rizwan
Doctoral Committee Member(s):Liang, Zhi-Pei; Georgescu, Bogdan; Abbaszadeh, Shiva; Fulvio, Angela Di
Department / Program:Nuclear, Plasma, & Rad Engr
Discipline:Nuclear, Plasma, Radiolgc Engr
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Machine learning
Computer vision
Image translation
Medical image processing
Image Caption
Image segmenation
Abstract:Humans consume visual content avidly. We are in the midst of an imaging revolution enabled by inexpensive digital cameras and the internet. Almost everyone's cell phone has a camera. The photos taken by these cameras are shared massively and rapidly on the internet. However, there is an asymmetry: Each individual can consume only limited visual content in his limited lifetime, such that only a chosen few are talented enough to both express and understand something unseen visually and effectively. The rest of us try to understand and express something unseen by translating them to something seen before. Similarly, in the medical image field and radiological science, tens of thousands of medical images (MRI, CT, etc) of patients are taken. These medical images need to be studied and interpreted. In this dissertation, we investigate a number of data-driven approaches for mapping from an 'unseen' or hard to understand domain to a 'seen' or easy-to-understand domain. Our work includes mapping between two image domains and mapping from an image domain to a language domain, which in computer vision are called, respectively, image-to-image translation and image captioning. The presented methods not only help users to easily and accurately synthesize useful photos, but also enable new visual and linguistic effects not possible before this work. In the clinical diagnosis, these approaches can improve the accuracy and efficiency of the diagnosis process for the experienced radiologist. What's more, the approach of mapping from image domain to text domain can mimic the work of the experienced radiologist for automatic medical report generation. Part I: This part describes image segmentation, which can be treated as a special case of image-to-image translation. This part includes two works. The first work solves the anisotropic resolution problem for 3D medical image semantic segmentation in the Appendix A. The second work describes our US patented cross-domain medical image segmentation. The first domain has labels while the second domain has no labels; by designing a special domain mapping, we enable image semantic segmentation on the second domain. Both of these works can improve computer aided medical image interpretation and help the radiologist read the medical images more efficiently and accurately. Part II: In the clinical diagnosis, in order to combine the advantages of multiple medical imaging modalities together, medical image registrations or cross-domain image translation is needed. A crucial requirement for both is one to one correspondence. Because the medical images from multiple image modalities (such as MRI, CT) are from the same patients. This part presents learning a self-inverse network to realize one-to-one mapping for both paired and unpaired image-to-image translation. Part III: In the clinical diagnosis, the final output of the diagnosis is in text domain(such as medical report, medical prescriptions etc). Since medical report writing based on medical image can be error-prone for inexperienced physicians, and time-consuming and tedious for experienced physicians, automatic generation of medical image report can make this tedious and difficult task efficient. This part expands to learn the mapping from the image domain to the language domain. Specifically, the mapping is done by learning a language representation to form the language domain.
Issue Date:2021-04-21
Rights Information:Copyright 2021 Zengming Shen
Date Available in IDEALS:2021-09-17
Date Deposited:2021-05

This item appears in the following Collection(s)

Item Statistics