Structure aware image repair as an image structure completion and shape recognition problem and its application to shadow removal /

Open in iSearch

Student Theses

2019

Liu, Bowen

Hong Kong : The Education University of Hong Kong

Thesis

This study aims to develop a robust image completion method with enhanced structure preservation ability and an in-depth understanding of the contents of images. Current patch based image completion methods transfer information from the known region into the hole. This study enhanced the transfer process with a system that can capture multiscale structures and expand the information sources with an object database and an efficient retrieval method. The technique developed in this study can be applied to other image editing tasks that required the transfer of image information. This study has three main contributions.This study started with improving the structure information transfer in the repair process. The first contribution focuses on the reconstruction of structures in the process of repairing damaged images. We designed a Dynamic Patch System (DPS) for implicit enhancement of the structure preservation ability of the patch-based image completion framework. It enables the use of adjustable patches to capture various scale structures in images. In addition, this DPS attempted to balance the computational workload in various image pyramid levels. Our approach and previous methods are applied to damaged images with complex structures. The results show that our approach can repair images with decent and connected structures with better run-time performance.The image completion is essentially a process of information transfer. Image editing tasks that can be abstracted as an information transfer process can be converted into an image completion task. The second contribution is the adoption of the image completion approach with the DPS to the task of shadow removal. We propose a new perspective on the task of shadow removal in which shadowed images are impaired in their illumination fields. Based on this perspective, the task of shadow removal can be converted to a task of image completion. We first decomposed the shadowed images into their illumination and reflectance, which are processed separately to avoid interference. The illumination is repaired with a patch-based "search and lighten" iteration to transfer light information from the lit region to the shadow region. The reflectance is optimized with our image completion approach using the DPS. The repaired illumination and optimized reflectance are then combined to generate shadow-free images. The results generated with this approach display better color consistency than those generated with previous methods and are considered more visually pleasing in a user study.Previous image completion methods assume that information transfer within the image is sufficient to repair the damaged region. But this assumption may be violated when unique structures exist, such as object contours that cannot be duplicated in many situations. The third contribution of this study is the introduction of a novel shape descriptor called Directed Chords Pattern (DCP) to repair object contours by referring to an external shape database. Our DCP shape descriptor uses the chord distribution at each sampling point to extract a shape's features and provides a similarity metric between shapes. Our shape matching method with DCP achieves an accuracy of 88:67% in benchmark testing with the MPEG-7 dataset and is competitive with previous methods while allowing a more flexible computational workload. We provide examples in which we repair horses in images. Equipped with the Weizmann horse database, our approach can generate reasonable structures for damaged horse contours according to the estimated transformation based on the DCP, which is challenging with existing completion methods.The image-repair technique presented in this study has enhanced structure-preservation ability implicitly with DPS and explicitly with DCP. The application scene of DPS is extended to the task of shadow removal. The DPS may be adapted to arbitrary patches other than square. The DCP shape descriptor proposed in this study can be further improved and applied to other tasks in the future. For example, it can be applied to the task of gesture recognition

LG51.H43 Dr 2019eb Liub

https://educoll.lib.eduhk.hk/records/VB9Up5F2