Interpolation of Scattered Visual Data Using Frequency Models

We are living in an era where digital visual content is an essential part of our daily live. Image and video data are key means of communication. Typical image processing applications yield pixels on scattered data positions that cannot be displayed on a digital screen nor can be stored efficiently. Thus, the scattered two-dimensional data has to be interpolated to regularly spaced grid points. Most applications demand for high-quality interpolation results. In this thesis, Key-Point Agnostic Frequency-Selective Mesh-to- Grid Resampling (AFSMR) is used for high-quality interpolation. AFSMR generates a frequency model that estimates the underlying visual signal. In this thesis, AFSMR is applied in three different scenarios. First, affine transforms are considered. Second, temporal upscaling of videos is conducted in Frame-Rate Up-Conversion (FRUC). Third, AFSMR is used in the preprocessing of neural networks for image classification. A distorted transmission scenario is considered. Furthermore, the images have to be resized to the desired input image size of the network. AFSMR conducts reconstruction and resizing jointly.
The representation of our world goes beyond two dimensions. Therefore, AFSMR is extended to three dimensions for augmented and virtual reality. Each data point in three dimensions has to be captured individually and is located at an arbitrary position in three-dimensional space. Thereby, three-dimensional scattered data is produced. The set of captured points in three dimensions is summarized in a point cloud. Here, the resolution of a point cloud has to be increased artificially which is referred to as point cloud upsampling. In this thesis, frequency models are applied for point cloud upsampling.