FlatNet: Towards photorealistic scenes
Sreedhu S S
credits: Varun Sundar
As a solution for realizing ultra-miniature cameras by give up bulky lens traditional cameras, lens less imaging is used commonly. Lens less camera depends on computational algorithms to recover the scenes from multiplexed measurements without a focusing lens. Present iterative optimisation based reconstruction algorithms produce nosier and poor images. FlatNet, sit backs a framework for reconstructing high-quality photorealistic images from Mask based lens less cameras, where the forward model formulation of the camera is known.
FlatNet consists of an inversion stage and a perceptual enhancement stage. Inversion stage maps the measurement into space with of intermediate reconstruction by learning parameters within the forward model formulation. Perceptual stage improves perceptual quality of the intermediate reconstruction. These are practiced in an end-to-end manner.
FlatCam and PhlatCam are to different types of lens less prototypes used to show high quality reconstructions by performing extensive experiments on real and challenging scenes.
Point Spread Function (PSF) can be obtained by the calibration of lens less cameras. Can be a time consuming process and has to be done for each individual camera. Severe degradation in the performance of reconstruction algorithm happens even a small error occurred in calibration.
$ads={1}
FlatNet occupies a trainable inversion layer, which doesn't wants careful calibration of the PSF. Which is provided by a initialisation scheme of the trainable layer for both separable and inseparable cases.
Non separable lens less cameras have sensors with finite size. They can be approximated by a convolutional model, and as a result, the sensor measurement is the weighed sum of various shifted PSFs. For a large measurement can often surpassed the size of sensor, leading to lost information.
FlatNet is based on convolutional model which can be extended to strongly handle smaller sensors with a simple padding scheme. padding scheme greatly improves the intermediate reconstruction. FlatNet recover scenes on smaller sensors without any significant performance deterioration by trainable inversion further reduces visual artefacts.
When FlatNet is trained using monitor-capture scheme, which allows to inexpensively gather large dataset. The motive is to recover scenes from real measurements captured in wild.