Twibright Registrator
Shows previously invisible things
A hazy picture was processed to improve contrast. In normal picture, the crane cables, background forest , exact shape of two square buildings and dark lines on the left foreground brown building cannot be recognized. In the Registrator output they can. See also Example on Wikimedia Commons
Quadruples the megapixels of the camera
Resolution quadrupling is not perfect because frequencies approaching the maximum one (pixel values 1,0,1,0,1,0,1,...) are lost in the camera. It is also 4 times slower than registration - works on upsampled images.
Removes JPEG artifacts
Removes noise from dark details
Digital effects don't degrade anymore
Misty Zurich was histogram equalized, gamma adjusted and filtered with Retinex. The normal image shows noisy meadows and red artifact over church dial. Twibright Registrator produces perfectly smooth meadows.
Hides people
Removes stripes on CRT's
Removes rain
Long exposure in daylight
Couple minutes of simulated exposure.
Download and install
Tested on Linux, should work on Mac, BSD. For Windows you need Cygwin.
Make sure
djpeg
from libjpeg is installed by running djpeg --help. If not,
Make sure libpng is installed in a way that programs can compile against it. Usual names on Linux distributions with packages: libpng-dev, libpng-devel, png-dev, png-devel
Download registrator.tgz
Unpack: tar xvf registrator.tgz
Enter the source: cd registrator
make
su and type in the root password
make install
How to use Registrator
Shoot images
Set ISO for noise in the image. Stronger than the JPEG artifacts, but not comparable with the image itself.
If not using tripod and delayed/remote trigger, the exposure time should be fast, but not faster that the camera limit. Minimize motion blur from camera vibration. Otherwise, the photo gets defocused.
The more shots the smoother result. Try to not shift the camera between images too much. Otherwise, if the lens warps, blur might occur. The signal to noise ratio grows with square root of image count.
Register images for lower noise
Copy JPEGs from the camera or convert RAW format into PPMs. Run registrator with filenames as parameters. Example:
registrator p100????.jpg p101????.ppm
Registrator will print a lot of matching progress info. When it ends, you have file registrator_output.png. It is a 16-bit depth PNG containing the registered output. It also makes partial results - for example - registrator_intermediate_7.png - which contains the first 7 images registered together. This is useful if the registration derails.
Quadruple resolution
This makes sense only if the camera lens is sharper than the resolution of the sensor. It depends on iris setting and tripod. It takes 4 times longer than registering.
Convert camera images to twice the size (scaling 200%) with nearest neighbor - without interpolation. example:
convert in.jpg -scale 200% -interpolate nearest-neighbor out.ppm
Run registrator on the ppm's like when registering. This is going to take 4 times longer than when registering!
Convert registrator_output.png into ppm:
convert registrator_output.png registrator_output.ppm
Deconvolve:
deconvolve 3 < registrator_output.ppm
Examine the result:
gimp registrator_deconvolved.png
If the result is too blurred, decrease the deconvolve parameter towards 2. If it's too fringy, increase the number. Repeat the deconvolution until you are satisfied with the image.
Dehaze
convert registrator_output.png registrator_output.ppm
If you don't have ImageMagick, you can use GIMP.
Load registrator_output.png into image editor and crop part of the image you want to dehaze. Save as part.ppm.
Let black and white levels be automatically determined:
deconvolve 2 < part.ppm
This prints 6 numbers example: 18 2 3 223 227 245
deconvolve 18 2 3 223 227 245 < registrator_output.ppm
Floating point numbers are accepted. No hexadecimal.
Examine the result:
gimp registrator_deconvolved.png
Register images for 3D or video stabilization
Copy JPEGs from the camera or convert RAW format into PPMs. Run registrator with OVERWRITE=1 and filenames as parameters. Example:
OVERWRITE=1 registrator p100????.jpg p101????.ppm
Registrator will print a lot of matching progress info. Let's assume you have 100 files on input. Then Registrator produces registrator_intermediate_1.png to registrator_intermediate_99.png and registrator_output.png which is the last frame.
How does it work?
How registration works
Twibright Registrator contains a high precision high performance bilinear interpolation engine working with 32-bit float pixel depth. Using this engine, it tries to find shift, rotation, zoom and exposure change so that the next picture matches the previous one with as little average pixel error as possible. It adds these pictures together and applies an adaptive deconvolution to cancel the blurring caused by stacking many pictures photographed with random fractional pixel offsets. If the count of photos were large, this deconvolution exactly cancels the blur. The work is done in photometrically linear space to ensure physical equivalency - the procedure is equivalent to a shakeless camera which exposes for very long time, gathering the photons, and then counting them.
How resolution quadrupling works
Let'sa imagine the real picture consists of pixels 2 times finer than the camera has. In 1D situation, The camera always takes 2 pixels together (if it's precisely aligned) or 1 pixel and parts of 2 other pixels. Now we assume the pixel offsets are uniformly distributed. Let's take a sample image 0,0,0,1,0,0,0 and calculate resulting images when many such images (upsampled by nearest neighbor after being photograped) are registered. In case the examined pixel falls within the big pixel, it produces the core 1,2,1. In the other case, it produces core 2,5,6,5,2. If the 1st core is scaled to the same sum 7,14,7 and the cores added, we get 2,12,20,12,2, or 1,6,10,6,1. This is a convolution of two cores: 1,2,1 and 1,4,1. The 1,4,1 is actually caused by the registration and the 1,2,1 by smearing always two pixels together. 1,4,1 is already removed by the registrator. So we have to remove 1,2,1. In comparison to 1,4,1 it's impossible to remove completely since some frequencies (the highest) is zeroed out so we would have to divide by zero. So we take a near core like 1,2.5,1 or 1,3,1 and remove that one. The particular middle number of the core has to be tried out manually to get the best tradeoff between fringes and blurriness.