Colorization Using Optimization

Anat Levin, Dani Lischinski, Yair Weiss

Abstract

Colorization is a computer-assisted process of adding color to a monochrome image or movie. The process typically involves segmenting images into regions and tracking these regions across image sequences. Neither of these tasks can be performed reliably in practice; consequently, colorization requires considerable user intervention and remains a tedious, time-consuming, and expensive task.

In this paper we present a simple colorization method that requires neither precise image segmentation, nor accurate region tracking. Our method is based on a simple premise: neighboring pixels in space-time that have similar intensities should have similar colors. We formalize this premise using a quadratic cost function and obtain an optimization problem that can be solved efficiently using standard techniques. In our approach an artist only needs to annotate the image with a few color scribbles, and the indicated colors are automatically propagated in both space and time to produce a fully colorized image or sequence. We demonstrate that high quality colorizations of stills and movie clips may be obtained from a relatively modest amount of user input.

Still image colorization examples
Progressive colorization example
Recoloring examples
Video clip colorization examples
Note: The camera ready version of the paper can be downloaded here.
Matlab code of our algorithm can be downloaded here .

Marked B/W image	Result
Marked B/W image	Result
Marked B/W image	Result
Marked B/W image	Result
Marked B/W image	Result
Marked B/W image	Result

a1 - Initial set of scribbles	b1 - Scribbles have been added on the table cloth and the wall.	c1 - A few red pixels have been marked on the beads.
a2 - Initial result. Note that the table cloth gets the pink color of the girl's dress. Also, there is a cyan color spill from the pacifier on the wall behind.	b2 - Second result. Problems visible in a2 have been fixed.	c2 - Final result. We now have a string of red beads.

Input image	Marked image	Result
Input image	Marked image	Result
Input image	Marked image	Result
Input image	Marked image	Result
Input image	Marked image	Result

Clip 1: Crater Lake. 83 frames, 7 marked frames Click here for a QuickTime MPEG-4 movie.	Input grayscale clip
Marked frames	Colorized result

Clip 2: Toddler. 43 frames, 5 marked frames Click here for a QuickTime MPEG-4 movie.	Input grayscale clip
Marked frames	Colorized result

Clip 3: Birthday. 62 frames, 10 marked frames Click here for a QuickTime MPEG-4 movie.	Input grayscale clip
Marked frames	Colorized clip

Clip 4: Driving. 33 frames, 9 marked frames Click here for a QuickTime MPEG-4 movie.	Input grayscale clip
Marked frames	Colorized clip

Colorization Using Optimization

Still Image Colorization Examples:

Progressive colorization:

Recoloring Examples:

Video Clip Colorization Examples: