Sampling image colors with Canvas
Create your own eyedropper effect to extract colors from a drawing or photograph using HTML5 canvas methods.
Introduction
Sampling colors from an image can add interest and flexibility to websites such as a fashion site or home decorating site. Users can pick colors from sample images to change the look of a pair of jeans, a car, or a house. The getImageData and putImageData methods in the canvas API make sampling pixels or a whole photograph relatively easy. Several other CanvasAPIs are used, drawImage to put the photo onto a canvas, the CanvasImageData object, and data property.
Here we talk about how to use getImageData to sample colors on a photo to make a color coordinated picture frame. The frame is created by using the border property with Cascading Style Sheets (CSS), so it's not an integral part of the canvas or image. However, the syntax is displayed so that you can copy into your own webpage code. For additional reference, the color sample is also displayed in the individual Red, Green, Blue, and Alpha (transparency) values, the CSS color value, and the Hue, Saturation, and Value (HSV) value.
The image data acquired using getImageData is also manipulated to convert a color photo to a black and white image. The putImageData method is used to put the manipulated data back onto the canvas.
The example's HTML code sets up the canvas and input elements to get and select the URLs of photos. Four images are hard coded into the webpage, but the input field lets you paste in any other image. When the app needs an simage to use, it gets the URL from the input field when the drop-down box (select element) changes or when the Load button is clicked. The full sample is available on the MSDN sample site.
.png)
Getting the photo onto a canvas
The first step in sampling the colors is to get the photo onto a canvas element. The canvas element by itself has no visual presence on the page. You need to add content for it to be useful, in this case an image.
The canvas object only has a few methods and properties. Most of the drawing and manipulation of images on a canvas is accomplished using the CanvasRenderingContext2D object. The context object is returned by the getContext method of the canvas object with statement var ctx = canvas.getContext("2d");. Currently only a 2D context is supported in Windows Internet Explorer 9 and Internet Explorer 10, but 3D contexts such as the WebGL are supported in some other browsers.
Unlike an img tag, the canvas element doesn't have a src property to assign a URL. In this example, an img element is created and the image is assigned to that. The image is later transferred to the canvas using the drawImage property on the context object.
This next example creates global variables for the canvas element ("canvas") and the Context object ("ctx"). An image object is created ("image") that holds the photos as they are loaded.
var canvas = document.getElementById("myCanvas");
var ctx = canvas.getContext("2d");
var image = document.createElement("img");
In the sample, the getImage() function sets the src of the image object to the value of the input field. The image takes some time to load, depending on the size of the file, so the webpage needs to wait for it to finish loading. To do that, the onreadystatechange event handler is used to monitor the image. Within the handler the readyState property is used to watch for a complete state value.
function getImage() {
var imgSrc = document.getElementById("fileField").value;
if (imgSrc == "") {
imgSrc = rootPath + samplefiles[0]; // Get first example
document.getElementById("fileField").value = imgSrc;
}
image.src = imgSrc;
//image.complete
image.addEventListener("load", function () {
var dimension = 380; // Deep dimensions reasonable.
var dw;
var dh;
// set max dimension
if ((image.width > dimension) || (image.height > dimension)) {
if (image.width > image.height) {
// scale width to fit, adjust height
dw = parseInt(image.width * (dimension / image.width));
dh = parseInt(image.height * (dimension / image.width));
} else {
// scale height to fit, adjust width
dh = parseInt(image.height * (dimension / image.height))
dw = parseInt(image.width * (dimension / image.height));
}
canvas.width = dw;
canvas.height = dh;
}
else {
canvas.width = image.width;
canvas.height = image.height;
}
ctx.drawImage(image, 0, 0, dw, dh); // Put the photo on the canvas
setBorder(); // Update color border
}, false);
}
To keep the photo at a size that fits on the screen, the photo is scaled if it's too large. When the width or height are greater than the size of the canvas that's been designated, in this case 380 x 380 pixels, a scaling value is calculated. Because photos aren't necessarily square, only the largest dimension (width or height) is used for a scaling percentage. That value is used to scale the other dimension so the aspect ratio is preserved when displayed in the canvas.
The photo is then copied to the canvas using the drawImage method. The drawImage method takes an image object, and x and y pixel coordinates that define the upper-left corner in the image. With these values you can specify where on the image to start displaying, which could be used to display portions of a tiled image for animation. The width and height parameters are also used, which specify the size to make the image. For this example, the width and height of the image are scaled to 380 pixels or less, shrinking the image to fit the canvas. The drawImage method also supports four more parameters that can be used to place the image in the canvas. For more info, see the drawImage reference page.
Get a pixel value
To get the value of a pixel, the getImageData method is used to get an imageData object. The imageData object contains an pixelArray that contains the actual pixels of the canvas image. The pixelArray is arranged in RGBA (red, green, blue, alpha) format. To find the value of a single pixel that is the target of a mouse click, you need to calculate the index into the pixelArray based on the x and y coordinates, and the width of the canvas. The formula ((y * canvas.width) + x) * 4 gets the offset into the array. To get each color, start with the first value at the offset into the pixelArray, and then get the next three values. This will give you the RGBA value for the pixel as shown here:
canvas.onclick = function (evt) {
// get mouse coordinates from event parameter
var mouseX = parseInt(evt.offsetX);
var mouseY = parseInt(evt.offsetY);
// get imageData object from canvas
var imagedata = ctx.getImageData(0, 0, canvas.width, canvas.height);
// get pixelArray from imagedata object
var data = imagedata.data;
// calculate offset into array for pixel at mouseX/mouseY
var i = ((mouseY * canvas.width) + mouseX) * 4;
// get RGBA values
var r = data[i];
var g = data[i+1];
var b = data[i+2];
var a = data[i+3];
What we've done here uses the getImageData method to read pixels directly from a canvas image. The getImageData method incorporates a security requirement that stops a canvas webpage from copying pixels from one computer domain to another (cross domain). A cross-domain image can be transferred to the canvas using drawImage, but when the getImageData method is used to copy the pixels to the pixelArray, it throws a DOM exception security error (18).
The example contains a function called getSafeImageData() to catch cross domain errors without crashing the whole page. The getSafeImageData() function contains a try and catch statements to catch exceptions. If no exceptions occur, the function returns the pixelArray object.
If a security error occurs on a canvas element, it sets the origin-clean flag (an internal flag) to false. No other image can be loaded until the flag is cleared, which is done by refreshing the page, or destroying and creating a new canvas. The getSafeImageData() function, upon catching an exception, removes the canvas element from the page, creates a new canvas, and assigns the same attributes and events to the new one. After the new canvas (with a clean-origin flag set to true) is created, the error message is printed to the new canvas. The getSafeImageData() function returns an empty string, which is used to signal that the getImageData method failed. From here, you should load another image from the samples, or your own in the same domain.
To keep compatibility of the code between Windows 7 and Windows 8, the JavaScript parseInt() method is used to normalize the mouse coordinates to integers. In Windows 8, mouse coordinates are returned as floating point values to provide sub-pixel information for CSS and other UI functions. In this example, parseInt() is applied before the calculation to prevent rounding errors that will cause the wrong values to be returned from the pixelArray. As a side note, if you use a floating point number as an index into the pixelArray, it returns nothing because array indices are integer values only.
The RGBA value of the pixel that is returned from the pixelArray is converted to a hex value and used as the background color for the CSS border. Only the RGB values are used, and the following example shows how to convert the three values to a single CSS compatible color.
function getHex(data, i) {
// Builds a CSS color string from the RGB value (ignore alpha)
return ("#" + d2Hex(data[i]) + d2Hex(data[i + 1]) + d2Hex(data[i + 2]));
}
function d2Hex(d) {
// Converts a decimal number to a two digit Hex value
var hex = Number(d).toString(16);
while (hex.length < 2) {
hex = "0" + hex;
}
return hex.toUpperCase();
}
In this example, the pixelArray value that was calculated based on the x/y mouse coordinates is passed to the getHex function. The getHex function calls the d2Hex() function that converts the decimal RGB values into two digit hex values. Each converted RGB value is concatenated into a string, and a "#" sign is added to the beginning to create the CSS color value format. This value is then used to set the color of the border style, along with some other properties. For example, a teal colored border with rounded corners and a 3-D look is style= 'border-color:#4E8087; border-width:30px; border-style:groove; border-radius:20px' .
CSS elements
The frame around the canvas is created using the border property. The default values are 30px wide, with a solid style and no border radius. As the values change, the syntax is displayed below the image. The color of the border is set when you click the image. The style and radius (rounded corners) values are picked from drop-down menus.
The example offers five border styles. In addition of solid, you can choose 3-D border styles properties: Outset, inset styles, groove, and ridge. The outset and inset styles are flat frame styles with lighting effects. Grove and ridge styles are a more 3-D frame with lighting effects. The lighting effects give the appearance of a keylight coming from either the upper-left, or lower-right corner.
function setDispColors(bColor, fColor) {
// Set the color of anything with the class colorDisp.
var oDisplay = document.getElementsByClassName("colorDisp");
// Set the colors for each element
for (i = 0; i < oDisplay.length; i++) {
oDisplay[i].style.backgroundColor = bColor;
oDisplay[i].style.color = fColor;
}
}
The border radius style drop-down menu provides four values, 5, 10, 20, and 30 pixels. These numbers are arbitrary, and can be set to any value you think is appropriate. As CSS style values are updated, the syntax line shown under the image is updated.
Keeping data display readable
The example uses the sampled color values as a background color for the color value (RGB, CSS, and HSV) display fields. All the display fields, and the image's border property are assigned a class="colorDisp. This allows the example to quickly set the background color on all the elements at one time. The displays are colored using the setDispColors() function, which passes in the background and font colors.
function setDispColors(bColor, fColor) {
// Set the color of anything with the class colorDisp.
var oDisplay = document.getElementsByClassName("colorDisp");
// Set the colors for each element
for (i = 0; i < oDisplay.length; i++) {
oDisplay[i].style.backgroundColor = bColor;
oDisplay[i].style.color = fColor;
}
}
Because the background colors can range from a light beige to a deep dark black, the color of the font that is printed over the background is a concern. To ensure that the values are readable, the font color is set to either white or black, depending on the lightness or darkness of the colors. To figure out when to display one or the other, the sample color is converted to a gray scale, and if the value is greater than a set point (lighter), the text is printed black. Otherwise it's printed in white. The sample is set to a threshold of 128, or halfway between 0 and 255, the range of RGB colors. The following example shows the calcGray() function which determines the font color.
function calcGray(data, i) {
// Calculates the gray scale value of the chosen color and
// assigns a contrasting text color.
var gray = ((data[i] + data[i + 1] + data[i + 2]) / 3); // Average RGB values
return ((gray > 128) ? "black" : "white"); // Assign black or white to return
}
This technique works with most colors, but you can experiment with the threshold to fine tune for you page.
Convert a color photo to black and white
The same technique we just used to get the colors from a single pixel can be used to manipulate all the pixels. In the next example, all the pixels in the pixelArray are sampled and modified to convert a color photo to black and white. Each pixel's Red, Green, and Blue values are averaged to get a single number. The original values of each color channel of the pixel is replaced by the averaged value. Because each channel has the same value, the result is a grayscale image made up of values that range from black to white.
The "makeBW()" function shown in this example starts by getting the pixelArray from the canvas. A pair of for loops work through the pixelArray, getting each pixel value. The RGB values are added together and the sum is divided by three to get the average. The Alpha (or transparency) value is ignored. The resulting value is then copied back into each color value (RGB) of the pixel in the array. When all pixels have been converted, the putImageData method is used to put the pixelArray back into the canvas.
function makeBW() {
// Converts image to B&W by averaging the RGB channels, and setting each to that value
ctx.drawImage(image, 0, 0, canvas.width, canvas.height); // refresh canvas
imgData = getSafeImageData(0, 0, canvas.width, canvas.height);
if (imgData != "") {
for (y = 0; y < imgData.height; y++) {
for (x = 0; x < imgData.width; x++) {
var i = ((y * 4) * imgData.width) + (x * 4);
var aveColor = parseInt((imgData.data[i] + imgData.data[i + 1] + imgData.data[i + 2]) / 3)
imgData.data[i] = aveColor;
imgData.data[i + 1] = aveColor;
imgData.data[i + 2] = aveColor;
}
}
ctx.putImageData(imgData, 0, 0);
}
}
The code in the example kicks off when you click the Grayscale button. Click Load to reload the color image.
Adding a sepia or cyanotype tint
Turning a photo from color to black and white is a nice way to give a more nostalgic look to an image. To push the clock back ever further, you can go for a sepia or cyanotype tone. These processes have been used for over a hundred years to create effects and enhance their archival properties. The original processes used harsh chemicals to replace the silver in photographic paper with other compounds producing the color effects. With digital images, the process is similar, but with a lot less mess.
Converting a photo to sepia or cyanotype involves essentially several steps. The following steps are done on each pixel:
- Retrieve a pixel from the pixelArray.
- Convert the pixel to black and white.
- Convert the grayscale pixel (RGB) to HSV color model.
- Add (or subtract) Hue, Sat, and Val to the pixel's HSV value to create the tint.
- Convert the tinted pixel back to RGB color model.
- Put the pixel back into the pixelArray.
The following example shows the "rgb2hsv()" and "hsv2rgb()" functions, and the "makeTint()" function that adds the HSV values to the pixels. The tints are added as HSV values because it's easier to do with a single value. The Hue is the color based on a 360 degree scale, often shown in software as a color wheel. Saturation and Value (sometimes known as Lightness, Luminosity, or Brightness) are expressed as percentages (0-100%). Tinting an image to a sepia tone is done by adding a "Hue = 30", and a "Sat = 30" to the grayscale values. Nothing is added to the Val parameter for sepia tone. To create a cyanotype image, the formula is to add a "Hue = 220", and "Sat = 40". Val is set to add 10% to lighten the image just slightly, as the bluish tint can appear darker than you might want. This value is arbitrary, and could be different depending on the photos. For more info about these models, see the article HSL and HSV in the Wikipedia.
function rgb2hsv(r, g, b) {
// Converts RGB value to HSV value
var Hue = 0;
var Sat = 0;
var Val = 0;
// Convert to a percentage
r = r / 255; g = g / 255; b = b / 255;
var minRGB = Math.min(r, g, b);
var maxRGB = Math.max(r, g, b);
// Check for a grayscale image
if (minRGB == maxRGB) {
Val = parseInt((minRGB * 100) + .5); // Round up
return [Hue, Sat, Val];
}
var d = (r == minRGB) ? g - b : ((b == minRGB) ? r - g : b - r);
var h = (r == minRGB) ? 3 : ((b == minRGB) ? 1 : 5);
Hue = parseInt(60 * (h - d / (maxRGB - minRGB)));
Sat = parseInt((((maxRGB - minRGB) / maxRGB) * 100) + .5);
Val = parseInt((maxRGB * 100) + .5); // Round up
return [Hue, Sat, Val];
}
function hsv2rgb(h, s, v) {
// Set up rgb values to work with
var r;
var g;
var b;
// Sat and value are expressed as 0 - 100%
// convert them to 0 to 1 for calculations
s /= 100;
v /= 100;
if (s == 0) {
v = Math.round(v * 255); // Convert to 0 to 255 and return
return [v, v, v]; // Grayscale, just send back value
}
h /= 60; // Divide by 60 to get 6 sectors (0 to 5)
var i = Math.floor(h); // Round down to nearest integer
var f = h - i;
var p = v * (1 - s);
var q = v * (1 - s * f);
var t = v * (1 - s * (1 - f));
// Each sector gets a different mix
switch (i) {
case 0:
r = v;
g = t;
b = p;
break;
case 1:
r = q;
g = v;
b = p;
break;
case 2:
r = p;
g = v;
b = t;
break;
case 3:
r = p;
g = q;
b = v;
break;
case 4:
r = t;
g = p;
b = v;
break;
default:
r = v;
g = p;
b = q;
break;
}
// Convert all decimial values back to 0 - 255
return [Math.round(r * 255), Math.round(g * 255), Math.round(b * 255)];
}
// Accept and add a Hue, Saturation, or Value for tinting.
function makeTint(h, s, v) {
// Converts color to b&w, then adds tint
var imgData = getSafeImageData(0, 0, canvas.width, canvas.height);
if (imgData != "") {
for (y = 0; y < imgData.height; y++) {
for (x = 0; x < imgData.width; x++) {
var i = ((y * imgData.width) + x) * 4; // our calculation
// Get average value to convert each pixel to black and white
var aveColor = parseInt((imgData.data[i] + imgData.data[i + 1] + imgData.data[i + 2]) / 3)
// Get the HSV value of the pixel
var hsv = rgb2hsv(aveColor, aveColor, aveColor);
// Add incoming HSV values (tones)
var tint = hsv2rgb(hsv[0] + h, hsv[1] + s, hsv[2] + v);
// Put updated data back
imgData.data[i] = tint[0];
imgData.data[i + 1] = tint[1];
imgData.data[i + 2] = tint[2];
}
}
// Refresh the canvas with updated colors
ctx.putImageData(imgData, 0, 0);
}
}
function sepia() {
// Refresh the canvas from the img element
ctx.drawImage(image, 0, 0, canvas.width, canvas.height);
makeTint(30, 30, 0);
}
function cyanotype() {
// Refresh the canvas from the img element
ctx.drawImage(image, 0, 0, canvas.width, canvas.height);
makeTint(220, 40, 10);
}
Summary
There are many other things you can do by grabbing the value of pixels in a canvas. For example, you can test sampled pixels for a specific color value, and overlay another image to do a simple chroma key (green screen) effect. This effect is used in television and movies to composite images that can put a weatherman in front of a raging storm or an actor on the top of a mountain, from the comfort of the studio.