
<WebGLImageProcessingIntro />

In this article we’ll look at how to implement Photoshop-like Blend Modes with WebGL. Many of these algorithms come from the [glsl-blend package](https://github.com/jamieowen/glsl-blend) which is licensed under the MIT license.

## What are Blend Modes

Blend Modes are algorithms that blend two color values. We take a base color, add a blend color and output a result color.  

Blend Modes were introduced with Photoshop 3.0 back in 1994. We’ll cover a subset of these and how to create them in GLSL.

## Neutral Colors

Blend Modes have Neutral Colors that have no effect when blended, we can group Blend Modes by their Neutral Colors:
- Normal. Does not have a Neutral Color.
- Darken. White is the Neutral Color and will become transparent.
- Lighten. Black is the Neutral Color and will become transparent.
- Contrast. 50% gray is the Neutral Color and will become transparent.

## Normal Blend Modes

### Normal

Normal is the default Blending Mode for Photoshop layers and does not have an algorithm that blends the color data. The result color will always be the blend color.

<Dither step="blend-normal" />

```glsl
vec4 blendNormal(vec4 base, vec4 blend){
  return blend;
}
```

## Darken Blend Modes

These Blend Modes make the Result Color darker. White is the Neutral Color for this category and will become transparent.

### Darken

Darken compares the base and blend color and outputs the darker value as its result color. This is the opposite of the Lighten Blend Mode we’ll cover later.

<Dither step="blend-darken" />

```glsl 
vec4 blendDarken(vec4 base, vec4 blend){
  return vec4(
    min(base.r, blend.r),
    min(base.g, blend.g),
    min(base.b, blend.b),
    1.0
  );
}
```

### Multiply

The multiply Blend Mode, multiplies the base and blends colors, resulting in a darker color. Multiplying any color by black results in black, while white will have no effect. This is the opposite of the Screen Blend Mode we’ll cover later.

<Dither step="blend-multiply" />

```glsl 
vec4 blendMultiply(vec4 base, vec4 blend){
  return vec4(
    base.r * blend.r,
    base.g * blend.g,
    base.b * blend.b,
    1.0
  );
}
```

### Color Burn

Color Burn darkens the base color to reflect the blend color by increasing the contrast between the two. This is the opposite of the Color Dodge Blend Mode we’ll cover later.

<Dither step="blend-color-burn" />

```glsl 
float blendColorBurn(float base, float blend){
  return (blend==0.0)?blend:max((1.0-((1.0-base)/blend)),0.0);
}

vec4 blendColorBurn(vec4 base, vec4 blend){
  return vec4(
    blendColorBurn(base.r, blend.r),
    blendColorBurn(base.g, blend.g),
    blendColorBurn(base.b, blend.b),
    1.0
  );
}
```

## Lighten Blend Modes

These Blend Modes make the Result Color lighter, and are generally complementary to the Darken Blend Modes. Black is the Neutral Color for this category and will become transparent.

### Lighten

Lighten compares the base and blend color and outputs the lighter value as its result color. This is the opposite of the Darken Blend Mode we covered previously.

<Dither step="blend-lighten" />

```glsl 
vec4 blendLighten(vec4 base, vec4 blend){
  return vec4(
    max(base.r, blend.r),
    max(base.g, blend.g),
    max(base.b, blend.b),
    1.0
  );
}
```

### Screen

The Screen Blend Mode, multiplies the inverse of the base and blend colors, resulting in a lighter color. Screening any color by white results in white, while black will have no effect. This is the opposite of the Multiply Blend Mode we covered previously.

<Dither step="blend-screen" />

```glsl 
vec4 blendScreen(vec4 base, vec4 blend){
  return vec4(
    1.0 - ((1.0 - base.r) * (1.0 - blend.r)),
    1.0 - ((1.0 - base.g) * (1.0 - blend.g)),
    1.0 - ((1.0 - base.b) * (1.0 - blend.b)),
    1.0
  );
}
```

### Color Dodge

Color Doge brightens the base color to  reflect the blend color by decreasing contrast between the two. This is the opposite of the Color Burn Blend Mode we covered previously.

<Dither step="blend-color-dodge" />

```glsl 
float blendColorDodge(float base, float blend){
  return (blend == 1.0) ? blend : min(base / (1.0 - blend), 1.0);
}

vec4 blendColorDodge(vec4 base, vec4 blend){
  return vec4(
    blendColorDodge(base.r, blend.r),
    blendColorDodge(base.g, blend.g),
    blendColorDodge(base.b, blend.b),
    1.0
  );
}
```

## Contrast Blend Modes

Contrast Blend Modes build on the Blend Modes we just covered. Rather than always darkening or lightening the color, they will darkening if the **base color** is darker than 50% gray or lighter if it’s lighter than 50% gray.

### Overlay

Overlay builds on the Screen and Multiply Blend Modes. It uses Screen on color brighter than 50% gray and Multiply on colors darker than 50% gray. 

<Dither step="blend-overlay" />

```glsl 
float blendOverlay(float base, float blend){
  return base < 0.5 ? (2.0*base*blend):(1.0-2.0*(1.0-base)*(1.0-blend));
}

vec4 blendOverlay(vec4 base, vec4 blend){
  return vec4(
    blendOverlay(base.r, blend.r),
    blendOverlay(base.g, blend.g),
    blendOverlay(base.b, blend.b),
    1.0
  );
}
```

### Soft Light

Soft Light darkens or lightens the colors, depending on the blend color. The effect mimics shining a diffused light on the image. If the blend color is lighter than 50% gray, the image is lightened. If the blend color is darker than 50% gray, the image is darkened.

<Dither step="blend-soft-light" />

```glsl 
float blendSoftLight(float base, float blend){
  return (blend<0.5)?(2.0*base*blend+base*base*(1.0-2.0*blend)):(sqrt(base)*(2.0*blend-1.0)+2.0*base*(1.0-blend));
}

vec4 blendSoftLight(vec4 base, vec4 blend){
  return vec4(
    blendSoftLight(base.r, blend.r),
    blendSoftLight(base.g, blend.g),
    blendSoftLight(base.b, blend.b),
    1.0
  );
}
```

### Hard Light

Hard Light darkens or lightens the colors, depending on the blend color.The effect mimics shining a harsh light on the image. If the blend color is darker than 50% gray, the image is darkened.

<Dither step="blend-hard-light" />

```glsl 
float blendHardLight(float base, float blend){
  return base < 0.5 ? (2.0*base*blend):(1.0-2.0*(1.0-base)*(1.0-blend));
}

vec4 blendHardLight(vec4 base, vec4 blend){
  return vec4(
    blendHardLight(base.r, blend.r),
    blendHardLight(base.g, blend.g),
    blendHardLight(base.b, blend.b),
    1.0
  );
}
```

In the [next article](/p/webgl-film-grain), we'll look at how we can add an analog feel to our images by simulating film grain.

<ArticlePreview slug="webgl-film-grain" />



<WebGLImageProcessingIntro />

## What is Chromatic Aberration?

Chromatic aberration, or color fringing, occurs in a camera lens when it's unable to bring all wavelengths of color to the same focal plane.

We can simulate this effect by moving one or more of our RGB channels from our texture along the $x$ or $y$ axis. Here are two examples, in which we can move the red and blue channels:

<Dither step="chromatic-aberration-red" />

<Dither step="chromatic-aberration-blue" />

The GLSL code for this is straightforward. Below, we pass our texel data into our chromatic aberration function. This generates a vec4 color value based on the texel at our program's UV coordinates (aka. the pixel's location on the canvas). We can then move our red channel by an `xOffset` and `yOffset`. 

```glsl
vec4 chromaticAberration(vec2 uv, vec4 fragColor) {
  return vec4(
    texture(
      u_image, 
      vec2(uv.x + (0.1 * xOffset), uv.y + (0.1 * yOffset))
    ).x,
    texture(
      u_image, 
      uv
    ).y,
    texture(
      u_image, 
      uv
    ).z,
    1.0
  );
}

void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  vec4 texel = texture(u_image, uv);
  outColor = chromaticAberration(uv, texel);
}
```

## Distance from edges

Above we moved an entire color channel across the whole image at a constant rate. To mimic the effect created by a camera lens we need to increase the movement of the channel as it approaches the edge of the image. This is similar to the vignette effect in the [previous article](/p/webgl-vignette).

<Dither step="chromatic-aberration" />

In GLSL, we can achieve this by getting the UV's distance from the center with `length(uv - 0.5)` and then multiplying this by an offset for each channel (this is controlled by the dials).

```glsl 
vec4 chromaticAberration(vec2 uv, vec4 fragColor) {
  float d = length(uv - 0.5);

  return vec4(
    texture(
      u_image, 
      uv - d * 0.1 * redOffset
    ).x,
    texture(
      u_image, 
      uv + d * 0.1 * blueOffset
    ).y,
    texture(
      u_image, 
      uv - d * 0.1 * greenOffset
    ).z,
    1.0
  );
}

void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  vec4 texel = texture(u_image, uv);
  outColor = chromaticAberration(uv, texel);
}
```

This is the most recent article in the series. Follow me on [twitter](https://twitter.com/maximmcnair) for future articles.




<WebGLImageProcessingIntro />

<Dither step="6-dither-8x8" />

We'll improve our thresholding algorithms from the [previous article](/p/webgl-thresholding) by replacing block colors with patterns that, from a distance, look like a shade of a color. Here's an example of rendering a horizontal gradient with different dithering effects we'll cover in this article:

<Dither step="dither-gradient" label="Dithering patterns on a gradient"/>

## Ordered Dithering 

We use an algorithm called **ordered dithering** to create this effect. We can define our patterns with a dither matrix such as:

$$
\begin{bmatrix}
  0.2 & 0.8 \\
  0.6 & 0.4
\end{bmatrix}
$$

Each pixel of our image is checked against this matrix, repeating for the size of the image. For the matrix above the first pixel on the top row will be thresholded to the value 0.2. Anything above 0.2 turns white and anything below turns black. The second pixel by 0.8 and the third pixel will loop back to the first value in our matrix 0.2. The second pixel row will start from 0.6. This will repeat for the whole image.

This 2x2 matrix can give us 5 different thresholds, as shown in the image below.

<Dither step="gradient-dither-2x2" />

The data for patterns is normally stored as integers, and we use this equation to create the threshold values.

$$
\begin{bmatrix}
  0 & 3 \\
  2 & 1 
\end{bmatrix}
$$

$$
d_{i,j} =
\frac{1 + M_{i,j}}{1 + m \times n}
$$

Here is our GLSL code that will take our dither_matrix, convert it into threshold data, and output our result:

```glsl
const int dither_matrix_2x2[4] = int[](
   0,  3,  
   2,  1
);
float dither2x2(vec2 uv, float luma) {
  float dither_amount = 2.0;
  int x = int(mod(uv.x, dither_amount));
  int y = int(mod(uv.y, dither_amount));
  int index = x + y * int(dither_amount);
  float limit = (float(dither_matrix_2x2[index]) + 1.0) / (1.0 + 4.0);
  return luma < limit ? 0.0 : 1.0;
}
```

## Patterns

The beauty of this algorithm is that we can easily expand on it with different patterns and matrix sizes for a number of different results:

### 2x2 Bayer ordered dither matrix

$$
\begin{bmatrix}
  0 & 3 \\
  2 & 1 
\end{bmatrix}
$$

<Dither step="4-dither-2x2" />

### 4x4 Bayer ordered dither matrix
$$
\begin{bmatrix}
  0 & 8 & 2 & 10 \\
  12 & 4 & 14 & 6 \\
  3 & 11 & 1 & 9 \\
  15 & 7 & 13 & 5
\end{bmatrix}
$$

<Dither step="5-dither-4x4" />

### 8x8 Bayer ordered dither matrix
$$
\begin{bmatrix}
   0 & 32 & 8 & 40 & 2 & 34 & 10 & 42 \\
  48 & 16 & 56 & 24 & 50 & 18 & 58 & 26 \\
  12 & 44 & 4 & 36 & 14 & 46 & 6 & 38 \\
  60 & 28 & 52 & 20 & 62 & 30 & 54 & 22 \\
   3 & 35 & 11 & 43 & 1 & 33 & 9 & 41 \\
  51 & 19 & 59 & 27 & 49 & 17 & 57 & 25 \\
  15 & 47 & 7 & 39 & 13 & 45 & 5 & 37 \\
  63 & 31 & 55 & 23 & 61 & 29 & 53 & 21
\end{bmatrix}
$$

<Dither step="6-dither-8x8" />

### 8x8 cluster matrix
$$
\begin{bmatrix}
  24 & 10 & 12 & 26 & 35 & 47 & 49 & 37 \\
  8 & 0 & 2 & 14 & 45 & 59 & 61 & 51 \\
  22 & 6 & 4 & 16 & 43 & 57 & 63 & 53 \\
  30 & 20 & 18 & 28 & 33 & 41 & 55 & 39 \\
  34 & 46 & 48 & 36 & 25 & 11 & 13 & 27 \\
  44 & 58 & 60 & 50 & 9 & 1 & 3 & 15 \\
  42 & 56 & 62 & 52 & 23 & 7 & 5 & 17 \\
  32 & 40 & 54 & 38 & 31 & 21 & 19 & 29
\end{bmatrix}
$$

<Dither step="7-dither-cluster" />

This 8x8 cluster dot matrix mimics the halftoning techniques used by newspapers.

## Color Halftone

We can produce an even more interesting effect by taking each RGB channel independently instead of using our grayscale value.

```glsl 
void main() {
  // load texture and set texel as outColor

  // get image grayscale
  vec4 luma = vec4(0.299, 0.587, 0.114, 0);
  float grayscale = dot(outColor, luma);

  outColor = vec4(
    outColor.r * dither(gl_FragCoord.xy, grayscale),
    outColor.g * dither(gl_FragCoord.xy, grayscale),
    outColor.b * dither(gl_FragCoord.xy, grayscale),
    1.0
  );
}
```

$$
\begin{bmatrix}
   0 & 32 & 8 & 40 & 2 & 34 & 10 & 42 \\
  48 & 16 & 56 & 24 & 50 & 18 & 58 & 26 \\
  12 & 44 & 4 & 36 & 14 & 46 & 6 & 38 \\
  60 & 28 & 52 & 20 & 62 & 30 & 54 & 22 \\
   3 & 35 & 11 & 43 & 1 & 33 & 9 & 41 \\
  51 & 19 & 59 & 27 & 49 & 17 & 57 & 25 \\
  15 & 47 & 7 & 39 & 13 & 45 & 5 & 37 \\
  63 & 31 & 55 & 23 & 61 & 29 & 53 & 21
\end{bmatrix}
$$

<Dither step="8-dither-color" />

## Halftone with Duotone

We can also add color to our image by reusing our [duotone function](/p/webgl-duotone) from the [duotone article](/p/webgl-duotone) in this series. Take the grayscale halftone image we produced earlier and pass into our duotone (mix) function to add a gradient map to it.

$$
\begin{bmatrix}
   0 & 32 & 8 & 40 & 2 & 34 & 10 & 42 \\
  48 & 16 & 56 & 24 & 50 & 18 & 58 & 26 \\
  12 & 44 & 4 & 36 & 14 & 46 & 6 & 38 \\
  60 & 28 & 52 & 20 & 62 & 30 & 54 & 22 \\
   3 & 35 & 11 & 43 & 1 & 33 & 9 & 41 \\
  51 & 19 & 59 & 27 & 49 & 17 & 57 & 25 \\
  15 & 47 & 7 & 39 & 13 & 45 & 5 & 37 \\
  63 & 31 & 55 & 23 & 61 & 29 & 53 & 21
\end{bmatrix}
$$

<Dither step="9-dither-duotone" />

```glsl 
void main() {
  // make image grayscale
  vec4 luma = vec4(0.299, 0.587, 0.114, 0.0);
  float grayscale = dot(outColor, luma);
  
  outColor = vec4(
    vec3(dither(gl_FragCoord.xy, grayscale)),
    1.0
  );

  vec4 lowcolor = vec4(0.141, 0.031, 0.318, 1.0);
  vec4 highcolor = vec4(0.957, 0.239, 0.122, 1.0);
  outColor = mix(lowcolor, highcolor, outColor);
}
```

In the [next article](/p/webgl-vignette), we'll be exploring how to create a vignette effect.

<ArticlePreview slug="webgl-vignette" />



<WebGLImageProcessingIntro />

This article was inspired by [threshold/halftone article](http://caca.zoy.org/study/index.html), but we'll look at this from a WebGL/GLSL viewpoint and show the code necessary to obtain these effects.

## What is Thresholding?
Thresholding segments an image's color values into a smaller range. It's a method of color quantization which was mainly developed for image compression (reducing the size of images) to meet the limitations of the technology available at that time.

Computers started out with extremely limited color rendering options. 8-bit pixel depth was 256 colors, while 4-bit was only 16 colors. And to make it worse, different machines had different color palettes. Any image rendered on a system had to be rendered in those colors.

We don't have these issues anymore - image compression is still in use but not with dramatic effects - but thresholding and dithering algorithms are still used for their aesthetic look.

We'll use this image for the rest of the article, with a gradient bar on the side to make it easier to understand what's happening to the color range.

## A basic example
Instead of using the whole range of GLSL's 0-1 floating point color channel, we could round our values to either 0 (if our color value is lower than 0.5) or 1 (if our color value is higher than 0.5).

The results of this segmentation depend on the threshold amount, and the white balance of our image. For different images there will be a better threshold amount to pick. Try changing the threshold amount for the image below, see how the gradient on the side is clamped to either black or white at a specific point.

<Dither step="1-threshold" />

## Threshold

This isn't the most specular image - in fact it looks pretty bad - but it's simple to understand and will lead us to more interesting effects. Let's break it down with GLSL.

We'll start with the grayscale function we covered in the [color correction article](/p/webgl-color-correction).

```glsl
  // make image grayscale
  vec4 lum = vec4(0.2126, 0.7152, 0.0722, 0);
  float grayscale = dot(outColor, lum);
```

Now that we have a grayscale image we can take our 0-1 color channel value and round up or down depending on our threshold value (we've used 0.5 here).

```glsl
float thresholdValue = 0.5;

float threshold(float color) {
  if (color > thresholdValue) {
    return 1.0;
  }
  return 0.0;
}

outcolor = vec4(vec3(threshold(grayscale)), 1.0);
```

## Multi-Step Threshold

In our binary threshold we lose a lot of information from our image, which makes it hard to recognise what we're looking at. A simple improvement to this would be to add more information: instead of a binary 2-value (white, black) image we could use 3 values (white, gray, black).

We can then set these threshold amounts to different values.

<Dither step="2-threshold-steps" />

We achieve this by adding an additional condition statement to our threshold function.

```glsl
float thresholdWhiteValue = 0.5;
float thresholdGreyValue = 0.25;

float threshold(float color) {
  if (color > thresholdWhiteValue) {
    return 1.0;
  }
  if (color > thresholdGreyValue) {
    return 0.5;
  }
  return 0.0;
}
```

## Threshold with noise

A simple way to improve our threshold is to add some noise between our thresholded steps.

<Dither step="3-threshold-noise" />

```glsl
float rand(vec2 uv) {
  return fract(sin(dot(uv.xy, vec2(12.9898,78.233))) * 43758.5453);
}

float grainMultiplier = 1.2;

void main() {
  // ... load image 
  float thresholded = threshold(
    grayscale + ((rand(uv) / 9.0) * grainMultiplier)
  );

  outColor = vec4(vec3(thresholded), 1.0);
}
```

In the [next article](/p/webgl-dithering), we'll extend this effect to use dithering/halftone patterns for each threshold step.

<ArticlePreview slug="webgl-dithering" />



<WebGLImageProcessingIntro />

## What is vignetting?

Vignetting is the reduction of an image's brightness or saturation towards its edges. While this was historically an undesired effect in photography – an indicator of incorrect camera settings or issues with the lens – it can also be used to lead the viewer's eye to a specific point on an image.

We look at code from Tyler Lindberg's [glsl-vignette](https://github.com/TyLindberg/glsl-vignette) here, which is licensed on a [MIT license](https://github.com/TyLindberg/glsl-vignette/blob/master/LICENSE).

In its simplest form we can take the distance from the center of the image and create a radial gradient that darkerns towards the edges. I've included an additional preview of the vignette over a white background to get a complete picture of the vignette's shape.

<Dither step="vignette-basic" />

```glsl
float vignette(vec2 uv, float radius){
  return radius - distance(uv, vec2(0.5));
}

void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  // load image
  vec4 texel = texture(u_image, uv);
  // apply vignette to texel (image)
  outColor = texel * vignette(uv, u_radius);
}
```

## Smoothstep

The main issue with the above approach is that we lose a lot of detail from the center of the image. We can use smoothstep to control how smooth the fade is between our image and the darkness at the edges. 

<Dither step="vignette-simple" />

```glsl 
// Based on https://github.com/TyLindberg/glsl-vignette/blob/master/simple.glsl

float vignette(
  vec2 uv, 
  float radius, // (0 - 1)
  float smoothness // (0 - 1)
) {
  float diff = radius - distance(uv, vec2(0.5));
  return smoothstep(-smoothness, smoothness, diff);
}

void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  // load image
  vec4 texel = texture(u_image, uv);
  // apply vignette to texel (image)
  outColor = texel * vignette(uv, u_radius, u_smoothness);
}
```

## Vignette shape

We can also use separate values for our width and height to control the shape of our vignette.

<Dither step="vignette" />

```glsl 
// Based on https://github.com/TyLindberg/glsl-vignette/blob/master/advanced.glsl

// Based on distance functions found at:
// http://iquilezles.org/www/articles/distfunctions/distfunctions.htm
float sdSquare(vec2 point, float width) {
  vec2 d = abs(point) - width;
  return min(max(d.x, d.y), 0.0) + length(max(d, 0.0));
}

float vignette(
  vec2 uv, 
  vec2 size, // (0 - 0.5)
  float roundness, // (0 - 1)
  float smoothness // (0 - 1)
) {
  // Center UVs
  uv -= 0.5;

  // Shift UVs based on the larger of width or height
  float minWidth = min(size.x, size.y);
  uv.x = sign(uv.x) * clamp(abs(uv.x) - abs(minWidth - size.x), 0.0, 1.0);
  uv.y = sign(uv.y) * clamp(abs(uv.y) - abs(minWidth - size.y), 0.0, 1.0);

  // Signed distance calculation
  float boxSize = minWidth * (1.0 - roundness);
  float dist = sdSquare(uv, boxSize) - (minWidth * roundness);

  return 1.0 - smoothstep(0.0, smoothness, dist);
}

void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  vec4 texel = texture(u_image, uv);
  outColor = texel * vignette(uv, vec2(u_width, u_height), u_roundness, u_smoothness);
}
```

In the [next article](/p/webgl-chromatic-aberration), we'll be exploring how to simulate the effect of chromatic aberration.

<ArticlePreview slug="webgl-chromatic-aberration" />



<WebGLImageProcessingIntro />

## What is duotone?

Duotone is a simple effect that maps a black-and-white image into a color range. Its usage dates back to the mid-20th Century, with Andy Warhol being one of the most famous people to popularize it. More recently, Spotify made extensive use of it in its brand communications.

<div style={{display: 'flex', gap: '10px', marginBottom: '20px'}}>
    <div style={{textAlign: 'center', marginBottom: 20}}>
        <img src="/andywarhol-museum.webp" />
        <small>Andy Warhol Museum. Photo by Dean Kaufman [[1]](https://www.warhol.org/)</small>
    </div>

    <div style={{textAlign: 'center'}}>
        <img src="/spotify-duotone.jpg" />
        <small>Spotify's Year in Music [[2]](https://www.dandad.org/en/d-ad-spotify-year-music-case-study-insights/)</small>
    </div>
</div>

Click on the color pairs below to see what happens to this image when you apply a duotone effect:

<Duotone />

I'm going to run through a simple method to create this effect by using a gradient map on a black and white image.

First we'll need a black and white image. Let's reuse our desaturated function we covered in the [color correction article](/p/webgl-color-correction) of this series to create one.

```glsl
vec4 desaturated(vec4 color) {
  // sRGB colorspace luminosity factor
  vec3 lumFactor = vec3(0.2126, 0.7152, 0.0722);
  vec3 rgb = vec3(dot(color, luminosityFactor));
  return vec4(rgb, color.a);
}
```

Now we have a black and white (desaturated) image, we want to map black to our low color and white to our high color.

```glsl
vec4 duotone(vec4 fragColor, vec4 lowcolor, vec4 highcolor) {
  vec4 desaturated = desaturated(fragColor);
  return mix(lowcolor, highcolor, desaturated);
}
```

In the [next article](/p/webgl-pixelate), we'll take a look at how to add a pixelate effect to our image.

<ArticlePreview slug="webgl-pixelate" />



<WebGLImageProcessingIntro />

## What is hue?

Hue is the main indication of a color. It's the value that defines the color itself. 

Changing the hue of an image can change the color of the object it depicts, meaning a tomato would no longer be red, or that a banana would no longer be yellow.

**HSB (hue, saturation, brightness)** is an alternative to the **RGB color model**. In this model, colors of each hue are arranged in a radial slice around the central axis of neutral colors, with complete brightness at the top and darkness at the bottom. 

Here are two diagrams from [POV-Ray](http://povray.org) to illustrate the difference between RGB on the left, and HSV (or HSB) on the right. 

<div style={{display: 'flex', gap: '10px', marginBottom: '20px'}}>
  <img src="/color-model-rgb.png" style={{backgroundColor: 'white', width: 'calc(50% - 5px)'}}/>
  <img src="/weblgl-hsb-cylinder.png" style={{backgroundColor: 'white', width: 'calc(50% - 5px)'}}/>
</div>

If we extract just the hue value, we can see how the color changes in the **range of 0° to 360°**. We have the red primary at 0°, the green primary at 120°, the blue primary at 240°, and then wrap back to red at 360°.

<Dither step="gradient-hsb" label="Hue color range from 0° to 360°" />

## Converting RGB to HSB

Similarly to how RGB values are mapped from 0 to 1 in GLSL instead of the common 0 to 255 range, we also need to adapt our HSB values. We want them to be mapped to 0 to 1, instead of 0° to 360°.

<Dither step="gradient-hsb" label="Hue color range from 0 to 1" />

In GLSL we can convert our RGB vec3 to a HSB vec3 with the following function:

```glsl
vec3 rgb2hsb(vec3 c) {
  vec4 K = vec4(0.0, -1.0 / 3.0, 2.0 / 3.0, -1.0);
  vec4 p = mix(vec4(c.bg, K.wz), vec4(c.gb, K.xy), step(c.b, c.g));
  vec4 q = mix(vec4(p.xyw, c.r), vec4(c.r, p.yzx), step(p.x, c.r));
  float d = q.x - min(q.w, q.y);
  float e = 1.0e-10;
  return vec3(abs(q.z + (q.w - q.y) / (6.0 * d + e)), d / (q.x + e), q.x);
}
```

And back to RGB from HSB with the following function:

```glsl
vec3 hsb2rgb(vec3 c){
  vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
  vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
  return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}
```

We'll use our standard photos (below) to see how these algorithms update an image's color values. The color gradient bar on the right is useful to see how individual color values change.

<Dither step="0-photos" />

## Shifting an image's hue

We can shift the hue of our image easily once we convert our color values from an RGB color model into an HSB color model.

We can adjust the hue value to morph our image into another color. This means updating the first HSB value of our color.

```glsl
void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  vec4 texel = texture(u_image, uv);
  outColor = texel;

  vec3 hsb = rgb2hsb(texel.rgb);
  hsb[0] = 0.4;
  outColor = vec4(hsb2rgb(hsb), 1.0);
}
```

<Dither step="hue" />

Notice that the entire color range on the right is changed to a specific color hue. Let's look at how we can change only a specific range of hues.

## Shifting hue range

For this example, we'll break our HSB color range into 6 sections to enable us to pick a range to change.

<Dither step="gradient-hsb" label="Hue color range from 0 to 1" />

<Dither step="gradient-hsb-rounded" label="Hue color range broken into 6 sections" />

We've broken our `0 - 1` range into 6 sections. Red here would be `0 - 0.166`, yellow `0.166 - 0.333` etc. This will give us some basic hue ranges to work with in our example below.

We can select our **red** range and then move the slider to change the hue. This will morph all the hues in this range into another hue. You'll see how a section of gradient on the right will change while the rest stays the same.

<Dither step="hue-range" />

For the flower image, our first **red** range doesn't cover the entire **red** range of the flower itself. We also need some hues from the last **magenta** range.

But the green range manages to change all the green leaves in the background. This is up to luck, as the ranges were choosen by spliting our hue range into 6 sections.

We can achieve this effect in GLSL by only updating the hue if the texel's original hue value lies in-between our range's min and max values.

```glsl
void main() {
  // ...load image
  vec3 hsb = rgb2hsb(texel.rgb);
  if (hsb[0] >= range_min && hsb[0] <= range_max) {
    hsb[0] = u_fx;
  }
  outColor = vec4(hsb2rgb(hsb), 1.0);
}
```

In the [next article](/p/webgl-blend-modes), we'll look into the many ways we can blend two color values together.

<ArticlePreview slug="webgl-blend-modes" />



<WebGLImageProcessingIntro />

## What is pixelation?

Pixelation is normally a side-effect of displaying an image at a higher resolution than it has data for. While this is generally seen as an issue, we can also mimic this effect for aesthetic reasons.

<Dither step="pixelate" />

In GLSL, we can achieve this effect by rounding our x/y values when fetching our texel data. Rounding by a specific ratio will control the size of the pixelation.

In layman's terms, this means collecting the data from a pre-determined selection of pixels, rather than every single one. For exammple, every other pixel, every 5 pixels, or every 10 pixels. The bigger the distance between pixels, the more pixellated the image will become.

```glsl
vec4 pixelate(float size, vec2 uv) {
  float dx = size / u_resolution.x;
  float dy = size / u_resolution.y;
  float x = dx * (floor(uv.x / dx) + 0.5);
  float y = dy * (floor(uv.y / dy) + 0.5);
  return texture(
    u_image,
    vec2(x, y)
  );
}

void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  vec4 texel = texture(u_image, uv);
  outColor = texel;

  outColor = pixelate(60.0, uv);
}
```

In the [next article](/p/webgl-thresholding), we'll take a look into thresholding our color data.

<ArticlePreview slug="webgl-thresholding" />



<WebGLImageProcessingIntro />

## What is film grain?

Film grain is an optical effect created from the presence of small particles of metallic silver or dye clouds found in the liquid when developing film stock.

We can simulate this effect in GLSL by generating noise, then blending this into our RGB values.

<Dither step="grain" />

Let's first take a look at how we can use a hash function to generate random noise (though generating truly random noise in WebGL is broadly considered impossible).

```glsl
float rand(vec2 co){
  return fract(sin(dot(co.xy, vec2(12.9898, 78.233))) * 43758.5453);
}

vec4 grain(vec4 fragColor, vec2 uv){
  vec4 color = fragColor;
  float diff = (rand(uv) - 0.0) * u_grain_amount;
  color.r += diff;
  color.g += diff;
  color.b += diff;
  return color;
}
```

<Dither step="grain-noise" />

Now that we have our grain, we can mix this with our image.

```glsl 
void main() {
  // map uv between 0 -> 1
  vec2 uv = gl_FragCoord.xy/u_resolution;
  vec4 texel = texture(u_image, uv);
  outColor = texel;

  vec4 grain = grain(outColor, uv);
  outColor = mix(outColor, grain, u_fx);
}
```

<Dither step="grain" />

This has added some texture to the image, but it looks more like digital noise than the grain from film processing.

## Creating more realistic film grain

Researching this further led me to Matt DesLauriers' [glsl-film-grain](https://github.com/mattdesl/glsl-film-grain) package, based on an article by [Martins Upitis](https://devlog-martinsh.blogspot.com/2013/05/image-imperfections-and-film-grain-post.html). It uses a mixture of perlin and simplex noise for the grain, along with a blend mode that adapts the amount of noise added to each pixel depending on its luminosity. 

Luckily, this package is under the [MIT license](https://github.com/mattdesl/glsl-film-grain/blob/master/LICENSE.md) so let's take a closer look. It uses the `pnoise3` and `snoise3` from [glsl-noise](https://github.com/hughsk/glsl-noise#readme).

```glsl
float grain(vec2 texCoord, vec2 resolution, float frame, float multiplier) {
  vec2 mult = texCoord * resolution;
  float offset = snoise3D(vec3(mult / multiplier, frame));
  float n1 = pnoise3D(vec3(mult, offset), vec3(1.0/texCoord * resolution, 1.0));
  return n1 / 2.0 + 0.5;
}

void main() {
    float grainSize = 2.0;
    float g = grain(texCoord, u_resolution / grainSize);
    vec3 color = vec3(g);
    gl_FragColor = vec4(color, 1.0);
}
```

This generates a more textured noise pattern, and we can increase the grain size with this dial.

<Dither step="grain-3d-noise" />

Using this algorithm on our photos from before with an increased grain size, gives us a more natural film grain look. But we still run into issues when we increase the **mix**, since it takes over the image. Worse still, this is different for different images - so we'd have to set an amount for each image manually.

<Dither step="grain-3d" />

Instead of using the GLSL `mix` function we can use a **blend-soft-light** function (this is the blend mode mentioned above) with the texel luminance. The below code is from the [blending tips section](https://github.com/mattdesl/glsl-film-grain/tree/master#blending-tips) on the glsl-film-grain repo.

```glsl
vec3 blendSoftLight(vec3 base, vec3 blend) {
  return mix(
    sqrt(base) * (2.0 * blend - 1.0) + 2.0 * base * (1.0 - blend), 
    2.0 * base * blend + base * base * (1.0 - 2.0 * blend), 
    step(base, vec3(0.5))
  );
}

void main() {
  vec3 g = vec3(grain(texCoord, p));

  //blend the noise over the background, 
  //i.e. overlay, soft light, additive
  vec3 color = blend(backgroundColor, g);

  //get the luminance of the background
  float luminance = luma(backgroundColor);

  //reduce the noise based on some 
  //threshold of the background luminance
  float response = smoothstep(0.05, 0.5, luminance);
  color = mix(color, backgroundColor, pow(response, 2.0));

  //final color
  gl_FragColor = vec4(color, 1.0);
}
```

<Dither step="grain-3d-blend" />

In the [next article](/p/webgl-duotone) of the series we'll look at using a gradient map to convert our image into a duotone image.

<ArticlePreview slug="webgl-duotone" />



<WebGLImageProcessingIntro />

Now that we know how to import and render our image, let’s look at how we can adjust it with some basic color correction algorithms such as brightness, contrast, exposure and saturation.

## Brightness

To increase the brightness of our image we'll want to increase our color value by adding the same amount to each RGB value.

```glsl
vec4 brightness(vec4 color, float value) {
  // Update only rgb values
  vec3 rgb = color.rgb + value;
  return vec4(rgb, color.a);
}
```

<WebGLFilters step="Brightness" />

## Exposure

To update the exposure, we want to _multiply_ each RGB value by the same amount. This changes the brightness of our image by its current luminosity (the strength of the individual texel). Since multiplying a zero value outputs another zero, we need to increase our original value by 1.0.

```glsl
vec4 exposure(vec4 color, float value) {
  // Update only rgb values
  vec3 rgb = (1.0 + color.rgb) * value;
  return vec4(rgb, color.a);
}
```

<WebGLFilters step="Exposure" />

## Contrast

This is the contrast between the highs and lows of your image. So to increase the contrast we need to increase the range between the higher and lower values. We increase values over 0.5, while decreasing values under 0.5.

```glsl
vec4 contrast(vec4 color, float value) {
  // Update only rgb values
  vec3 rgb = 0.5 + (1.0 + value) * (color.rgb - 0.5);
  return vec4(rgb, color.a);
}
```

<WebGLFilters step="Contrast" />

## Saturation

Saturation alters the vibrance of colors in an image. The more saturated an image is, the more colorful it appears.

When we increase the saturation, we increase the contrast between the RGB channels of a texel. Meaning a fully saturated texel would be one of 3 values `vec4(1, 0, 0, 1)` ,  `vec3(0, 1, 0, 1)`  or  `vec4(0, 0, 1, 1)`.

To create a desaturated (black-and-white or grayscale) image, we can take the average of the channels.

```glsl
vec4 desaturated(vec4 color) {
  vec4 luminance = vec4(color.r + color.g + color.b / 3.0);
  return vec4(vec3(luminance), 1.0);
}
```

But the human eye doesn't perceive the luminosity of each color channel in a way that matches the WebGL value range. As a result, we need to adapt our RGB values by a factor that has been set out by the Web Content Accessibility Guidelines (WCAG) section on [relative luminance](https://www.w3.org/TR/WCAG21/#dfn-relative-luminance).


```glsl
vec4 desaturated(vec4 color) {
  // sRGB colorspace luminosity factor
  vec3 luma = vec3(0.2126, 0.7152, 0.0722);
  vec3 rgb = vec3(dot(color, luma));
  return vec4(rgb, color.a);
}

vec4 saturation(vec4 color, float value) {
  vec4 desaturated = desaturated(color);
  return mix(desaturated, color, 1.0 + value);
}
```

<WebGLFilters step="Saturation" />

## Let's try a blend 

<WebGLFilters step="Article1" />

With these four functions, we can start to edit our image dramatically. They give us the power to fix a lot of issues that can arise when taking photos.

But we're just getting started with what we can do with GLSL. In the [next article](/p/webgl-hue) we'll take a look at how to we can morph the colors with hue adjustments.

<ArticlePreview slug="webgl-hue" />



In this series of articles we’re going to look at image processing with the **GPU**. We’ll use **WebGL** (Web Graphics Library) and **GLSL** (OpenGL Shading Language) shaders for this, which will enable interactive examples throughout the series.

While the theory behind image processing is quite math-heavy, this series will focus on the code for producing these effects and the theory behind them.

We’ll start off with a basic introduction to WebGL and GLSL, and then move onto simple color correction algorithms that you’d find in software such as Lightroom, VSCO or Instagram. We will then take a tour through numerous filters and effects that would be at home in Photoshop.

## Articles in this series:

- Intro to WebGL (this article)
- Color Correction
- Hue
- Film Grain
- Duotone
- Pixelation
- Thresholding
- Dithering
- Vignette
- Chromatic Aberration

<br />

Here's a preview of what we'll build over the course of these articles. You can also view the [full screen version here](/webgl-filters-and-effects).

<iframe title="WebGL Photo - filters & effects" src="/webgl-filters-and-effects?header=false&preview=true" />

## Basic WebGL concepts

WebGL is a JavaScript API for rendering high-performance, interactive, 3D and 2D graphics in the browser. It uses your **GPU** to compute your programs in parallel rather than the linear functionality of the **CPU**.

WebGL enables us to **send data to, and run programs (shaders) on, the GPU**. This series will focus on a type of shader program known as a fragment shader. A fragment shader will run once for each pixel in the rendered image. These shader programs run in parallel so you can't have any shared state between them.

Shader programs are written in a C-inspired language called GLSL (OpenGL Shading Language). Graphics cards are extremely fast in comparison to the CPU, and have many mathematical functions built in that GLSL enables us to take advantage of from our shader programs.

We pass data to our fragment shader as a **Uniform**. A Uniform is a variable that is the same for every fragment shader in a render pass. As this series will focus on editing images, we'll pass this into our shader program as a special kind of Uniform known as a **Texture**. Textures are WebGL's built in support for images. It stores the image data as a 2x2 matrix (row x column) of RGBA (red, green, blue, alpha) color values.

## A brief recap:
- **Shader**: a program that runs on the GPU, as opposed to the CPU
- **Fragment shader**: a program (on the GPU) that runs for every pixel in your image in parallel
- **Uniform**: a variable that is the same for every fragment shader in a render pass
- **Texture**: WebGL’s built-in support for images

## GLSL colors

Colors in GLSL are a 4-value vector, containing the red, green, blue, alpha channels. These map to the RGBA values you know in CSS, but are 0.0 to 1.0 instead of 0 to 255.
To get a feel for how these relate to each other, have a play around with the values below using your arrow keys:

<ColorVec4 color={[255, 0, 0, 255]} />

## Our foundational shader

The foundations of our shader lie in a few input `uniform`s and one color value `output`. Let's have a quick look, then I'll break it down.

```glsl
#version 300 es
precision highp float;

uniform vec2 u_resolution;
uniform sampler2D u_image;

out vec4 outColor;

void main() {
  vec2 uv = gl_FragCoord.xy/u_resolution;

  vec4 texel = texture(u_image, uv);
  outColor = texel; 
}
```

Starting from the top, we have `#version 300 es`. This tells WebGL which version of GLSL our shader is written in. This is quite important as there are many breaking changes between version `100` and `300`. 

The `precision` value tells our program how many decimals a floating point can contain. The more decimals, the more data can be stored from a calculation. 

We pass our image in as a uniform `u_image`, which we can read with GLSL's `texture` function. We also need the resolution (width and height) of our canvas to correctly map our image data to our canvas size, passing this into the `texture` function.

The `texture` function outputs a `texel`: a texture element rather than a pixel. We can then assign this to our `outColor`.

<WebGLFilters step="Intro" />

This program doesn't do much at the moment. It simply outputs the image we pass in, as we can see above. In the [next article](/p/webgl-color-correction) we’ll expand on this basic program to support color correction methods such as brightness, exposure and saturation.

<ArticlePreview slug="webgl-color-correction" />


I’m a Creative Engineer interested in the intersection of Math, Comp Sci and Design.

Currently exploring the world of image manipulation in WebGL.

WebGL Image Manipulation

Work

WebGL Photo Filters

Nmblr

Cables

Open Type Collective

InMySize

Experiments