libvips can process HDR images encoded with UltraHDR. These are ordinary SDR images, but with a gainmap embedded within them — SDR displays will just show the regular SDR image, but for an HDR display, the extra gainmap can be used as an exponent to recover the full HDR range. This ability to show the same file in good quality on both SDR and HDR displays makes the format very useful.

libvips uses Google’s libultrahdr for UltraHDR load and save. The current version of this library only supports UltraHDR JPEG images; the next version is expected to add support for a wider range of image formats.

There are two main paths for UltraHDR images in libvips: as an SDR image with a separate gainmap, and as a full HDR image. The separate gainmap path is relatively fast but you will sometimes need to update the gainmap during processing. The full HDR path does not require gainmap updates, but can be slower, and will usually lose the original image’s tone mapping.

UltraHDR as SDR with a separate gainmap

libvips will detect JPEG images with an embedded gainmap and automatically invoke the vips_uhdrload() operation to load them. This operation attaches the gainmap (a small JPEG-compressed image) as the "gainmap-data" metadata item, plus some other gainmap tags.

Load and save

For example:

$ vipsheader -a ultra-hdr.jpg
ultra-hdr.jpg: 3840x2160 uchar, 3 bands, srgb, uhdrload
width: 3840
height: 2160
bands: 3
format: uchar
coding: none
interpretation: srgb
xoffset: 0
yoffset: 0
xres: 1
yres: 1
filename: ultra-hdr.jpg
vips-loader: uhdrload
icc-profile-data: 588 bytes of binary data
gainmap-data: 31738 bytes of binary data
gainmap-max-content-boost: 100 100 100
gainmap-min-content-boost: 1 1 1
gainmap-gamma: 1 1 1
gainmap-offset-sdr: 0 0 0
gainmap-offset-hdr: 0 0 0
gainmap-hdr-capacity-min: 1
gainmap-hdr-capacity-max: 100
gainmap-use-base-cg: 1

This gainmap metadata is copied unmodified through any processing operations. If you save an image with gainmap metadata to a JPEG file, libvips will do the write with the vips_uhdrsave() operation, embedding the gainmap and the associated metadata in the output image.

Intermediate operations which change the image geometry will also need to update the "gainmap-data" metadata item, the mechanisms for doing this are described below. The other gainmap fields should probably not be changed unless the intention is to alter the image appearance.

High-level libvips operations

Two high-level libvips operations will automatically update the gainmap for you during processing: vips_dzsave() and vips_thumbnail().

vips_dzsave() always strips all metadata by default, so you’ll need to set keep="gainmap" to write the gainmap to the tiles. For example:

$ vips dzsave ultra-hdr.jpg x --keep gainmap

À la carte processing

Other operations will NOT automatically update the gainmap for you. If you call something like vips_crop(), an operation which changes the image geometry, the gainmap and the image will no longer match. When you save the cropped image, the gainmap is likely to be incorrect.

A helper function, vips_image_get_gainmap(), makes updating the gainmap relatively easy: it returns a VipsImage for the gainmap, and attaches the image pointer as the metadata item "gainmap". Once you have updated the gainmap, you can overwrite this value.

For example, in C you could write

VipsImage *image = ...;
VipsImage *out;
int left, top, width, height;
if (vips_crop(image, &out, left, top, width, height, NULL))
    return -1;

// also crop the gainmap, if there is one
VipsImage *gainmap;
if ((gainmap = vips_image_get_gainmap(out))) {
    // the gainmap can be smaller than the image, we must scale the
    // crop area
    double hscale = (double) gainmap->Xsize / image->Xsize;
    double vscale = (double) gainmap->Ysize / image->Ysize;

    VipsImage *x;
    if (vips_crop(gainmap, &x, left * hscale, top * vscale,
        width * hscale, height * vscale, NULL))
        return -1;
    g_object_unref(gainmap);

    // vips_image_set_image() modifies the image, so we need to make a
    // unique copy ... you can skip this step if you know your image is
    // already unique
    VipsImage *x2;
    if (vips_copy(out, &x2, NULL))
        return -1;
    g_object_unref(out);
    out = x2;

    // update the gainmap
    vips_image_set_image(out, "gainmap", x);

    g_object_unref(x);
}

Performance and quality considerations

Doing gainmap processing explicitly like this has two big advantages: first, you have exact control over this processing, so you can make sure only the gainmap transformations that are strictly necessary take place. Secondly, since you supply the gainmap to the UltraHDR save, you can also be certain any user tone mapping is preserved.

The disadvantage is the extra development work necessary. The second UltraHDR path in libvips avoids this problem.

Full HDR processing

You can transform an UltraHDR SDR plus gainmap image to full HDR with vips_uhdr2scRGB(). This will compute an scRGB image: a three-band float with sRGB primaries, black to white as linear 0-1, and out of range values used to represent HDR.

For example:

$ vips uhdr2scRGB ultra-hdr.jpg x.v
$ vipsheader x.v
x.v: 3840x2160 float, 3 bands, scrgb, uhdrload
$ vips max x.v
15.210938

If you save an scRGB image as JPEG, it will be automatically written as UltraJPEG. Any associated gainmap is thrown away and basic tonemapping performed to make a new gainmap for SDR display.

Full HDR processing with scRGB is simple, but potentially slower than the separate gainmap path, and will not preserve any user tone map.