Jekyll2023-11-11T14:09:17+00:00https://www.libvips.org/feed.xmllibvipsA fast image processing library with low memory needs.What’s new in libvips 8.152023-10-10T00:00:00+00:002023-10-10T00:00:00+00:00https://www.libvips.org/2023/10/10/What's-new-in-8.15<p>libvips 8.15 is now done, so here’s a summary of what’s new. Check the
<a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
if you need more details.</p>
<p>The headline features are SIMD optimisations with Highway, some useful
speedups to <code class="language-plaintext highlighter-rouge">dzsave</code> and TIFF save, metadata improvements, and the usual
small improvements to image format support. Details below!</p>
<p>We need to thank lovell, kleisauke, miltoncandelero, MathemanFlo,
donghuikugou, jcupitt, DarthSim, shado23, a3mar, and others for their great
work on this release.</p>
<h1 id="performance-improvements">Performance improvements</h1>
<p>Traditionally, libvips has relied on <a href="https://gitlab.freedesktop.org/gstreamer/orc">liborc’s runtime compiler</a>
to dynamically generate optimised SIMD/vector code specifically for the target
architecture. However, maintaining this code proved challenging, and it didn’t
generalize to architectures like <a href="/2020/09/01/libvips-for-webassembly.html#performance">WebAssembly</a>.
Additionally, it lacked support for newer instruction sets like AVX2 and
AVX-512, and the vector paths of liborc didn’t match the precision of the C
paths.</p>
<p>In 8.15, we’ve used <a href="https://github.com/google/highway">Highway</a> to
reimplement vector paths in many operations. Highway is a C++ library with
carefully-chosen functions that map well to CPU instructions without extensive
compiler transformations. Highway supports five architectures, allowing our
code to target various instruction sets, including those with ‘scalable’
vectors (size unknown at compile time). At runtime, dynamic dispatch selects
the best available implementation based on the processor’s capabilities,
ensuring optimal performance. While Highway is our preferred choice, the
liborc paths remain available as a fallback whenever Highway is unavailable.</p>
<p>Additionally, for x86/x86-64 targets, a couple of functions are now marked
with the <code class="language-plaintext highlighter-rouge">target_clones</code> attribute to improve performance on AVX CPUs by ~10%.</p>
<p>An improvement to the shrink operators has allowed a dramatic speedup in
edge cases where there was upstream coordinate transformation. Performance
should now be more predictable in these cases.</p>
<h1 id="general-image-load-and-save-improvements">General image load and save improvements</h1>
<p>There are two improvements to all loaders and savers. First, you can pass a
<code class="language-plaintext highlighter-rouge">revalidate</code> flag to all loaders which will make them bypass the libvips cache
and refetch the image from the source. This is useful if you are loading a file
where the contents might change.</p>
<p>Secondly, we’ve deprecated the <code class="language-plaintext highlighter-rouge">strip</code> option to savers and added a new <code class="language-plaintext highlighter-rouge">keep</code>
option which you can use to select what classes of metadata item you’d like to
<em>not</em> remove. For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips copy k2.tif x.jpg[keep=icc:exif]
</code></pre></div></div>
<p>Will copy a JPEG image, keep any ICC profile and EXIF metadata, but delete
everything else, such as XMP and IPTC. Use <code class="language-plaintext highlighter-rouge">keep=none</code> to remove everything.
This new colon syntax for flag options works everywhere, including in language
bindings.</p>
<p>We’ve added a new <code class="language-plaintext highlighter-rouge">bitdepth</code> metadata item which all loaders and savers
now support, and deprecated the old <code class="language-plaintext highlighter-rouge">palette-bit-depth</code> and <code class="language-plaintext highlighter-rouge">heif-bitdepth</code>
fields.</p>
<h2 id="better-tiffsave">Better <code class="language-plaintext highlighter-rouge">tiffsave</code></h2>
<p>libvips used to rely on libtiff to manage write of compressed tiles. This
meant that the selected compression library (libjpeg, perhaps) would run
inside libtiff, and was therefore single-threaded.</p>
<p>In libvips 8.15, we’ve taken over control of JPEG and JPEG2000 compression
and we now do this ourselves in parallel, leaving only the raw tile
write to libtiff.</p>
<p>When making a JPEG-compressed TIFF with libvips 8.14 I see:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips copy CMU-1.svs[rgb] x.tif[compression=jpeg,tile,pyramid]
real 0m11.732s
user 1m8.123s
sys 0m5.810s
</code></pre></div></div>
<p>But with 8.15 it’s now:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips copy CMU-1.svs[rgb] x.tif[compression=jpeg,tile,pyramid]
real 0m5.332s
user 1m2.410s
sys 0m7.543s
</code></pre></div></div>
<p>More than twice as fast.</p>
<p>We’ve also added support for load and save of 16-bit float TIFFs, and improved
the compatibility of 32-bit float TIFFs.</p>
<h2 id="better-dzsave">Better <code class="language-plaintext highlighter-rouge">dzsave</code></h2>
<p>libvips used to save each tile in <code class="language-plaintext highlighter-rouge">dzsave</code> by building and executing a complete
libvips pipeline. With small tiles, the setup and teardown cost
was often a significant part of the runtime, and this limited speed.</p>
<p>With libvips 8.15 we’ve added a direct path for JPEG tiles (the default case)
which avoids this overhead.</p>
<p>With libvips 8.14 I saw:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips dzsave CMU-1.svs[rgb] x
real 0m14.435s
user 1m26.434s
sys 0m50.474s
</code></pre></div></div>
<p>With 8.15 it’s now:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips dzsave CMU-1.svs[rgb] x
real 0m5.586s
user 1m4.462s
sys 0m21.581s
</code></pre></div></div>
<p>Nearly 3x faster. There’s a new <code class="language-plaintext highlighter-rouge">--Q</code> flag to set the Q factor for direct JPEG
tile save.</p>
<p>We’ve made another improvement to <code class="language-plaintext highlighter-rouge">dzsave</code>: it now uses a better ZIP write
library, <code class="language-plaintext highlighter-rouge">libarchive</code>, which should improve portability.</p>
<h1 id="general-improvements">General improvements</h1>
<ul>
<li>PDF loading with PDFium now includes support for PDF forms, making it
capable of rendering user-provided input in checkboxes and text fields.</li>
<li>We’ve added two new edge detectors,
<a href="/API/current/libvips-convolution.html#vips-scharr"><code class="language-plaintext highlighter-rouge">vips_scharr()</code></a> and
<a href="/API/current/libvips-convolution.html#vips-prewitt"><code class="language-plaintext highlighter-rouge">vips_prewitt()</code></a>,
and improved the accuracy of
<a href="/API/current/libvips-convolution.html#vips-sobel"><code class="language-plaintext highlighter-rouge">vips_sobel()</code></a>.</li>
<li><a href="/API/current/libvips-arithmetic.html#vips-find-trim"><code class="language-plaintext highlighter-rouge">vips_find_trim()</code></a>
features a <code class="language-plaintext highlighter-rouge">line_art</code> option.</li>
<li>GIF load now sets <code class="language-plaintext highlighter-rouge">interlaced=1</code> for interlaced GIF images.</li>
<li>Improved C++ binding, taking advantage of C++11 features.</li>
<li>The built-in ICC profiles have been replaced with smaller and more accurate
ICC v4 variants.</li>
</ul>libvips 8.15 is now done, so here’s a summary of what’s new. Check the ChangeLog if you need more details.vipsdisp 2.5 is out2023-07-23T00:00:00+00:002023-07-23T00:00:00+00:00https://www.libvips.org/2023/07/23/vipsdisp-2.5<p>There’s a new version of <a href="https://github.com/jcupitt/vipsdisp">vipsdisp</a>,
a gtk4 image viewer.</p>
<h2 id="another-image-viewer">Another image viewer??</h2>
<p>It can display huge (eg. 100,000 x 100,000 pixel and more) images quickly
and without using much memory. It supports many scientific and technical
image formats, including TIFF (with some OME-TIFF support), WEBP, JP2, JXL,
HEIC, AVIF, PNG, JPEG, SVS, MRXS, NDPI, OpenEXR, GIF, PDF, SVG, FITS, HDR
(Radiance), Matlab, NIfTI, Analyze, CSV, PPM, PFM, etc. It supports many
pixel types, from 1 bit mono to 128-bit double precision complex.</p>
<h2 id="whats-new">What’s new</h2>
<p>Version 2.5 adds save-as (contributed by angstyloop), better SVG and PDF
support, improved mousewheel handling, selectable image backgrounds, and
faster and smoother image animation.</p>
<p>Save-as supports all the image write features of libvips, so (for example),
you can save any image as a DeepZoom pyramid, or as tiled pyramidal
JPEG-compressed TIFF, or as JXL.</p>
<h2 id="install">Install</h2>
<p>There’s a <a href="https://flathub.org/apps/details/org.libvips.vipsdisp">binary on
flathub</a>,
so it should be easy to try out. And there’s also a <a href="https://github.com/jcupitt/homebrew-core/commit/faae7e9568f49508316a60d6f70660b9f35b54a6">homebrew
formula</a>,
which works surprisingly well.</p>
<h2 id="technical-background">Technical background</h2>
<p>It has some interesting features:</p>
<ul>
<li>
<p>It doesn’t need to keep the whole image in memory. When possible, it will
only decompress the pixels that it needs for display, and it understands most
pyramidal image formats. This means you can open and view huge images quickly.</p>
</li>
<li>
<p>It has threaded, asynchronous image repaint, so image tiles are computed
in the background by a pool of workers and the screen is updated as they
are finished. The interface should stay live even under very heavy load.</p>
</li>
<li>
<p>It keeps a sparse pyramid of tiles as textures on the GPU. For each frame,
it computes the set of visible tiles, and then the GPU scales, positions
and composites just those tiles to the screen. CPU load should be low
(except for the background workers haha). Hold down i (for “in”) or +
to do a smooth zoom on the cursor. If you press “d” it toggles a debug
display mode which shows the tiles being computed.</p>
</li>
<li>
<p>Select Display Control Bar from the top-right menu and a useful set of
visualization options appear. It supports four main display modes: Toilet
Roll (sorry), Multipage, Animated, and Pages as Bands.</p>
</li>
<li>
<p>In Toilet Roll mode, a multi-page image is presented as a tall, thin strip
of images. In Multipage, you see a single page at a time, with a page-select
spinner (you can also use the crtl-< and ctrl-> keys to flip pages). In
animated mode, pages flip automatically on a timeout. In pages-as-bands
mode, many-page single-band images (eg. OME-TIFF) are presented as a single
colour image.</p>
</li>
<li>
<p>You can select falsecolour and log-scale filters, useful for many scientific
images. Scale and offset sliders let you adjust image brightness to see into
darker areas (useful for HDR and science images). Pick Scale to search the
window for min and max and set the scale and offset sliders to fill the range.</p>
</li>
</ul>There’s a new version of vipsdisp, a gtk4 image viewer.Charity bike ride2023-04-06T00:00:00+00:002023-04-06T00:00:00+00:00https://www.libvips.org/2023/04/06/Charity-bike-ride<p>This isn’t libvips at all, but I’ve entered a charity bike ride:</p>
<p>https://2023ridelondon.enthuse.com/pf/john-cupitt</p>
<p>It’s 100 miles from central London to Braintree and back on May 28.</p>
<p>The ride is in aid of
<a href="https://www.streetchildren.org/what-we-do/">streetchildren.org</a> – they
campaign to give street children across the world a voice, to be treated
equally, and to have access to healthcare and education. </p>
<p>It’s a really worthwhile cause, and I’d love to raise some money for them if I
can. Any sponsorship will be very gratefully received!</p>This isn’t libvips at all, but I’ve entered a charity bike ride:What’s new in libvips 8.142022-12-22T00:00:00+00:002022-12-22T00:00:00+00:00https://www.libvips.org/2022/12/22/What's-new-in-8.14<p>libvips 8.14 is now done, so here’s a summary of what’s new. Check the
<a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
if you need more details.</p>
<p>The headline features are the final switch to meson build system, a new
thread pool and thread recycling system, some useful speedups to <code class="language-plaintext highlighter-rouge">dzsave</code>,
<code class="language-plaintext highlighter-rouge">arrayjoin</code> and TIFF load, and the usual small improvements to image format
support. Details below!</p>
<p>We need to thank aksdb, dloebl, ewelot, tlsa, remicollet, DarthSim,
ejoebstl, lovell, shado23, kleisauke, and others for their great work on
this release.</p>
<h1 id="build-system-changes">Build system changes</h1>
<p>The previous release added a meson build system alongside the autotools build
system we’d used for decades. This release removes the autotools system
completely, and a lot of other old stuff too, so it’s now meson all the way
down.</p>
<p>The README has some notes to help you get started if you need to build from
source, but the quick summary is something like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd libvips-x.y.x
meson setup build --prefix /my/install/prefix
cd build
meson compile
meson test
meson install
</code></pre></div></div>
<h1 id="n-colour-profile-support">N-colour profile support</h1>
<p>Printers usually work with four colours: cyan, magenta, yellow and black.
libvips has supported (via <a href="https://github.com/mm2/Little-CMS">LittleCMS</a>)
conversion to and from CMYK images for a long time.</p>
<p>In 8.14 we’ve added support for N-colour profiles. These profiles add extra
colours, usually the complements of CMY, to expand the colour gamut available
and reduce dithering effects. The <code class="language-plaintext highlighter-rouge">icc_transform</code> set of operators now
support rendering to and from N-colour device images in the obvious way.</p>
<p>The TIFF saver also knows how to read and write these CMYRGK (for example)
files, and hopefully does it in a way that’s compatible with PhotoShop.</p>
<p>With <a href="https://github.com/libvips/libvips/files/9550789/6clr-test.icc.zip">this sample 6CLR
profile</a>
I can run:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips icc_export nina.jpg x.tif --output-profile 6clr-test.icc
</code></pre></div></div>
<p>To make an image split to 6 colour separations:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ tiffinfo x.tif
=== TIFF directory 0 ===
TIFF Directory at offset 0x8b89008 (146313224)
Image Width: 6048 Image Length: 4032
Resolution: 300, 300 pixels/inch
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: None
Photometric Interpretation: separated
Extra Samples: 2<unassoc-alpha, unassoc-alpha>
Orientation: row 0 top, col 0 lhs
Samples/Pixel: 6
Rows/Strip: 128
Planar Configuration: single image plane
InkSet: 1
ICC Profile: <present>, 13739188 bytes
</code></pre></div></div>
<p>We’ve also added support for character as well as word wrapping
to <code class="language-plaintext highlighter-rouge">vips_text()</code> with the <code class="language-plaintext highlighter-rouge">wrap</code> parameter, added an <code class="language-plaintext highlighter-rouge">rgb</code> mode to
<code class="language-plaintext highlighter-rouge">vips_openslideload()</code>, and <code class="language-plaintext highlighter-rouge">vips_smartcrop()</code> now optionally returns the
location of interest in attention-based cropping.</p>
<h1 id="improvements-to-the-libvips-core">Improvements to the libvips core</h1>
<h2 id="faster-threading">Faster threading</h2>
<p>libvips uses threadpools to compute pixels – these are groups of worker
threads cooperating on a task. For 8.14, we’ve rewritten the threadpool
system and added several very useful new features:</p>
<ol>
<li>
<p>Threadpools now resize dynamically. Each threadpool is able to tell how
busy the workers are, and is able to either expand or shrink
depending on load. The old <code class="language-plaintext highlighter-rouge">vips-concurrency</code> setting now sets the maximum
threadpool size.</p>
<p>The aim is to improve utilisation on machines with many cores. Why create
16 workers, for example, for a pipeline that only has a small amount
of parallelism?</p>
<p>Few idle threads means libvips should make better use of hardware resources
on large machines with complex mixed workloads. The new threadpool should
also be a bit quicker.</p>
</li>
<li>
<p>You can also set hints for the amount of parallelism you expect in a
pipeline. Again, this helps prevent overcomitting of thread resources.</p>
</li>
<li>
<p>Finally, there’s a new thread recycling system. Some platforms have very
slow or tricky thread start and stop, so rather than killing and recreating
threads all the time, libvips will make a set of threads and then recycle
them between threadpools.</p>
<p>We had a thread recycling system before, but this new one should be
noticably simpler and faster.</p>
</li>
</ol>
<h2 id="faster-dzsave-and-arrayjoin">Faster <code class="language-plaintext highlighter-rouge">dzsave</code> and <code class="language-plaintext highlighter-rouge">arrayjoin</code></h2>
<p>We’ve used this new threading system to revise <code class="language-plaintext highlighter-rouge">dzsave</code>, and it’s now quite a
bit quicker. Here’s the previous release, libvips 8.13, running on a
46,000 x 32,914 pixel slide image:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e vips dzsave CMU-1.svs x
881892:36.65
</code></pre></div></div>
<p>That’s 900mb of peak memory and 37s of run time. Here’s libvips 8.14:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e vips dzsave CMU-1.svs x
704360:19.50
</code></pre></div></div>
<p>Almost twice as fast, and noticably lower memory use. This all comes from the
new threading system.</p>
<p>This new release has another feature which can improve slide read
performance: the <code class="language-plaintext highlighter-rouge">rgb</code> flag to openslideload. This drops the redundant alpha
plane earlier, saving time and memory:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e vips dzsave CMU-1.svs[rgb] x
547832:13.02
</code></pre></div></div>
<p>Now it’s three times faster than 8.13 and runs in about half the memory.</p>
<p>The matching <code class="language-plaintext highlighter-rouge">arrayjoin</code> operator has been reworked and is now faster
for large sets of images. Here’s libvips 8.13 reassembling 10,000 tiles:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips dzsave CMU-1.svs x --depth one --tile-width 400 --tile-height 400 --overlap 0 --suffix .jpg
$ cd x_files/0/
$ ls | wc
9545 9545 94715
$ time vips arrayjoin "$(ls *.jpg | sort -t_ -k2g -k1g)" x.jpg --across 115
real 0m17.357s
user 0m37.003s
sys 0m44.812s
</code></pre></div></div>
<p>And here’s 8.14:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips arrayjoin "$(ls *.jpg | sort -t_ -k2g -k1g)" x.jpg --across 115
real 0m11.883s
user 0m47.085s
sys 0m46.906s
</code></pre></div></div>
<p>The speedup becomes dramatic with larger tile sets.</p>
<h2 id="bash-completions">bash completions</h2>
<p>This won’t appeal to many people, but libvips now ships with a simple
bash completion script in <code class="language-plaintext highlighter-rouge">libvips-x.y.z/completions</code>, have a look at the
README in there for install notes. It knows how to complete operator names,
filenames, and required arguments, including enum values.</p>
<p>It’s useful now, and we hope to improve it in the next version, perhaps by
expanding optional arguments, for example.</p>
<h1 id="image-format-improvements">Image format improvements</h1>
<p>There have been quite a few improvements to file format support.</p>
<h2 id="tiff"><strong>TIFF</strong></h2>
<p>Previous releases used libtiff to fetch decoded tiles from compressed
TIFF files. This ran the decompressor inside the libtiff lock, so it was
single threaded.</p>
<p>For libvips 8.14, we’ve moved jpeg2000 and jpeg decompression outside the
libtiff lock so they now run multi-threaded. This can give a really nice
speedup.</p>
<p>First, make a large, tiled, JPEG-compressed TIFF:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips copy CMU-1.svs[rgb] x.tif[tile,compression=jpeg]
</code></pre></div></div>
<p>Then read the file and compute the pixel average. Here’s the previous 8.13
release:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips avg x.tif
226.581443
real 0m42.776s
user 0m48.380s
sys 0m0.428s
</code></pre></div></div>
<p>You can see that the total CPU time (the <code class="language-plaintext highlighter-rouge">user</code> line) is almost equal to the
real clock time (the <code class="language-plaintext highlighter-rouge">real</code> line), so there was very little parallelism. It
was able to parallelize the computation of the average value, but the
decompress (which was almost all of the run time) was single threaded.</p>
<p>Here’s 8.14:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips avg x.tif
226.581443
real 0m3.371s
user 0m17.413s
sys 0m1.573s
</code></pre></div></div>
<p>Now tiles are decompressed in parallel and on this 16-core PC there’s a
huge speedup, more than 10x. Zoom!</p>
<p>This is just accelerating TIFF load. Perhaps TIFF save will get the same
treatment in the next version.</p>
<h2 id="gif"><strong>GIF</strong></h2>
<p>GIF save has a new <code class="language-plaintext highlighter-rouge">interlace</code> option, and the default save behaviour has
changed: for safety it now always recomputes the palette. If you want to
reuse the input palette, set the new <code class="language-plaintext highlighter-rouge">reuse</code> flag.</p>
<p>GIF load now handles truncated files much more gracefully.</p>
<h2 id="heicavif"><strong>HEIC/AVIF</strong></h2>
<p>The saver has a new <code class="language-plaintext highlighter-rouge">encoder</code> parameter you can use to select
the exact save codec library you want.</p>
<h2 id="fits"><strong>FITS</strong></h2>
<p>The FITS load and save operations have been rewritten. Many band
FITS images should load dramatically faster, and FITS save is much better at
handling duplicated header fields.</p>
<h2 id="other">Other</h2>
<p>PNG load and save operations now support EXIF metadata. Animated webp save
has been rewritten and should perform much better. Saving as <code class="language-plaintext highlighter-rouge">.pnm</code> will
now pick the bext <code class="language-plaintext highlighter-rouge">p*m</code> subformat for you automatically. The jp2k saver
now defaults to chroma subsample off for better compatibility, and writes
<code class="language-plaintext highlighter-rouge">jp2</code> images rather than a simple codestream.</p>
<p>Plus the usual range of small improvements and bugfixes. See the ChangeLog.</p>libvips 8.14 is now done, so here’s a summary of what’s new. Check the ChangeLog if you need more details.What’s new in libvips 8.132022-05-28T00:00:00+00:002022-05-28T00:00:00+00:00https://www.libvips.org/2022/05/28/What's-new-in-8.13<p>libvips 8.13 is now done, so here’s a summary of what’s new. Check the
<a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
if you need more details.</p>
<p>The headline features are a new system for blocking unsafe file formats,
much better GIF handling, and a new build system. Details below!</p>
<p>Many thanks to remicollet, DarthSim, GavinJoyce, tintou, lovell, shado23,
dloebl, tlsa, kleisauke, f1ac, richardmonette and others for their great
work on this release.</p>
<h1 id="blocking-of-unfuzzed-loaders">Blocking of unfuzzed loaders</h1>
<p>libvips support many image format libraries. Some of these are well tested
against malicious input files, but some are not.</p>
<p>If you were developing a web service that used libvips to handle
untrusted image files, our advice used to be to build your own libvips
binary that only had support for the file types you wanted to handle
(this is what projects like <a href="https://sharp.pixelplumbing.com">sharp</a> and
<a href="https://kleisauke.github.io/net-vips/">NetVips</a> do). This was safe, but
also hard work — it could make deployment significantly more complex for
some users.</p>
<p>libvips 8.13 has a new feature which can block untrusted operations at
runtime, and at a very low level. This means you can use any libvips binary
and be confident that unfuzzed code is not being exposed to internet data.</p>
<p>If the environment variable <code class="language-plaintext highlighter-rouge">VIPS_BLOCK_UNTRUSTED</code> is set, then any operation
that we’ve tagged as untrusted will be prevented from running. This should
be very simple to add to existing projects.</p>
<p>There’s also an API which gives much finer control. For example, in Python
you can write:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pyvips</span><span class="p">.</span><span class="n">operation_block_set</span><span class="p">(</span><span class="s">"VipsForeignLoad"</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span>
<span class="n">pyvips</span><span class="p">.</span><span class="n">operation_block_set</span><span class="p">(</span><span class="s">"VipsForeignLoadJpeg"</span><span class="p">,</span> <span class="bp">False</span><span class="p">)</span>
</code></pre></div></div>
<p>Now all operations derived from <code class="language-plaintext highlighter-rouge">VipsForeignLoad</code> (so all loaders) are
blocked, but we’ve marked all loaders derived from <code class="language-plaintext highlighter-rouge">VipsForeignLoadJpeg</code>
(so all the libjpeg loaders) as runnable. After these calls, libvips will
only load JPEG files.</p>
<p>See <a href="/API/current/libvips-vips.html#vips-block-untrusted-set"><code class="language-plaintext highlighter-rouge">vips_block_untrusted_set()</code></a> and
<a href="/API/current/VipsOperation.html#vips-operation-block-set"><code class="language-plaintext highlighter-rouge">vips_operation_block_set()</code></a> for more details.</p>
<h1 id="much-better-gif-save">Much better GIF save</h1>
<p>GIF handling has been reworked again, and <code class="language-plaintext highlighter-rouge">gifsave</code> should now produce
smaller files (sometimes much smaller) with lower CPU and memory load.</p>
<p>Here’s a benchmark with a short video clip:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># libvips 8.12</span>
<span class="nv">$ </span>/usr/bin/time <span class="nt">-f</span> %M:%e vipsthumbnail 3198.gif[n<span class="o">=</span><span class="nt">-1</span><span class="o">]</span> <span class="nt">-o</span> vips-12.gif <span class="nt">--size</span> 224
57764:3.96
<span class="c"># libvips 8.13</span>
<span class="nv">$ </span>/usr/bin/time <span class="nt">-f</span> %M:%e vipsthumbnail 3198.gif[n<span class="o">=</span><span class="nt">-1</span><span class="o">]</span> <span class="nt">-o</span> vips-13.gif <span class="nt">--size</span> 224
57344:0.62
<span class="nv">$ </span><span class="nb">ls</span> <span class="nt">-l</span> vips-12.gif vips-13.gif
<span class="nt">-rw-r--r--</span> 1 john john 3441032 Jun 8 16:34 vips-12.gif
<span class="nt">-rw-r--r--</span> 1 john john 2487189 Jun 8 16:25 vips-13.gif
</code></pre></div></div>
<p>So about 7x faster and 30% smaller.</p>
<p>There’s a good improvement against imagemagick too:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>/usr/bin/time <span class="nt">-f</span> %M:%e convert 3198.gif <span class="nt">-resize</span> 75% im.gif
200796:6.44
<span class="nv">$ </span><span class="nb">ls</span> <span class="nt">-l</span> im.gif vips-13.gif
<span class="nt">-rw-rw-r--</span> 1 john john 3176859 Jun 8 16:26 im.gif
</code></pre></div></div>
<p>On this task, compared to imagemagick6, libvips is around 10x faster,
needs 4x less memory, makes GIFs which are 20% smaller, and produces higher
quality output.</p>
<p>The new GIF saver now has quite a few options to control
compression, take <a href="/API/current/VipsForeignSave.html#vips-gifsave">a look at the docs</a>.</p>
<h1 id="image-resize-quality-improvements">Image resize quality improvements</h1>
<p>Image resizing with libvips 8.12 could in some cases lose up to 0.5 pixels
of detail on the image edge due to losses in the handling of intermediate
values. Kleis has reworked the image resize code to retain these extra
pixels in all cases, so image edges are now always complete.</p>
<p>It is possible that this may result in different image sizes compared with
libvips 8.12.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>vips black x.jpg 1600 1000
<span class="c"># libvips 8.12</span>
<span class="nv">$ </span>vipsthumbnail x.jpg <span class="nt">-s</span> 10x
<span class="nv">$ </span>vipsheader tn_x.jpg
tn_x.jpg: 10x7 uchar, 1 band, b-w, jpegload
<span class="c"># libvips 8.13</span>
<span class="nv">$ </span>vipsthumbnail x.jpg <span class="nt">-s</span> 10x
<span class="nv">$ </span>vipsheader tn_x.jpg
tn_x.jpg: 10x6 uchar, 1 band, b-w, jpegload
</code></pre></div></div>
<h1 id="file-format-support-improvements">File format support improvements</h1>
<p>There have been the usual range of minor improvements to file format support.
Briefly:</p>
<h3 id="tiffsave-and-dzsave-to-target"><code class="language-plaintext highlighter-rouge">tiffsave</code> and <code class="language-plaintext highlighter-rouge">dzsave</code> to target</h3>
<p>You can now write DeepZoom and TIFF images to the new libvips target API. This
means you can write these formats directly to pipes (for example)
with no need for intermediate files.</p>
<p>There are some complications due to the way these formats work, and you can
need up to 30% of the image size in memory for extra buffering, but at least
it’s possible now.</p>
<h3 id="libspng-save"><code class="language-plaintext highlighter-rouge">libspng</code> save</h3>
<p>If possible, libvips will now use the
<a href="https://github.com/randy408/libspng"><code class="language-plaintext highlighter-rouge">libspng</code></a> library for PNG write as
well as PNG read. This can produce a useful increase in PNG speed.</p>
<h3 id="heic-and-avif">HEIC and AVIF</h3>
<p>These now support HDR images, and the loader has a new option to disable
DoS attack limits.</p>
<h3 id="jxl">JXL</h3>
<p>JXL now supports ICC profiles, and has better support for HDR images.</p>
<h3 id="other-loader-improvements">Other loader improvements</h3>
<ul>
<li>add <code class="language-plaintext highlighter-rouge">password</code> option to <code class="language-plaintext highlighter-rouge">pdfload</code>, fix byte ordering of <code class="language-plaintext highlighter-rouge">background</code></li>
<li>add <code class="language-plaintext highlighter-rouge">mixed</code> to <code class="language-plaintext highlighter-rouge">webpsave</code> [dloebl]</li>
<li>add <code class="language-plaintext highlighter-rouge">bitdepth</code> to <code class="language-plaintext highlighter-rouge">magicksave</code> [dloebl]</li>
<li><code class="language-plaintext highlighter-rouge">jp2kload</code> load left-justifies bitdepth</li>
<li>add <code class="language-plaintext highlighter-rouge">fail-on</code> to <code class="language-plaintext highlighter-rouge">thumbnail</code></li>
</ul>
<h1 id="new-meson-build-system">New <a href="https://mesonbuild.com">meson</a> build system</h1>
<p>libvips has been using <a href="https://www.gnu.org/software/automake/manual/html_node/Autotools-Introduction.html">GNU
autotools</a>
as its build system since the 1990s. It’s worked well for us, but the
world is moving on and a new generation of build systems are trying hard
to displace it.</p>
<p>We’ve settled on <a href="https://mesonbuild.com">meson</a> and tintou has very
generously done all the work. The old autotools system is still there for this
release, but it will be removed for 8.14 and meson will the only one we
support.</p>
<p>The new build cheatsheet is:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd libvips-x.y.x
meson setup build-dir --prefix=/aaa/bbb/ccc
cd build-dir
meson compile
meson test
meson install
</code></pre></div></div>
<p>The new system is a <em>lot</em> faster. On this PC, the autotools build system used
to take 17s to configure and 6s to build libvips. Meson takes 2.8s to
configure and ninja takes 4.6s to build. A four times speedup is very welcome.</p>
<p>The <a href="https://github.com/libvips/libvips#building-from-source">libvips
README</a>
has some more notes.</p>
<h1 id="general-minor-improvements">General minor improvements</h1>
<ul>
<li>add <code class="language-plaintext highlighter-rouge">extend</code>, <code class="language-plaintext highlighter-rouge">background</code> and <code class="language-plaintext highlighter-rouge">premultiplied</code> to <a href="/API/current/libvips-resample.html#vips-mapim"><code class="language-plaintext highlighter-rouge">vips_mapim()</code></a> to fix edge antialiasing
[GavinJoyce]</li>
<li>improve the pixel RNG for the noise operators</li>
<li>add support for regions in C++ API [shado23]</li>
<li>improve introspection annotations [tintou]</li>
</ul>libvips 8.13 is now done, so here’s a summary of what’s new. Check the ChangeLog if you need more details.New pyvips and php-vips2022-04-18T00:00:00+00:002022-04-18T00:00:00+00:00https://www.libvips.org/2022/04/18/New-pyvips-and-php-vips<p>pyvips and php-vips have launched interesting new versions.</p>
<h1 id="pyvips">pyvips</h1>
<p>pyvips version 2.2 is just out and thanks to work by erdmann features a range
of useful new features.</p>
<h2 id="pil-and-numpy">PIL and numpy</h2>
<p>pyvips now has much better integration with numpy and PIL. For example,
you can make a Numpy array like this:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="n">a</span> <span class="o">=</span> <span class="p">(</span><span class="n">np</span><span class="p">.</span><span class="n">random</span><span class="p">.</span><span class="n">random</span><span class="p">((</span><span class="mi">100</span><span class="p">,</span> <span class="mi">100</span><span class="p">,</span> <span class="mi">3</span><span class="p">))</span> <span class="o">*</span> <span class="mi">255</span><span class="p">).</span><span class="n">astype</span><span class="p">(</span><span class="n">np</span><span class="p">.</span><span class="n">uint8</span><span class="p">)</span>
</code></pre></div></div>
<p>Then wrap a pyvips image around it using
<a href="https://libvips.github.io/pyvips/vimage.html#pyvips.Image.new_from_array"><code class="language-plaintext highlighter-rouge">Image.new_from_array</code></a>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">pyvips</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">pyvips</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">new_from_array</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</code></pre></div></div>
<p>pyvips will even guess a sensible interpretaion for you (sRGB in this case),
or you can specify with the optional <code class="language-plaintext highlighter-rouge">interpretation=</code> argument. This works
by sharing a memory buffer between the two libraries, so no data is copied
or duplicated, just a pointer.</p>
<p>Going the other way, you can make a numpy array from a pyvips image using
<a href="https://libvips.github.io/pyvips/vimage.html#pyvips.Image.numpy"><code class="language-plaintext highlighter-rouge">Image.numpy()</code></a>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a1</span> <span class="o">=</span> <span class="n">image</span><span class="p">.</span><span class="n">numpy</span><span class="p">()</span>
</code></pre></div></div>
<p>Or just use numpy’s <code class="language-plaintext highlighter-rouge">asarray()</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a1</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">image</span><span class="p">)</span>
</code></pre></div></div>
<p>Again, there’s no copying of data, just a pointer, so this is a fast way
for numpy to load many image formats.</p>
<p>You can use the same method to make a PIL image:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">PIL.Image</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">pyvips</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">black</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">100</span><span class="p">,</span> <span class="n">bands</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
<span class="n">pil_image</span> <span class="o">=</span> <span class="n">PIL</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">fromarray</span><span class="p">(</span><span class="n">image</span><span class="p">.</span><span class="n">numpy</span><span class="p">())</span>
</code></pre></div></div>
<p>Or to make a pyvips image from PIL:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pil_image</span> <span class="o">=</span> <span class="n">PIL</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">new</span><span class="p">(</span><span class="s">'RGB'</span><span class="p">,</span> <span class="p">(</span><span class="mi">60</span><span class="p">,</span> <span class="mi">30</span><span class="p">),</span> <span class="n">color</span><span class="o">=</span><span class="s">'red'</span><span class="p">)</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">pyvips</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">new_from_array</span><span class="p">(</span><span class="n">pil_image</span><span class="p">)</span>
</code></pre></div></div>
<p>Again, no copying.</p>
<h2 id="improved-indexing">Improved indexing</h2>
<p>Band indexing now supports an optional step. For example:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iamge</span> <span class="o">=</span> <span class="n">image</span><span class="p">[::</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div></div>
<p>Will reverse the bands in an image, so RGB becomes BGR. You
can also index with a list of bools. The docs have <a href="https://libvips.github.io/pyvips/vimage.html?highlight=getitem#pyvips.Image.__getitem__">all the
details</a>.</p>
<h2 id="other-improvements">Other improvements</h2>
<p>There’s a new
<a href="https://libvips.github.io/pyvips/vimage.html?highlight=invalidate#pyvips.Image.invalidate"><code class="language-plaintext highlighter-rouge">invalidate</code></a> method you can use to throw images out of the
various libvips caches, a useful speedup for pyvips method call, and support
for <code class="language-plaintext highlighter-rouge">Path</code> objects for load and save.</p>
<h1 id="php-vips">php-vips</h1>
<p>Version 2.0 of <a href="https://github.com/libvips/php-vips">php-vips</a>
has been rebuilt on top of php’s new <a href="https://www.php.net/manual/en/book.ffi.php">FFI
module</a>. This new version should
be much simpler to maintain, develop, support and install.</p>
<p>php-vips used to come in two parts: there was a PHP extension written in C
called <a href="https://github.com/libvips/php-vips-ext">php-vips-ext</a> which gave
very low-level access to libvips, and a pure PHP layer (called php-vips) which
implemented the public API.</p>
<p>This setup was tricky to manage for a range of reasons, but maintaining the C
extension was the hardest part. It was nasty to install as well, with many
users having issues getting it to work.</p>
<p>PHP now has an FFI interface to external libraries, so we’ve rewritten
php-vips to use that to make calls directly into libvips. The public API
hasn’t changed, so everyone’s code should still work.</p>pyvips and php-vips have launched interesting new versions.What’s new in 8.122021-11-14T00:00:00+00:002021-11-14T00:00:00+00:00https://www.libvips.org/2021/11/14/What's-new-in-8.12<p>Here’s a quick overview of what’s new in libvips 8.12. Check
the <a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
if you need more details.</p>
<p>Many thanks to Andrewsville, lovell, LionelArn2, vibbix, manthey,
martimpassos, indus, hroskes, kleisauke and others for their great work on
this release.</p>
<h1 id="better-gif-save">Better GIF save</h1>
<p>Previous libvips versions have used imagemagick for GIF save. This worked
well, but was slow and needed a <em>lot</em> of memory.</p>
<p>For example, with libvips 8.11 and <code class="language-plaintext highlighter-rouge">3198.gif</code>, a 140 frame video clip with
300 x 200 pixels per frame, I saw:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e vipsthumbnail 3198.gif[n=-1] --size 128 -o x.gif
221668:5.57
</code></pre></div></div>
<p>That’s a peak of 220MB of memory and 5.6s of real time to make a 128 x 128
thumbnail video.</p>
<p>Thanks to work by Lovell Fuller, libvips now has a dedicated GIF
writer using the <a href="https://github.com/dloebl/cgif">cgif</a> library and
<a href="https://github.com/lovell/libimagequant">libimagequant</a>. I see:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e vipsthumbnail 3198.gif[n=-1] --size 128 -o x.gif
46128:2.40
</code></pre></div></div>
<p>Now it’s 46MB and 2.4s – twice the speed and five times
less memory. Thanks to libimagequant, quality should be better too:
it’ll pick a more optimised palette, and dithering should be more accurate.</p>
<h1 id="much-lower-memory-and-file-descriptor-use-for-join-operations">Much lower memory and file descriptor use for join operations</h1>
<p>libvips has supported minimisation signals for a while now. At the end of
a save operation, for example, savers will emit a <code class="language-plaintext highlighter-rouge">minimise</code> signal,
and operations along the pipeline will do things like dropping caches and
closing file descriptors. In 8.12, we’ve expanded the use of this system
to help improve the performance of things like <code class="language-plaintext highlighter-rouge">arrayjoin</code>.</p>
<h2 id="the-problem">The problem</h2>
<p>For example, here’s how you might use <code class="language-plaintext highlighter-rouge">arrayjoin</code> to untile a Google Maps
pyramid:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/bin/python3
</span>
<span class="c1"># untile a google maps pyramid
</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">pyvips</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sys</span><span class="p">.</span><span class="n">argv</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">3</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s">"usage: untile-google.py ~/pics/somepyramid out.jpg"</span><span class="p">)</span>
<span class="n">sys</span><span class="p">.</span><span class="nb">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">indir</span> <span class="o">=</span> <span class="n">sys</span><span class="p">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">outfile</span> <span class="o">=</span> <span class="n">sys</span><span class="p">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="c1"># open the blank tile
</span><span class="n">blank</span> <span class="o">=</span> <span class="n">pyvips</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">new_from_file</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">indir</span><span class="si">}</span><span class="s">/blank.png"</span><span class="p">)</span>
<span class="c1"># find number of pyramid layers
</span><span class="n">n_layers</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">layer</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1000</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">isdir</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">indir</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">layer</span><span class="si">}</span><span class="s">"</span><span class="p">):</span>
<span class="n">n_layers</span> <span class="o">=</span> <span class="n">layer</span>
<span class="k">break</span>
<span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">n_layers</span><span class="si">}</span><span class="s"> layers detected"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">n_layers</span> <span class="o"><</span> <span class="mi">1</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s">"no layers found!"</span><span class="p">)</span>
<span class="n">sys</span><span class="p">.</span><span class="nb">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="c1"># find size of largest layer
</span><span class="n">max_y</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">filename</span> <span class="ow">in</span> <span class="n">os</span><span class="p">.</span><span class="n">listdir</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">indir</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">n_layers</span> <span class="o">-</span> <span class="mi">1</span><span class="si">}</span><span class="s">"</span><span class="p">):</span>
<span class="n">max_y</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">max_y</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="n">filename</span><span class="p">))</span>
<span class="n">max_x</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">filename</span> <span class="ow">in</span> <span class="n">os</span><span class="p">.</span><span class="n">listdir</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">indir</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">n_layers</span> <span class="o">-</span> <span class="mi">1</span><span class="si">}</span><span class="s">/0"</span><span class="p">):</span>
<span class="n">noext</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">splitext</span><span class="p">(</span><span class="n">filename</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">max_x</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">max_x</span><span class="p">,</span> <span class="nb">int</span><span class="p">(</span><span class="n">noext</span><span class="p">))</span>
<span class="k">print</span><span class="p">(</span><span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">max_x</span> <span class="o">+</span> <span class="mi">1</span><span class="si">}</span><span class="s"> tiles across, </span><span class="si">{</span><span class="n">max_y</span> <span class="o">+</span> <span class="mi">1</span><span class="si">}</span><span class="s"> tiles down"</span><span class="p">)</span>
<span class="n">tiles</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">max_y</span> <span class="o">+</span> <span class="mi">1</span><span class="p">):</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">max_x</span> <span class="o">+</span> <span class="mi">1</span><span class="p">):</span>
<span class="n">tile_name</span> <span class="o">=</span> <span class="sa">f</span><span class="s">"</span><span class="si">{</span><span class="n">indir</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">n_layers</span> <span class="o">-</span> <span class="mi">1</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">y</span><span class="si">}</span><span class="s">/</span><span class="si">{</span><span class="n">x</span><span class="si">}</span><span class="s">.jpg"</span>
<span class="k">if</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">isfile</span><span class="p">(</span><span class="n">tile_name</span><span class="p">):</span>
<span class="n">tile</span> <span class="o">=</span> <span class="n">pyvips</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">new_from_file</span><span class="p">(</span><span class="n">tile_name</span><span class="p">,</span>
<span class="n">access</span><span class="o">=</span><span class="s">"sequential"</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">tile</span> <span class="o">=</span> <span class="n">blank</span>
<span class="n">tiles</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">tile</span><span class="p">)</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">pyvips</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">arrayjoin</span><span class="p">(</span><span class="n">tiles</span><span class="p">,</span>
<span class="n">across</span><span class="o">=</span><span class="n">max_x</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="n">background</span><span class="o">=</span><span class="mi">255</span><span class="p">)</span>
<span class="n">image</span><span class="p">.</span><span class="n">write_to_file</span><span class="p">(</span><span class="n">outfile</span><span class="p">)</span>
</code></pre></div></div>
<p>You can run it like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips dzsave ../st-francis.jpg x --layout google
$ /usr/bin/time -f %M:%e ~/try/untile-google.py x x.jpg
8 layers detected
118 tiles across, 103 tiles down
4487744:15.24
</code></pre></div></div>
<p>So libvips 8.11 joined 12,154 tiles in 15s and needed 4.5GB of memory. This
is quite a substantial amount of RAM.</p>
<p>There’s another, less obvious problem: this program will need a file
descriptor for every tile. You can see this with a small shell script:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
~/try/untile-google.py x x.jpg &
<span class="nv">pid</span><span class="o">=</span><span class="nv">$!</span>
<span class="k">while </span><span class="nb">test</span> <span class="nt">-d</span> /proc/<span class="nv">$pid</span><span class="p">;</span> <span class="k">do
</span><span class="nb">ls</span> /proc/<span class="nv">$pid</span>/fd | <span class="nb">wc
sleep </span>0.1
<span class="k">done</span>
</code></pre></div></div>
<p>This script runs the program above and counts the number of open file
descriptors 10 times a second. It peaks at over 12,000 open descriptors!
You’ll probably need to make some tricky changes to your machine if you
want to be able to run things like this.</p>
<h2 id="minimise-during-processing">Minimise during processing</h2>
<p>In libvips 8.12, <code class="language-plaintext highlighter-rouge">arrayjoin</code> will emit <code class="language-plaintext highlighter-rouge">minimise</code> signals during
processing. It detects that it is operating in a sequential context and
will signal minimise on an input when the point of processing moves beyond
that tile. This means that input image resources are created and discarded
during computation, not just at the end.</p>
<p>With libvips 8.12, I see:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e ~/try/untile-google.py x x.jpg
8 layers detected
118 tiles across, 103 tiles down
2089992:14.66
</code></pre></div></div>
<p>So less than half the memory use, and it’s even a little quicker.</p>
<p>The saving in file descriptors is even better: it peaks at under 600, about 20
times fewer. That’s low enough to be well inside the limit on most machines,
so you’ll no longer need to reconfigure your server to do operations
like this.</p>
<p>This is quite a general system, and operations like <code class="language-plaintext highlighter-rouge">insert</code>, <code class="language-plaintext highlighter-rouge">join</code> and
<code class="language-plaintext highlighter-rouge">merge</code> all benefit.</p>
<h1 id="other-improvements-to-loaders-and-savers">Other improvements to loaders and savers</h1>
<p>As usual, there have been many small improvements to file format support.</p>
<ul>
<li>The TIFF writer has better progress feedback for many-page images.</li>
<li>The JPEG writer has a new option to set the restart interval.</li>
<li><code class="language-plaintext highlighter-rouge">dzsave</code> now supports IIIF3.</li>
<li>The PPM writer has an option to set the exact save format, and is better at
picking the correct format for you automatically.</li>
<li>The JPEG 2000 loader needs much less memory with very large, untiled images.</li>
<li>EXIF support is fixed for string fields containing metacharacters.</li>
<li>A new <code class="language-plaintext highlighter-rouge">fail-on</code> flag gives better control over detecting load errors. You
can spot image truncation easily now.</li>
<li>Save to buffer and target will pick the correct format automatically for
savers which implement multiple formats.</li>
</ul>
<h1 id="more-trigonometric-functions">More trigonometric functions</h1>
<p>Thanks to work by indus and hroskes, libvips now supports <code class="language-plaintext highlighter-rouge">atan2</code> and has
a full set of hyperbolic trig functions.</p>
<h1 id="minor-improvements">Minor improvements</h1>
<p>And of course other minor features and bug fixes. The
<a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
has more details, if you’re interested.</p>Here’s a quick overview of what’s new in libvips 8.12. Check the ChangeLog if you need more details.What’s new in 8.112021-06-04T00:00:00+00:002021-06-04T00:00:00+00:00https://www.libvips.org/2021/06/04/What's-new-in-8.11<p>libvips 8.11 is now out, so here’s a quick overview of what’s new. Check
the <a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
if you need more details.</p>
<p>Many thanks to Zeranoe, DarthSim, Projkt-James, afontenot, 781545872, erdmann,
kleisauke and others for their great work on this release.</p>
<p><a href="https://www.lunaphore.ch/">Lunaphore</a> kindly sponsored the development of
the new JPEG2000 features, see below.</p>
<h1 id="experimental-jpeg-xl-support">Experimental JPEG-XL support</h1>
<p>We’ve added experimental support for <a href="https://jpeg.org/jpegxl/">JPEG-XL</a>.
This is a new iteration of the JPEG standard, currently in development. The
Chrome web browser supports it behind a flag and it looks like it might be
enabled by default this autumn in Chrome 93.</p>
<p>There have been several attempts to replace JPEG with something better in the
last few years. HEIC is perhaps the best known: it can compress files to about
half the size of comparable JPEGs and supports a range of useful features
like animations, transparency and lossless compression. Unfortunately,
it has some patent issues which may limit its usefulness.</p>
<p>AVIF is rather like HEIC, but has no patents attached to it. Sadly the
available load and save libraries are extremely slow.</p>
<p>JPEG-XL looks like it might avoid all these problems: it offers the same great
compression and useful features as HEIC and AVIF, but has no problems with
patents and is fast enough to be practical.</p>
<p>I made a sample image. You’ll need to zoom in to check details:</p>
<p><img src="/assets/images/astronauts.png" alt="image compression comparison" /></p>
<p>Compression and decompression is quite a bit quicker than HEIC:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time vips copy astronauts.png x.jxl
real 0m0.218s
user 0m0.291s
sys 0m0.204s
$ time vips copy astronauts.png x.heic
real 0m0.413s
user 0m1.273s
sys 0m0.048s
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">-all</code> version of the Windows binaries include <code class="language-plaintext highlighter-rouge">jxlload</code> and <code class="language-plaintext highlighter-rouge">jxlsave</code>, but
remember the API is likely to change in 8.12.</p>
<h1 id="thread-recycling">Thread recycling</h1>
<p>We’ve revamped the libvips threading system.
The new implementation uses a <a href="https://developer.gnome.org/glib/stable/glib-Thread-Pools.html">GLib thread
pool</a>
to provide a single set of threads for the whole of libvips. This means
threads get reused rather then being constantly created and destroyed.</p>
<p>This was originally intended for WebAssembly but it turned out to be useful
for native environments as well. Relative speed-ups are varying between ~4%
and ~15% compared to the previous implementation. Especially for short-lived
pipelines and long-running processes, a performance improvement can be observed.</p>
<h1 id="loadable-modules-for-some-dependencies">Loadable modules for some dependencies</h1>
<p>libvips now supports building OpenSlide, libheif, Poppler and libMagick
as dynamically loadable modules. This makes it easier for distributions to
provide separate (optional) packages for these dependencies, making the core
package much slimmer and thus reducing the attack surface. Distributing
separate packages could also help to comply with GPL licensing (“mere
aggregation” clause) or patent-encumbered software.</p>
<p><a href="https://developer.gnome.org/glib/stable/glib-Dynamic-Loading-of-Modules.html">GModule</a>
is used to accomplish this in a portable way. We already had a simple plugin
system based on this, but the build part was not yet implemented.</p>
<p>These loadable modules are built automatically when GModule is supported,
which should be at least Linux, Windows and macOS. It can be disabled by
passing <code class="language-plaintext highlighter-rouge">--disable-modules</code> while configuring libvips.</p>
<h1 id="full-colour-text-rendering">Full-colour text rendering</h1>
<p>The libvips <code class="language-plaintext highlighter-rouge">text</code> operator was designed a while ago, when fonts were still
black and white. It outputs a one-band mask image which you then process with
the other libvips operations to do things like rendering text on an image.</p>
<p>Many fonts, especially emoji fonts, now have full-colour SVG characters.
There’s a new <code class="language-plaintext highlighter-rouge">rgba</code> flag to <code class="language-plaintext highlighter-rouge">text</code> to enable colour rendering. For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips text x.png "😀<span foreground='red'>red</span><span background='cyan'>blue</span>" --dpi 300 --rgba
</code></pre></div></div>
<p>Makes this:</p>
<p><img src="/assets/images/text.png" alt="coloured text" /></p>
<p>You can then use <code class="language-plaintext highlighter-rouge">composite</code> to render these text RGBA images on to another
image.</p>
<h1 id="jpeg2000-support">JPEG2000 support</h1>
<p>Thanks to generous sponsorship from Lunaphore, we’ve added support for the
(now rather elderly) JPEG2000 format.</p>
<p>The <code class="language-plaintext highlighter-rouge">jp2kload</code> and <code class="language-plaintext highlighter-rouge">jp2ksave</code> operations support all the file format
features: shrink-on-load, 8-, 16- and 32-bit data, any number of
image bands, tiled images, YCC colourspace, lossless compression, and
optional chroma subsampling. The lossy compression profile (controlled
by a <code class="language-plaintext highlighter-rouge">Q</code> parameter) is derived from the <a href="https://purl.pt/24107/1/iPres2013_PDF/An%20Analysis%20of%20Contemporary%20JPEG2000%20Codecs%20for%20Image%20Format%20Migration.pdf">British Museum’s JPEG2000
recommendations</a>.</p>
<p>We’ve also added support for JPEG2000 as a codec for TIFF load and save. This
means you can now directly load and save some popular slide image formats.
This should be useful for people in the medical imaging community.</p>
<p>It’s easy to use – for example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ vips copy k2.jpg x.tif[compression=jp2k,Q=90,tile]
$ vips copy k2.jpg x.tif[compression=jp2k,lossless,tile]
</code></pre></div></div>
<h1 id="c-api-improments">C++ API improments</h1>
<p>We’ve spent some time improving the C++ API.</p>
<p>We now use <a href="https://www.doxygen.nl">doxygen</a> to generate C++ API docs
automatically, and we’ve written a set of API comments. Hopefully the <a href="/API/8.11/cpp">new
documentation</a> should be a big improvement.</p>
<p>There are a couple of functional improvements too. There’s support for
<code class="language-plaintext highlighter-rouge">VipsInterpolate</code>, <code class="language-plaintext highlighter-rouge">guint64</code> and a new constructor,
<code class="language-plaintext highlighter-rouge">VImage::new_from_memory_steal</code>, which can save a copy operation.</p>
<h1 id="minor-improvements">Minor improvements</h1>
<ul>
<li>
<p>The <code class="language-plaintext highlighter-rouge">perlin</code>, <code class="language-plaintext highlighter-rouge">worley</code> and <code class="language-plaintext highlighter-rouge">gaussnoise</code> operators have a new <code class="language-plaintext highlighter-rouge">seed</code>
parameter to set the seed for their random number generator.</p>
</li>
<li>
<p>The <code class="language-plaintext highlighter-rouge">rank</code> operator has a new path for large windows on 8-bit images. It can
be up to 20 times faster in some cases.</p>
</li>
<li>
<p>Image histograms on large images now use <code class="language-plaintext highlighter-rouge">double</code> values. Previously,
we were limited to images with under 2^32 pixels.</p>
</li>
<li>
<p>There’s a new <code class="language-plaintext highlighter-rouge">black_point_compensation</code> option for colour operations
involving ICC profiles, and detection of bad profiles and fallback to
default profiles is much better.</p>
</li>
<li>
<p>The loaders and savers for PDFium, OpenSlide, vips, NIfTI, and FITS have
been moved to the new libvips IO framework.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">vipsthumbnail</code> can now load and save to and from stdin and stdout.</p>
</li>
<li>
<p>PNG save selects between 8- and 16-bit output more intelligently, and
supports background colour as metadata.</p>
</li>
<li>
<p>We’ve switched GIF load to the excellent libnsgif library, and libvips
embeds the library code.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">tiffsave</code> can write premultipled alpha.</p>
</li>
<li>
<p>PDFium support has been revised. It should now build very simply, and ought
to be much faster in threaded applications.</p>
</li>
<li>
<p>We’ve fixed a range of reference leaks in the mosaicing package, and we
now run the leak checker as part of CI.</p>
</li>
</ul>
<p>As usual,
the <a href="https://github.com/libvips/libvips/blob/master/ChangeLog">ChangeLog</a>
has more details, if you’re interested.</p>libvips 8.11 is now out, so here’s a quick overview of what’s new. Check the ChangeLog if you need more details.ruby-vips and image mutability2021-03-08T00:00:00+00:002021-03-08T00:00:00+00:00https://www.libvips.org/2021/03/08/ruby-vips-mutate<p>ruby-vips is now at version 2.1 with a few useful bug fixes and an interesting
new <code class="language-plaintext highlighter-rouge">mutate</code> feature. This new block makes it possible to modify images
efficiently and safely.</p>
<h1 id="draw-operations">Draw operations</h1>
<p>Up until now ruby-vips has been purely functional, in other words, all
operations created new images and no operations modified their arguments.</p>
<p>For example, you can draw a circle on an image, but you are given a new
image back and the original is not changed:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">y</span> <span class="o">=</span> <span class="n">x</span><span class="p">.</span><span class="nf">draw_circle</span> <span class="mi">255</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="ss">fill: </span><span class="kp">true</span>
</code></pre></div></div>
<p>This takes image <code class="language-plaintext highlighter-rouge">x</code>, makes a copy in memory, draws a circle with
centre at (50, 50) and radius 10 filled with pixels of value 255, and returns
this new image as <code class="language-plaintext highlighter-rouge">y</code>.</p>
<p>Purely functional operations have the huge advantage of allowing safe
sharing: if another part of your program is using the image referred to by
<code class="language-plaintext highlighter-rouge">x</code>, it won’t see a circle unexpectedly appear on its image.</p>
<p>All this copying and duplication is fine for small images, but can become
very slow for large ones. And what if you want to draw a series of circles? It
can become very painful indeed. For example:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/bin/ruby</span>
<span class="nb">require</span> <span class="s1">'vips'</span>
<span class="n">x</span> <span class="o">=</span> <span class="no">Vips</span><span class="o">::</span><span class="no">Image</span><span class="p">.</span><span class="nf">new_from_file</span> <span class="no">ARGV</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="mi">1000</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">x</span><span class="p">.</span><span class="nf">draw_circle</span> <span class="no">Array</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">){</span><span class="nb">rand</span><span class="p">(</span><span class="mi">255</span><span class="p">)},</span>
<span class="nb">rand</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="nf">width</span><span class="p">),</span> <span class="nb">rand</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="nf">height</span><span class="p">),</span> <span class="nb">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">),</span> <span class="ss">fill: </span><span class="kp">true</span>
<span class="k">end</span>
<span class="n">x</span><span class="p">.</span><span class="nf">write_to_file</span> <span class="no">ARGV</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div></div>
<p>I can run the program like this (<code class="language-plaintext highlighter-rouge">nina.jpg</code> is 6,000 x 4,000 pixels, not
unusual for modern DSLR camera):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e ./circles.rb ~/pics/nina.jpg x.jpg
4700668:13.29
</code></pre></div></div>
<p>To make this:</p>
<p><img src="/assets/images/circles1.jpg" alt="random circles" /></p>
<p>It works, but 13s and almost 5gb of memory to draw 1,000 circles is really
not good.</p>
<h1 id="metadata">Metadata</h1>
<p>There’s a second case where mutability is important: metadata updates.</p>
<p>ruby-vips lets you set image metadata. For example, you can set the EXIF
orientation tag on an image like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">x</span> <span class="o">=</span> <span class="no">Vips</span><span class="o">::</span><span class="no">Image</span><span class="p">.</span><span class="nf">new_from_file</span> <span class="s2">"k2.jpg"</span>
<span class="n">x</span><span class="p">.</span><span class="nf">set</span> <span class="s2">"orientation"</span><span class="p">,</span> <span class="mi">6</span>
<span class="n">x</span><span class="p">.</span><span class="nf">write_to_file</span> <span class="s2">"x.jpg"</span>
</code></pre></div></div>
<p>It works in simple cases, but actually this is not correct. The <code class="language-plaintext highlighter-rouge">x.set</code>
is modifying image <code class="language-plaintext highlighter-rouge">x</code> (though only modifying the image metadata rather
than any pixels) and in a large program, <code class="language-plaintext highlighter-rouge">x</code> could be shared.
In some large programs, you can get bizarre behaviour and even races
and crashes.</p>
<p>To be safe, you need to make a private copy of the image before you change
it, like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">x</span> <span class="o">=</span> <span class="no">Vips</span><span class="o">::</span><span class="no">Image</span><span class="p">.</span><span class="nf">new_from_file</span> <span class="s2">"k2.jpg"</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">x</span><span class="p">.</span><span class="nf">copy</span>
<span class="n">x</span><span class="p">.</span><span class="nf">set</span> <span class="s2">"orientation"</span><span class="p">,</span> <span class="mi">6</span>
<span class="n">x</span><span class="p">.</span><span class="nf">write_to_file</span> <span class="s2">"x.jpg"</span>
</code></pre></div></div>
<p>This is annoying, and worse than that, ruby-vips does not enforce this
rule.</p>
<h1 id="the-mutate-block">The <code class="language-plaintext highlighter-rouge">mutate</code> block</h1>
<p>ruby-vips 2.1 has a new feature that
tries to fix both these problems: <a href="https://www.rubydoc.info/gems/ruby-vips/2.1.0/Vips/Image#mutate-instance_method">the <code class="language-plaintext highlighter-rouge">mutate</code>
method</a>.</p>
<p>You use it like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/bin/ruby</span>
<span class="nb">require</span> <span class="s1">'vips'</span>
<span class="n">x</span> <span class="o">=</span> <span class="no">Vips</span><span class="o">::</span><span class="no">Image</span><span class="p">.</span><span class="nf">new_from_file</span> <span class="no">ARGV</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">x</span><span class="p">.</span><span class="nf">mutate</span> <span class="k">do</span> <span class="o">|</span><span class="n">y</span><span class="o">|</span>
<span class="mi">1000</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="n">y</span><span class="p">.</span><span class="nf">draw_circle!</span> <span class="no">Array</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="p">{</span><span class="nb">rand</span><span class="p">(</span><span class="mi">255</span><span class="p">)},</span>
<span class="nb">rand</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="nf">width</span><span class="p">),</span> <span class="nb">rand</span><span class="p">(</span><span class="n">x</span><span class="p">.</span><span class="nf">height</span><span class="p">),</span> <span class="nb">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">),</span> <span class="ss">fill: </span><span class="kp">true</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">x</span><span class="p">.</span><span class="nf">write_to_file</span> <span class="no">ARGV</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">mutate</code> method builds a private copy of the image,
uses it to construct an instance of <a href="https://www.rubydoc.info/gems/ruby-vips/2.1.0/Vips/MutableImage">a new class called
<code class="language-plaintext highlighter-rouge">MutableImage</code></a>,
and then yields that instance to the block.</p>
<p>An instance of <code class="language-plaintext highlighter-rouge">MutableImage</code> behaves just like an image object, except
that it is guaranteed not to be shared. There are new destructive versions
of operations like <code class="language-plaintext highlighter-rouge">draw_circle</code> (with the usual <code class="language-plaintext highlighter-rouge">!</code> naming convention)
which really do modify their argument.</p>
<p>After the block finishes, <code class="language-plaintext highlighter-rouge">mutate</code> unwraps the mutable image and returns a
new <code class="language-plaintext highlighter-rouge">Image</code> object. Because it manages the transition to <code class="language-plaintext highlighter-rouge">MutableImage</code> and
back, ruby-vips can enforce all the obvious rules to guarantee run-time
safety.</p>
<p>Because there’s only one allocate and copy, performance is much better. I see:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time -f %M:%e ./circles-mutate.rb ~/pics/nina.jpg x.jpg
290348:1.04
</code></pre></div></div>
<p>It’s 13x faster and needs 15x less memory. It’s now fast enough that
operations like <code class="language-plaintext highlighter-rouge">draw_circle!</code> could actually be useful.</p>
<p>You can use <code class="language-plaintext highlighter-rouge">mutate</code> to safely modify image metadata too:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">x</span> <span class="o">=</span> <span class="no">Vips</span><span class="o">::</span><span class="no">Image</span><span class="p">.</span><span class="nf">new_from_file</span> <span class="s2">"k2.jpg"</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">x</span><span class="p">.</span><span class="nf">mutate</span> <span class="k">do</span> <span class="o">|</span><span class="n">y</span><span class="o">|</span>
<span class="n">y</span><span class="p">.</span><span class="nf">set!</span> <span class="s2">"orientation"</span><span class="p">,</span> <span class="mi">6</span>
<span class="n">y</span><span class="p">.</span><span class="nf">remove!</span> <span class="s2">"icc-profile-data"</span>
<span class="k">end</span>
<span class="n">x</span><span class="p">.</span><span class="nf">write_to_file</span> <span class="s2">"x.jpg"</span>
</code></pre></div></div>
<p>For compatibility, the old <code class="language-plaintext highlighter-rouge">set</code> and <code class="language-plaintext highlighter-rouge">remove</code> methods are still there,
but we plan to make them start issuing warnings at some point.</p>
<p>The other libvips language bindings probably need a feature like this too,
but for now it’s just ruby-vips.</p>ruby-vips is now at version 2.1 with a few useful bug fixes and an interesting new mutate feature. This new block makes it possible to modify images efficiently and safely.libvips for WebAssembly2020-09-01T00:00:00+00:002020-09-01T00:00:00+00:00https://www.libvips.org/2020/09/01/libvips-for-webassembly<p>There’s a new full libvips binding for the browser and Node.js. It supports
reading and writing JPEG, PNG, WebP and TIFF images out-of-the-box on browsers
that <a href="https://caniuse.com/#feat=sharedarraybuffer">supports the SharedArrayBuffer API</a>,
it’s on NPM, and it comes with TypeScript declarations.</p>
<p>All the features of libvips can be viewed and executed in the browser within
this playground:</p>
<p><a href="https://kleisauke.github.io/wasm-vips/playground">https://kleisauke.github.io/wasm-vips/playground</a></p>
<p>The README in the repository for the binding has more details, including some
install notes and an example:</p>
<p><a href="https://github.com/kleisauke/wasm-vips">https://github.com/kleisauke/wasm-vips</a></p>
<p>But briefly, just enter:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm <span class="nb">install </span>wasm-vips
</code></pre></div></div>
<h1 id="how-its-done">How it’s done</h1>
<p>The whole of libvips and its dependencies has been compiled to WebAssembly
with <a href="https://emscripten.org/">Emscripten</a>. The resulting WASM binary is
~4.6 MB in size. It took several patches to make libvips usable in the
browser:</p>
<ul>
<li>
<p>The thread pool has been patched to reuse already started threads
(<a href="https://github.com/libvips/libvips/issues/1492">#1492</a>). The aim is that
this will also be integrated into a further version of libvips, as this
could also be useful for native environments.</p>
</li>
<li>
<p>A couple of <a href="https://emscripten.org/docs/porting/guidelines/function_pointer_issues.html">function pointer issues</a>
has been fixed (<a href="https://github.com/libvips/libvips/pull/1697">#1697</a>).</p>
</li>
<li>
<p>libffi needed to be ported to WebAssembly. See these blog posts for
background info:<br />
<a href="https://brionv.com/log/2018/05/06/emscripten-fun-porting-libffi-to-webassembly-part-1/">emscripten fun: porting libffi to WebAssembly part 1</a><br />
<a href="https://brionv.com/log/2018/05/27/emscripten-fun-libffi-on-webassembly-part-2/">emscripten fun: libffi on WebAssembly part 2</a></p>
</li>
<li>
<p>GLib needed a couple of patches throughout the build system
(<a href="https://github.com/emscripten-core/emscripten/issues/11066">emscripten-core/emscripten#11066</a>).</p>
</li>
</ul>
<h1 id="performance">Performance</h1>
<p>It’s rather tempting to benchmark how close WebAssembly gets to native speed,
and how much faster it is than pure JS image processing libraries (i.e.
no native code). The repo includes <a href="https://github.com/kleisauke/wasm-vips/tree/master/test/bench">benchmarks which test the performance
against alternative Node.js modules</a>, including
sharp and jimp.</p>
<p>On this benchmark and on my pc, sharp is 8.3x faster for JPEG, 3.6x faster
for PNG, and 2.2x faster for WebP images in comparison with wasm-vips.</p>
<p>wasm-vips on the other hand is 5.9x faster for JPEG and 8% faster for PNG
images in comparison with jimp.</p>
<table>
<thead>
<tr>
<th>Image format</th>
<th>Module</th>
<th>Ops/sec</th>
<th>Speed-up</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3">JPEG</td>
<td>jimp</td>
<td>0.91</td>
<td>1.0</td>
</tr>
<tr>
<td>wasm-vips</td>
<td>5.36</td>
<td>5.9</td>
</tr>
<tr>
<td>sharp</td>
<td>44.35</td>
<td>45.8</td>
</tr>
<tr>
<td rowspan="3">PNG</td>
<td>jimp</td>
<td>5.20</td>
<td>1.0</td>
</tr>
<tr>
<td>wasm-vips</td>
<td>5.63</td>
<td>1.1</td>
</tr>
<tr>
<td>sharp</td>
<td>20.30</td>
<td>3.9</td>
</tr>
<tr>
<td rowspan="3">WebP</td>
<td>wasm-vips</td>
<td>6.20</td>
<td>1.0</td>
</tr>
<tr>
<td>sharp</td>
<td>13.73</td>
<td>2.2</td>
</tr>
</tbody>
</table>
<p>The substantial slowdown for JPEG images could be caused due to
<code class="language-plaintext highlighter-rouge">libjpeg-turbo</code> is compiled <em>without</em> SIMD support. This dependency uses
native inline SIMD assembly, which is currently not supported in Emscripten.
All code should be written using SIMD intrinsic functions or compiler vector
extensions.</p>
<p>Although the dependencies for the other image formats (i.e. <code class="language-plaintext highlighter-rouge">libspng</code> and
<code class="language-plaintext highlighter-rouge">libwebp</code>) are compiled with SIMD support there is still a slowdown
noticeable. A possible reason for that is that <code class="language-plaintext highlighter-rouge">liborc</code> is not built for
WebAssembly. This dependency is used by libvips to improve the performance of
the resize, blur and sharpen operations, but this is quite difficult to
compile for WebAssembly as it generates SIMD instructions on-the-fly.</p>
<p>Note that these benchmarks are expected to run faster when the WebAssembly
proposals for <a href="https://github.com/WebAssembly/simd">SIMD</a> and <a href="https://github.com/WebAssembly/threads">threads</a> have been standardized.</p>
<h1 id="how-it-works">How it works</h1>
<p>All libvips operations and enumerations are exposed through
<a href="https://emscripten.org/docs/porting/connecting_cpp_and_javascript/embind.html">Embind</a>,
so that the compiled code can be used in JavaScript.</p>
<p>The binding itself is a variant of
<a href="/API/current/using-from-cpp.html">libvips’ C++ API</a>,
with additional support for the <code class="language-plaintext highlighter-rouge">emscripten::val</code> C++ class to transliterate
JavaScript code to C++. For example, consider this JavaScript code:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Image source: https://www.flickr.com/photos/jasonidzerda/3987784466</span>
<span class="kd">const</span> <span class="nx">thumbnail</span> <span class="o">=</span> <span class="nx">vips</span><span class="p">.</span><span class="nx">Image</span><span class="p">.</span><span class="nx">thumbnail</span><span class="p">(</span><span class="dl">'</span><span class="s1">owl.jpg</span><span class="dl">'</span><span class="p">,</span> <span class="mi">128</span><span class="p">,</span> <span class="p">{</span>
<span class="na">height</span><span class="p">:</span> <span class="mi">128</span><span class="p">,</span>
<span class="na">crop</span><span class="p">:</span> <span class="nx">vips</span><span class="p">.</span><span class="nx">Interesting</span><span class="p">.</span><span class="nx">attention</span> <span class="cm">/* or: 'attention' */</span>
<span class="p">});</span>
</code></pre></div></div>
<p>Which shrinks an image to fit within a 128×128 box. Excess pixels are trimmed
away using the <code class="language-plaintext highlighter-rouge">attention</code> strategy that positioned the crop box over the
most significant feature:</p>
<p><a href="/API/current/tn_owl.jpg"><img src="/API/current/tn_owl.jpg" alt="Attention strategy" /></a></p>
<p>This function and enumeration was automatically generated within C++ as:</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">EMSCRIPTEN_BINDINGS</span><span class="p">(</span><span class="n">my_module</span><span class="p">)</span> <span class="p">{</span>
<span class="n">enum_</span><span class="o"><</span><span class="n">VipsInteresting</span><span class="o">></span><span class="p">(</span><span class="s">"Interesting"</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"none"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_NONE</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"centre"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_CENTRE</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"entropy"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_ENTROPY</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"attention"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_ATTENTION</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"low"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_LOW</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"high"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_HIGH</span><span class="p">)</span>
<span class="p">.</span><span class="n">value</span><span class="p">(</span><span class="s">"all"</span><span class="p">,</span> <span class="n">VIPS_INTERESTING_ALL</span><span class="p">);</span>
<span class="n">class_</span><span class="o"><</span><span class="n">Image</span><span class="o">></span><span class="p">(</span><span class="s">"Image"</span><span class="p">)</span>
<span class="p">.</span><span class="n">constructor</span><span class="o"><></span><span class="p">()</span>
<span class="p">.</span><span class="n">function</span><span class="p">(</span><span class="s">"thumbnail"</span><span class="p">,</span> <span class="o">&</span><span class="n">Image</span><span class="o">::</span><span class="n">thumbnail</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>There's a new full libvips binding for the browser and Node.js. It supports reading and writing JPEG, PNG, WebP and TIFF images, it's on NPM, and it comes with TypeScript declarations.