A JPEG encoder gives every part of a picture the same care. It does not know that the reader is looking at the face and not at the hedge behind it, so the hedge gets the same number of bits as the face. On a portrait, on a product shot, on almost any photograph with a subject in it, a meaningful share of the file is spent on the part nobody looks at.
This is not a flaw in the encoder. The encoder is doing exactly what it was designed to do, which is to reproduce the whole picture to a stated fidelity. It has no way of knowing that the fidelity requirement is not uniform across the frame, because nobody has ever told it that.
The usual response is to turn the quality down, which reduces fidelity everywhere including on the face. That is the trade the quality slider offers and it is the only trade it offers.
Where a person actually looks
Saliency prediction is the field that answers the missing question. A saliency model is trained on where people’s eyes go when shown a picture, and what it returns is a map: bright where attention lands, dark where it does not. The one in the tool below is small, about 4 MB.
The interesting part is what you can do with that map in a browser, and the answer involves a constraint that shapes everything. Drop a photograph in, look at the map, and then look at the table of measured sizes. Everything happens in this browser tab and the picture is never uploaded.
Saliency guided JPEG
A JPEG encoder spends the same number of bits everywhere. It does not know that you are looking at the face and not at the hedge behind it, so the hedge gets the same care. This runs a small saliency model over the picture, smooths only what the model says nobody looks at, and hands that to the ordinary browser encoder at the quality you set. Smooth areas cost an encoder very few bits, so the file comes out smaller while the subject is left exactly as it was. The model is 4.0 MB, fetched from this site only after you ask for it, alongside a 3.5 MB runtime shared with the other model tools here. Your picture is read in this browser tab, never uploaded, and there is nowhere for it to be uploaded to.
Two other tools on this blog stand next to it. The Image quality curve shows what a quality number costs on an ordinary picture, with no model in sight. The Page weight budget calculator turns those kilobytes back into a loading time. This one asks the third question: whether the same kilobytes can be spent somewhere more useful.
or press Enter to pick one. JPEG, PNG or WebP.
Sample photograph, drawn in this browser.
That second measurement encodes the picture a handful of extra times to find the matching quality number. Turn it off if you only want the two files.
How far the untouched zone reaches beyond the subject the model found. At 100% nothing is smoothed at all and the two files are identical, which is the honest way to check that the rest of this is doing something.
How much of the smoothed version is mixed in out there. At 0% the background is left alone and nothing is saved.
| Measured | Value |
|---|
Nothing has been measured yet.
Why the encoder is never told any of this. In a browser you cannot set the quality per region. canvas.toBlob takes exactly one number for the whole picture and there is no way to hand it a map. So the encoder is not steered at all, its food is prepared instead. A smooth area costs an encoder very few bits, so the saving is taken where smoothing does not show, and the encoder does the rest without knowing why its job got easier.
What this cannot do. It is no better at looking than the model is. On a picture without a clear subject, a landscape, a texture, a crowd, the model still finds something, everything else is smoothed, and that can be exactly the wrong thing to smooth. This is why the saliency view exists: look at it before you keep the file. It is not magic either, it is a redistribution. The saving comes from real detail being thrown away in the background, and if you zoom in out there you will see it gone. A picture that is subject from edge to edge saves nothing, and the table says so rather than rounding it up. Comparable tools exist. What is unusual here is only that nothing leaves the tab.
The encoder is never told any of this
Here is the constraint, and it is worth stating plainly because it explains the whole design. In a browser you cannot set JPEG quality per region. The canvas encoding call takes exactly one number for the whole picture, and there is no argument, no option and no workaround that lets you hand it a map.
So the encoder is not steered. Its input is prepared instead. The background, meaning everything the saliency map says nobody looks at, is smoothed before the encoder ever sees it. A smooth area is cheap to encode, because smoothness is exactly what a frequency transform represents compactly, so the same encoder at the same quality setting produces a substantially smaller file.
On the tool’s own sample the measured result was 35109 bytes down to 20571, a saving of 41 per cent, at a quality setting that was not changed at all. The subject is byte for byte what it would have been.
Why smoothing is what makes a file smaller
It is worth being precise about why smoothing the background is the lever, rather than, say, reducing its colours or adding noise to it. The reason is in how JPEG stores a picture.
The format cuts the image into blocks and converts each block into a set of frequency coefficients: one coefficient for the average brightness of the block, then progressively more for finer and finer variation within it. Compression happens by storing the fine coefficients with less precision, and most of them, in most blocks, end up at zero. Long runs of zeroes are what the final entropy coding stage compresses so effectively.
A block of hedge is full of fine variation, so few of its coefficients are zero and all of them have to be stored. The same block smoothed has almost nothing but its average left, so nearly every coefficient goes to zero and the block costs a handful of bits instead of hundreds. Smoothing is not a trick played on the encoder. It is handing the encoder exactly the kind of content the format was built to store cheaply.
busy block -> many non-zero coefficients -> expensive
smooth block -> almost all zero -> nearly freeWhere the saving actually comes from
It comes from throwing detail away. That needs saying without hedging, because the numbers look like something for nothing and they are not.
What makes it a good trade rather than a con is where the detail is thrown away, and the reason the tool has a view that shows you the prepared picture is so you can check that judgement yourself rather than take it on trust. Zoom into the background of the prepared version and you will see it: leaves that have become a wash, texture on a wall that has become a gradient. Look at the whole picture at the size it will actually be seen and you will not.
This is the same reasoning that sits underneath every compression decision, which is why the site’s quality curve is worth reading alongside this: both are about spending bits where they are noticed. The difference is only that the curve varies fidelity over the whole picture and this varies it across the frame.
There is one more reason the map is worth looking at before the file. A saliency model predicts where attention lands on a picture shown on its own, with nothing around it. On a web page your picture has a headline above it, text beside it and a button below it, all of which compete for the same attention, and none of which the model has ever seen.
Most of the time that does not matter, because the subject of a photograph is its subject whatever surrounds it. It matters when the reason a picture is on the page has nothing to do with its visual subject: a screenshot where the important part is a small piece of interface in a corner, a diagram where the point is a label. The model will find the visually prominent thing and smooth the thing you actually needed legible.
The three dials and the one that proves the tool works
Quality is the ordinary JPEG quality number, and it applies to both files so the comparison stays fair. Smoothing is how strongly the background is treated. Protection is the interesting one.
Protection sets how far the untouched zone reaches beyond the subject the model found. Turned all the way up, nothing is smoothed at all and the two files come out identical, which is not a wasted setting: it is the honest way to check that everything else in the tool is actually doing something. If the guided file and the plain file are the same size at full protection and different at half, the mechanism is real.
The optional second measurement is the one that turns a number into an argument. It encodes the picture repeatedly at plain quality settings to find the one that lands on the same file size as the guided version, and reports it. On the sample that was quality 64 against a nominal 78. So the choice is not between a smaller file and a larger one, it is between a picture where the background lost detail and a picture where everything did, at identical bytes.
Why the map has to be blurred before it is used
A detail that is invisible when it works and obvious when it does not. The saliency map comes back at a low resolution and with fairly hard boundaries, and using it directly leaves a visible line in the finished picture where the treated area meets the untreated one.
The eye is extremely good at detecting a boundary between two textures, much better than it is at detecting either texture being wrong. A background that is uniformly a bit soft reads as depth of field. A background that is soft on one side of an invisible line and sharp on the other reads as a mistake, and the reader will see it without being able to say what they saw.
So the map is softened before it drives anything, and the transition is spread over enough pixels that no line exists. It costs one blur and it is the difference between a technique and an artefact.
Judging the result honestly
There is a trap in evaluating any compression change, and it catches people who are being careful rather than people who are being careless. If you compare the two files by flipping between them at full magnification, the guided one will look worse, because you will be staring at the background and the background is genuinely softer.
That comparison answers a question nobody asked. The question is whether a reader, seeing the picture once, at the size it appears on the page, in the flow of reading something else, notices anything. So look at both at the size they will be used, in sequence rather than side by side, and ask whether you can tell which is which without knowing in advance.
If you can, turn the smoothing down until you cannot, and take whatever saving is left. That is a smaller number than the maximum the tool will produce and it is the number worth having. A technique that requires you to accept a picture you can see is worse has stopped being an optimisation and started being a compromise, and those need a different kind of decision.
What this cannot do
It is no better at looking than the model is. On a picture without a clear subject, a landscape, a texture, a crowd, the model still finds something and everything else gets smoothed, and that can be exactly the wrong thing to smooth. The sample the tool opens with is deliberately one of these: a sunset with no single subject, so that the first thing you see is the awkward case rather than the flattering one.
A picture that is subject from edge to edge saves nothing, and the table says so rather than rounding a small number up into a claim. A picture that is already small enough gains nothing worth the extra step. And this is a redistribution rather than a compression breakthrough: the encoder is unchanged, the format is unchanged, and anybody with a better encoder will beat it on both files equally.
Where it fits
Comparable techniques exist in server side pipelines, where quality genuinely can be varied per region and the encoder can be driven directly. What is unusual here is only the setting: it runs in the tab, the model comes to the picture, and nothing goes the other way.
For a site, the honest place for this is on the small number of images that matter and are heavy: the hero, the product shot, the one photograph on a landing page. Running it across a whole media library would be work for a marginal return, and the more effective lever there is usually picking a better format or serving the right size in the first place. When you do want to know what the kilobytes are actually costing your visitors, the performance budget is the tool that turns them back into seconds.
The idea underneath is more portable than the tool. Almost every optimisation problem in publishing is really a question of where the budget goes rather than how large it is, and most of them are solved by treating everything identically because nobody has the information to do otherwise. A saliency map is that missing information for one narrow case. What is worth taking away is the habit of asking, whenever something is being applied uniformly, whether it needs to be.
