Photo Editing

Repair a photo that messengers and re-saves have turned to blocks

JPEG damage is not random. It is made in blocks of eight by eight pixels, in a known order, which is why a model trained on compressed images beats any amount of sharpening. It runs in your browser, repairs at four times the size and hands the result back clean.

Repair a photo that messengers and re-saves have turned to blocks

Somebody sends you a photograph in a group chat. You save it, crop it, save it again, send it on, and somebody else does the same. By the time it reaches the page it is covered in square patches in the flat areas, coloured smearing along every hard edge, and a fizz of dots around any lettering. It is recognisably the same picture and it is unusable.

The instinct is to reach for sharpening, which makes it worse by amplifying exactly the artefacts you were trying to lose, or for noise reduction, which softens the whole picture in order to hide them. Neither is addressing what actually happened, because neither knows what actually happened.

What happened is specific and it is worth understanding, because the damage has a structure and structure is something a model can be taught.

The shape of JPEG damage

JPEG does not compress pixels. It cuts the picture into blocks of eight by eight pixels, converts each block into a set of frequency coefficients, and then throws away precision in the coefficients that carry fine detail. Reconstructing the block from what is left gives you something close to the original, and the error it makes is not random: it is a pattern that lives inside those eight by eight boundaries.

That is why heavy compression produces visible squares. Two neighbouring blocks each get their own approximation, the approximations do not quite meet at the boundary, and the eye is very good at seeing a straight line where two flat areas fail to match. It is also why the damage clusters around edges: a hard edge needs high frequency coefficients to describe it, those are the first ones to be crushed, and what comes back is a ripple running along the edge.

Colour gets treated worse than brightness. Most JPEG encoders store colour at half resolution in each direction, on the reasoning that the eye is less sensitive to colour detail, which is true and also the reason a red logo on a white background comes back with a pink halo. And every one of these losses compounds: each save decodes the previous approximation and approximates it again, which is why the fifth generation of a picture is so much worse than the first.

Table matching four visible JPEG artefacts to their causes: square patches in flat areas come from neighbouring eight by eight blocks being approximated separately, ripples along hard edges from high frequency coefficients being crushed first, coloured halos on a red logo from colour being stored at half resolution, and the picture getting worse every save from each pass approximating the previous approximation.

A model that knows the grid

The network in the tool below was trained specifically on compressed images, which means it has seen this damage several million times and has learned what tends to lie underneath it. It knows that a straight discontinuity every eight pixels is probably a block boundary and not a feature of the scene. A general purpose sharpener knows no such thing and treats that boundary as an edge to enhance.

It weighs about 9.8 MB. Drop a battered picture in, press the button once to fetch the model, and compare the magnified before and after. Everything happens in this browser tab: the picture is never uploaded, and the only thing that travels is the model, coming down from this site the first time you ask for it.

JPEG artefact repair

A picture that has been through a group chat, saved again, forwarded and saved again comes back with square blocks in the flat areas, coloured smearing along hard edges and a fizz of dots around lettering. This runs a small network over it that was trained on exactly that damage, so it knows the eight by eight grid JPEG works in and rebuilds inside it instead of softening everything in sight. The network is about 9.8 MB, fetched from this site only after you ask for it, alongside a 3.5 MB runtime shared with the other model tools here. Your picture is read in this browser tab, never uploaded, and there is nowhere for it to be uploaded to.

Two other tools on this blog stand next to it. The Generation loss simulator shows the damage being done, saving pass after saving pass. The Image quality curve measures what a given quality setting costs before you commit to it. This one is the third step, the attempt to take some of it back.

Picture
Drop a battered JPEG here
or press Enter to pick one. JPEG, PNG or WebP.

Sample picture, drawn and then deliberately wrecked in this browser.

Result size

The network only ever answers at four times the width and four times the height. The answer is then drawn back down to whichever size you picked here, and that is the whole trick: the mending happened in the large version, where a block edge is sixteen pixels across and there is room to reason about it, so shrinking it again keeps the mended edge and quietly discards the invented detail. Original size is the setting for a picture that has to go back on a page.

Load the model to begin.
Click the overview, or use the arrow keys, to move the magnifier.
Before, magnified four times
After, magnified four times

What this deliberately cannot do. What the compression threw away is gone. The model writes a plausible replacement for it, not the original, and on lettering and thin lines you can usually see which is which: a letter comes back sharp and slightly the wrong shape. A picture that was never damaged does not get better from this, it only gets different, because the model has been taught that every input is broken and will treat a clean edge as one more thing to fix. It is not an enlarger either. The four times output is a by-product of how the network was built, and it carries no detail that was not already implied in the small version, so printing from it will disappoint you. And it has no idea what your picture is of, so where the damage is heavy it will invent something confident and wrong rather than leave a gap.

How it is worked out. The picture is cut into tiles of 128 pixels with 16 pixels of margin on each inner side. The margin is fed to the network so the pixels at a tile border have their surroundings, and is then thrown away, which is what stops a grid appearing in the result. Each tile is padded with a mirrored strip until both its edges are multiples of eight, because that is the block size the network was built around and it refuses anything else. Everything the network hands back is four times the size, so every destination coordinate is multiplied by four before the piece is put in place, and the mirrored overhang is cut off at the same time. Pictures longer than 1200 pixels on their long edge are drawn down to that first, because the four times answer for a full sized photograph does not fit in a browser tab. Comparable tools exist and several of them are good. The difference worth naming here is the mechanism: this one runs in the tab, and the only thing that travels is the model, in the other direction.

WunderPaint
The Dynamic Design and Automation Studio
WunderPaint is a layered image editor for your WordPress media library. WunderPaint Studio is the same thing in any browser, free and without an account.

Use the magnifier, not the thumbnail

The tool shows the picture small and then shows a crop of it at four times, without smoothing, before and after. That is deliberate, because judging this kind of repair from a thumbnail is close to useless.

Scaled down, almost anything looks better: the shrinking itself averages the artefacts away, so a picture you have done nothing to will appear improved next to its own full sized version. Everything that matters here happens at the scale of an eight pixel block, and at thumbnail size an eight pixel block is a fraction of a screen pixel.

Move the magnifier onto the places where this kind of damage lives rather than onto the middle of the subject. A flat area of sky, where blocking shows up as patchwork. A hard edge between two very different colours, where the ripple lives. Any lettering at all, which is where both the damage and the model’s invention are easiest to see. If the repair holds up in those three places, it holds up.

Why it repairs at four times the size

The model only ever answers at four times the width and four times the height. That looks like an odd constraint for a repair tool and it turns out to be the mechanism rather than an inconvenience.

A block boundary in the original is a single hard step between two adjacent pixels. There is nowhere to put a repair: whatever the correct value is, it has to be one of two pixels. At four times the size that same boundary is sixteen pixels across, and the network has room to lay down a gradient, to bend an edge back into shape, to spread a correction over a distance instead of into a single pixel.

The answer is then drawn back down to whatever size you asked for. Shrinking it keeps the mended edge, because the mend is a real change in the shape of the transition, and quietly discards the detail the network invented, because invented detail does not survive being averaged down. You get the repair without the hallucination, which is a better trade than it sounds.

That is also why the size control does not re-run anything. The four times answer is kept, and switching between original size, twice and four times just redraws from it.

Tiles, margins, and a number that is not negotiable

A whole photograph cannot go through in one pass, because the answer is sixteen times the area of the input and a browser tab will not hold that for anything larger than a thumbnail. So the picture is cut into tiles of 128 pixels.

Tiles need margins. A network that looks at a neighbourhood gives the wrong answer for a pixel whose neighbourhood has been cut away, and if you write those wrong edge pixels into the result you get a faint grid, which is a particularly bitter failure in a tool whose entire job is removing a grid. Each tile is therefore fed with sixteen extra pixels of margin on every inner side, and those sixteen pixels are discarded before the tile is written out. In the four times answer that margin is sixty four pixels, and every destination coordinate has to be multiplied by four, which is the single most likely place for this kind of code to go quietly wrong.

The constraint that is not negotiable is that both edge lengths must be divisible by eight. Feed the model anything else and it does not degrade, it aborts, because internally it reshapes the data into a grid of eight by eight windows and the arithmetic does not come out. A 50 by 70 tile has to become 56 by 72 first, and the way to do that without inventing content is to mirror the existing pixels outwards and cut the overhang off afterwards.

50 wide  ->  56 wide, columns 50 to 55 mirror columns 49 down to 44
70 high  ->  72 high, rows 70 and 71 mirror rows 69 and 68
after the model, cut the overhang back off
Flow diagram of the four times repair: a block boundary is a single hard step between two adjacent pixels, at four times the size it is sixteen pixels across, which gives the model room to lay down a gradient and bend the edge back into shape, and drawing the answer back down keeps the mend while averaging the invented detail away.

The result comes back as PNG, deliberately

A repaired JPEG saved as a JPEG is a JPEG that has been through one more generation. Everything the model just undid gets partially redone on the way out, at whatever quality setting the encoder happened to use, and the file you download is measurably worse than the one you were looking at.

So the download is PNG. It is a larger file and that is the correct trade for an intermediate: the point of this step is to hand you something clean to work from, not something finished to publish. When you do come to publish it, choose the output quality deliberately at that point, with the measured curve rather than a remembered default, and pick the format that suits the picture rather than the one the last tool defaulted to.

When not to use it

There is a category of picture where running this is actively the wrong move, and it is worth naming before you reach for it out of habit.

A photograph that came off a camera and has been saved once at a decent quality has damage well below the level anybody will notice on a page. Running a repair over it means accepting a small reworking of every edge in exchange for fixing something nobody could see. The same is true of a screenshot of an interface, where the flat areas are genuinely flat and the text was genuinely sharp: the model will treat crisp lettering as damaged lettering and give you back something marginally rounder.

The tool is for pictures whose history you can see. If you have to hunt for the artefacts, there is nothing here for you.

What it deliberately cannot do

What the compression threw away is gone. The model writes a plausible replacement for it, not the original. On lettering and thin lines you can usually see which is which if you look: a letter comes back sharp and slightly the wrong shape, because the network reconstructed a letter rather than recovering yours.

A picture that was never damaged does not get better from this. It gets different. The network was taught that every input is broken, so it will treat a clean edge as one more thing to fix, and the result is subtly softer and subtly reworked for no gain.

It is not an enlarger either. The four times output is a by-product of how the network was built and it carries no detail that was not already implied in the small version, so printing from it will disappoint you. And it has no idea what your picture is of, so where the damage is heavy it will invent something confident and wrong rather than leave a gap.

Pictures longer than 1200 pixels on their long edge are drawn down to that first, because the four times answer for a full sized photograph does not fit in a browser tab.

Where it fits in a workflow

Comparable tools exist and several of them are good. The difference worth naming here is mechanical rather than qualitative: this one runs in the tab, and the only thing that travels is the model, in the other direction. For a picture somebody else sent you, that distinction is often the whole reason to use it.

The more useful thing to take away is where the repair belongs in the order of operations. It goes first, before cropping, before colour work, before resizing, because every one of those steps will otherwise carry the artefacts along and interact with them. And it does not belong at all if you can get the original file instead: a repair is an approximation of a picture you could simply ask for. Understanding how a picture accumulates damage across saves is what tells you whether you are looking at a fixable problem or a chain you should step out of.

The broader lesson is one that applies to more than JPEG. The reason a specialised model beats a general filter here is not that it is larger or newer. It is that the damage was not random in the first place. It was made by a specific process, with a specific block size and a specific ordering of what gets discarded first, and anything that knows that process has an enormous advantage over anything that treats the result as an unfortunate mess. Compression is not noise. It is a decision, made eight pixels at a time, and it can be partly argued with.

Repair a photo that messengers and re-saves have turned to blocks

Table of Contents

Learn it by building something

Every week one thing you can make the same afternoon, from dynamic templates to 3D type. Written down step by step.

One mail a week, and then it ends.
Unsubscribe in one click.

Photo Editing

Blend Modes, Actually Explained: Six Formulas Worth Knowing

Most people click through the list until something looks nice. There is a small amount of maths underneath that turns the whole menu into a set of tools.

Security & Privacy

What Your Photos Tell Your Visitors: EXIF, GPS and WordPress

WordPress keeps the file you uploaded exactly as it arrived, sitting next to the resized copies it actually serves. If that file came off a phone, it very likely still carries the coordinates of the room it was taken in.

WordPress Images

Working Out the Image Size Your Theme Actually Wants

A slot measured in CSS pixels needs a file measured in image pixels, and on a Retina screen the second number is twice the first. Here is how to measure the slot, choose a ratio without losing half the photograph, and check that WordPress will actually generate the size you need.

SEO & Structured Data

WordPress Image SEO: What Works and What Is Folklore

Most image SEO checklists are ordered by how easy each item is to write about. This one ranks the advice by mechanism instead, naming the core function behind every claim, and says plainly which parts are folklore.

Security & Privacy

Black out every piece of text in a screenshot before you share it

A tool that guesses which text is sensitive will miss the one that mattered. This one covers every text field it finds and lets you click back what can stay. It also refuses to default to a blur, because blurring and pixelation can be undone, and there is published work showing exactly how.

Troubleshooting

Why Is Your WordPress Site Slow? A Diagnostic Order, Not a Checklist

WordPress performance advice is usually handed over as a flat, alphabetized checklist. This walks through the same fixes in the order they actually pay off, starting with the thing most sites get wrong first: images.

Download the free WunderPaint Plugin for WordPress

The WunderPaint workspace with the layers panel, adjustment sliders, text style presets and the asset library along the bottom

The Image Editor & Design Studio

Everything described here can be done in the browser, on your own site. The live demo runs the full editor with nothing to install.

Free

Chaos Art

Autonomous painters make one-of-a-kind abstract art in 3D space - gestures, art movements, painterly media, and embeds that paint a new original for every visitor.

Pro

Particle Strokes

Paint with swarms of light: twenty-two movements, a stamp you draw yourself, and curves that give a stroke a shape - the swarm keeps painting for a few seconds after you let go.

Pro

City Diorama

Any place on earth as a miniature you could hold: real streets, water and building footprints raised into a 3D diorama - or wrapped around a sphere as your own tiny planet.

Free

Papercut Art

Layered paper pictures with real depth - a photo sliced along its actual depth into up to twenty layers, parametric landscapes, animals and clouds you can shape, letters with real counters, and a look that runs on three dials.

Pro

3D Earth Studio

A hyperrealistic globe - day and night with city lights, live clouds, atmosphere halo, country borders and highlights, click-to-place markers with flight-route arcs, satellite orbits, seamless rotation video and a live website embed.

Free

Mystic Studio

Turn a birth date into wall art - a real natal chart with houses and aspects, the moon of that night, zodiac and Chinese zodiac posters, numerology cards and a synastry wheel for two, in eight artful themes.