The photograph is good apart from one thing. A wheelie bin at the edge of the frame, a stranger who walked into the shot, a cable running across the wall behind the product, a road sign standing exactly where the eye wants to go. Everything else about the picture is fine, and that one object is why it does not get used.
The traditional fix is a clone tool: pick a patch of pixels from somewhere else in the image and paint it over the offending object. It works, and on a plain wall it works well. On grass, on a hedge, on brickwork with a vanishing point, on anything with structure that changes across the frame, it takes patience and a good eye, and it leaves a smear that other people can see even if they cannot name it.
The reason is that cloning copies and does not predict. It can only move pixels you already have to a place they did not come from. It has no notion of what should have been behind the bin, only of what is somewhere else in the picture.
Filling a hole rather than covering it
Inpainting turns the problem around. Instead of choosing a patch, you cut a hole and ask a network what most plausibly belongs in it. The model has seen a great many photographs with holes cut in them and has learned to continue what runs into the hole from every side: the line of a hedge, the perspective of a pavement, the gradient of a sky.
The model in the tool below is called MI-GAN, and it was built explicitly to run on weak hardware, which is what makes it usable here at all. It weighs about 23 MB. Paint over the object, press the button, and the picture is repaired in this browser tab. Nothing is uploaded, and there is nowhere for it to be uploaded to: the only thing that travels is the model, coming down from this site the first time you ask for it.
Object eraser
Paint over the bin, the stranger in the background, the logo on the shirt, and a small network called MI-GAN works out what was most likely behind it. It was built for weak devices, which is why it is small enough to run here: about 23 MB of weights, fetched from this site only after you ask for them. Your photograph is read in this browser tab and stays in it.
A stroke is already painted over the signpost, as an example. Your first stroke replaces it.
Drag with the mouse or a finger. With the keyboard: the arrow keys move the brush, hold Shift to move faster, the space bar stamps a dab, Enter holds the brush down so the arrow keys draw a line, and Escape lifts it again.
or press Enter to pick one. JPEG, PNG or WebP.
Sample photo, drawn in this browser.
Cover the object and a little of what surrounds it. A mask that stops short of an edge leaves a rim of the object behind, and the model will happily build on that rim.
| What happened | Value |
|---|
What this deliberately cannot do. It fills the gap with something plausible, not with something true. A large hole in the middle of a face comes back as a face that never existed, and a hole where a house number was comes back as a wall. Regular patterns break visibly at the seam: tiles, brickwork, railings, and writing of any kind. It is at its best on a small object in front of grass, sky, road, water or a plain wall, and at its worst on anything a reader will look at closely. It is also not a tool for taking watermarks off pictures that are not yours, and that is the one use it is not offered for.
How the rest of the photograph survives. The model sees the picture at up to 1024 pixels on the long edge, padded out to a multiple of 64 because that is the only shape it accepts, and it answers with a whole picture rather than with a patch. That answer is scaled back to the original size and then copied in only where you painted. Everywhere else the original bytes are left exactly as they were, and the tool counts them for you after each pass. One detail decides whether any of this works: MI-GAN reads a keep mask, in which zero marks the hole and 255 means leave that pixel alone, so the painted area is turned round on the way in. The result becomes the picture you paint on next, so a second object can go in a second pass. Comparable tools exist and several of them are better. What is different here is the mechanism: the model comes to the photograph, the photograph never goes anywhere.
Paint generously, and past the edge
The single most common way to get a disappointing result is to paint carefully. A mask that follows the outline of the object exactly leaves a one or two pixel rim of that object standing, and the model, doing exactly what it was asked, treats that rim as real context and builds on it. A red bin masked precisely comes back as a red smear, because the only thing the model could see at the boundary was red.
Cover the object and a good margin of what surrounds it. Include the shadow. Include the reflection. The extra area the model has to invent is area it is good at inventing, whereas a rim it has to build on is a rim you will be looking at afterwards. If the first attempt leaves a ghost, undo it, paint wider and run it again.
The tool takes a second pass on its own result, so a picture with two problems in it can be done in two goes rather than one enormous mask. That is usually better anyway: two modest holes in different places are two easy problems, while one hole spanning half the frame is a hard one.
Painting with a keyboard, and why that is not a footnote
The brush can be driven with the mouse, with a finger on a touch screen, and with the keyboard. The last of those is the one that usually gets left out of tools like this, and leaving it out quietly excludes people from using them at all.
It costs very little to include. The arrow keys move the brush, holding shift moves it faster, the space bar stamps a single dab, Enter holds the brush down so the arrow keys draw a continuous line, and Escape lifts it again. That is five keys and roughly thirty lines of code, and it turns a tool that requires a pointing device into one that does not.
The undo stack helps here too, and it is worth saying what it stores. It keeps the strokes, not the pixels: a list of what was drawn rather than a series of snapshots of the canvas. That is why undo is instant regardless of how large the picture is, and why the mask can be re-rendered at full resolution from the same list even though you painted it on a smaller preview. A tool that stored canvas snapshots instead would use a great deal of memory to do a worse job.
The convention that decides whether any of it works
There is a detail in this model that is worth writing down, because it is counterintuitive, because it is not documented anywhere prominent, and because getting it backwards fails in a way that looks like something else entirely.
MI-GAN takes two inputs, the picture and a mask, and the mask is a keep mask. A value of 255 means leave this pixel alone. A value of 0 marks the hole to be filled. That is the opposite of what most people write on the first attempt, because when you paint over a bin you naturally think of the painted area as the marked area, and marked usually means one rather than zero.
Feed it the other way round and it does not throw an error and it does not return the original. It regenerates the entire photograph, keeping only the small region you painted, and hands back a hallucinated scene with your bin still sitting neatly in the middle of it. The first time you see it you assume the model is broken or the weights are wrong. It is neither. It did exactly what it was asked.
painted by the visitor -> the hole -> mask value 0
everything else -> keep as is -> mask value 255So the tool inverts on the way in, and there is a test that reads the tensor the model was actually handed and asserts that the painted centre is 0 and a far corner is 255. That is the kind of thing worth pinning down with a test rather than a comment, because a comment cannot notice when somebody flips a sign.
Why the rest of the photograph comes back untouched
The model does not return a patch. It returns a whole picture, and every pixel of it has been through the network, including the pixels far away from the hole that did not need anything doing to them. Those pixels come back very slightly different: not visibly, but not byte for byte.
For a repair tool that is not acceptable, so the result is composited rather than adopted. The model’s answer is scaled back to the original size and copied in only where you painted. Everywhere else the original bytes stay exactly as they were, and the tool counts them for you after each pass so the claim is checkable rather than asserted.
This matters more than it sounds. It means the tool can be run twice without the picture degrading, it means the untouched parts of a photograph keep whatever grain and detail they had, and it means the operation is local in the way a person expects a repair to be local. It is the same instinct behind stripping metadata by rewriting a container rather than re-encoding a picture, which is the approach the site’s EXIF tool takes for the same reason.
Why it is not done in tiles
Several of the other model tools on this site cut a large picture into tiles and process them one by one, which keeps memory bounded and lets a progress bar move. This one does not, and the reason is instructive.
Tiling is safe when a model only ever looks at a small neighbourhood around each pixel, because then a pixel in the middle of a tile gets the same answer it would have got in the middle of the whole picture. Inpainting is the opposite case. The whole point is that the model looks across the hole and out into the rest of the scene to work out what belongs in the middle. Cut that context into tiles and the model is reasoning about a hole whose surroundings you have just removed.
So the picture goes through in one pass, at up to 1024 pixels on the long edge, padded out to a multiple of 64 because that is the shape the model accepts. Progress is reported in named stages instead of tiles, which is less satisfying to watch and more honest about what is happening.
What it deliberately cannot do
It fills the gap with something plausible, not with something true. A large hole in the middle of a face comes back as a face that never existed. A hole where a house number was comes back as a wall. If the thing you removed carried information that the rest of the picture does not imply, that information is gone and what replaces it is an invention.
Regular patterns break visibly at the seam. Tiles, brickwork, railings, window frames and writing of any kind are the cases where a person spots the repair immediately, because human vision is unreasonably good at periodic structure and the model is only approximately good at it. It is at its best on a small object in front of grass, sky, road, water or a plain wall, and at its worst on anything a reader is going to look at closely.
It is also not a tool for taking watermarks off pictures that are not yours, and that is the one use it is not offered for. A watermark is a claim of authorship, and removing one from somebody else’s photograph does not change who took it.
Where it sits among the alternatives
Comparable tools exist and several of them are better, because they run larger models on hardware you do not have. The difference here is mechanical rather than qualitative: the weights come down to your browser once and the photograph never leaves the tab. For a holiday snap that is a curiosity. For a photograph of an interior, a document on a desk, a person who has not agreed to be uploaded anywhere, it is the whole point.
It is also worth knowing which problem you actually have. If the object is at the edge of the frame, the honest answer is often a crop rather than a repair, and if the frame then has the wrong shape for where it needs to go, changing the shape without cutting the subject is a different technique with fewer ways to go wrong. If the object is a person you cannot remove cleanly, the same family of models will at least let you separate them from the background, and if the object you wanted gone was a reflection or a highlight rather than a thing, the answer is usually a curve and not a model at all and treat them deliberately rather than pretending they were never there.
What the tool changes is not the standard of the result, which is variable and always will be. It changes who is allowed to try. Removing an object used to require either a steady hand with a clone brush or a willingness to upload the picture to somebody else’s computer. Neither of those is required now, and the second one stopped being required for reasons that have nothing to do with the model and everything to do with where it runs.
