Cutting a subject out of a photograph used to be twenty minutes of careful path work. A segmentation model now does a credible job in a few seconds. It is worth knowing what it is actually doing, because that explains its failure modes precisely.
What the model produces
A background-removal model does not identify objects and draw an outline around them. It performs semantic segmentation: for every pixel in the image, it outputs a value between 0 and 1 representing confidence that the pixel belongs to the foreground.
The result is a greyscale mask the same size as the image, white where the model is confident it is subject, black where it is confident it is background, grey where it is unsure. That mask becomes the alpha channel of the output PNG.
Architectures like U²-Net are built for exactly this: an encoder progressively downsamples the image to understand global context ("this is a person standing in a room"), and a decoder upsamples back to full resolution while pulling in fine detail from earlier layers via skip connections. The two-scale design is why these models get both the overall shape and the edges reasonably right.
Why edges are the hard part
For most pixels the answer is obvious. The interesting pixels are the boundary ones, where a single pixel contains both subject and background, this is a real physical property of how cameras sample light, not a modelling shortcut.
Getting those pixels right is a separate problem called alpha matting, and it is where quality differences between tools show up:
- Hair. Thousands of strands thinner than one pixel each. A hard mask cuts them off and leaves a helmet-like silhouette; a good soft mask preserves partial transparency and keeps them looking like hair.
- Fur and feathers. Same problem, more of it.
- Glass, smoke, water, veils. Truly semi-transparent, the "correct" alpha is somewhere in the middle, and models trained mostly on opaque subjects tend to guess wrong.
- Motion blur. A blurred hand has no defined edge to find.
What predicts a good result
Contrast between subject and background. A dark jacket against a dark wall is hard for the same reason it is hard for you, there is little signal to separate them.
A single, clear subject. Models are typically trained on images with one salient foreground object. A group photo, or a person holding a bicycle, produces inconsistent decisions about what counts as subject.
In-focus edges. Shallow depth of field that blurs the subject's outline removes the information the model needs.
Ordinary subject matter. These models are trained heavily on people, animals, products and vehicles. Unusual subjects, machinery, abstract objects, cluttered scenes, fall outside the distribution and results degrade.
Reasonable resolution. Most models process at a fixed internal resolution (often around 320–1024 px) and the mask is scaled back up. Very fine detail in a huge image can be lost simply because it never survived the downscale.
Getting better output
- Crop before removing. Less background clutter means fewer chances to be wrong, and the subject occupies more of the model's internal resolution.
- Improve contrast first if the subject is poorly separated from its surroundings.
- Always save as PNG or WebP. JPEG has no alpha channel, export a cut-out as JPEG and the transparency is flattened to white or black, and the work is lost.
- Check on both light and dark backgrounds. Fringing from the original background colour is invisible against white and glaring against black.
- Retouch rather than re-run. If 95% is right, fixing the remaining edge by hand is faster than hunting for a better model.
Running it on your own device
This is the part that has changed. These models are a few tens of megabytes and can be exported to ONNX and run in a browser through WebAssembly or WebGPU, using your own CPU or graphics card rather than a rented one.
That is what NextTool's AI background remover does. The first run downloads the model, then every image is processed locally. It is slower than a datacentre GPU and it means your product photography, portraits and personal pictures never get uploaded to a service that may retain them for training.
Expect a few seconds per image after the model loads, and expect a modern browser to be required, WebGPU support makes a substantial difference to speed.