LensErase is a mask-based image-erasure foundation model that adapts a text-to-image generator by injecting the erase mask through position encodings, trained on 101M source–mask–target triplets and refined with contract-guided rewards.
Object removal in images is a mess of narrow tools. One model handles general objects well but botches translucent watermarks. Another erases text but leaves shadow residue. A third preserves backgrounds but can’t handle a large foreground that covers 40% of the frame. In practice, a photo-editing product ends up stitching several specialized models together, each with its own failure modes.
The practical difficulty is twofold. First, training data: existing object-removal datasets contain roughly 100K–1M examples, which is small for teaching broad coverage. Second, supervision quality: if you ask a generative model to erase an object, the “clean” target is itself hallucinated, so the model learns from a fake ground truth. A prior line of work (ObjectDrop) addressed effects like shadows through counterfactual supervision but did not aim at a single general-purpose eraser.
LensErase, from Microsoft and academic collaborators, pushes for one model that covers general objects, visual text, watermarks, multi-object selections, and target-attached effects (shadows, reflections), across a wide range of mask sizes and image resolutions.
The core trick for data is addition-to-removal reversal. Instead of asking a model to erase something from a real photo (which produces a fake target), the authors start with a real photo, use an editing model to add an object to it, and then reverse the pair. The real photo becomes the clean target; the edited photo becomes the input; a recovered mask marks the inserted object. This gives real-image supervision at scale. Watermarks are composited with known alpha; posters are rendered from layered designs with known z-order; visual text uses OCR-verified partial edits blended back into originals. The result is LensErase-100M, with 101.07M triplets.
The model starts from Lens, a text-to-image backbone with two streams (one for text, one for the image being generated). Since erasure’s “condition” is itself visual (the source image), the authors initialize both streams from the image branch and discard the text-side weights. The source image is passed in intact, including the region to be erased, because the pixels under a translucent watermark or near a shadow carry useful signal.
To tell the model which pixels to erase, they introduce Mask-Aware RoPE. Normal Rotary Position Embedding (RoPE) gives each token a 2D position (row, column). They add a third coordinate that is 0 for unmasked source tokens, 1 for masked source tokens, and 2 for the noisy target tokens. No separate mask tokens, no extra embedding layers, mask information rides along in the position encoding.
Training uses Flow matching with a region-weighted loss: tokens inside a dilated mask get 5× the loss weight so the small edited region isn’t drowned out by the unchanged background. Then supervised fine-tuning on 140K curated examples (via LoRA). Then reinforcement learning via DiffusionNFT with a contract-guided reward.
The reward construction is the interesting post-training piece. For each image–mask pair, a VLM (GPT-5.6 Sol) first writes an “erase contract” before seeing any model output: what should disappear (targets T, attached effects E) and what must stay (protected objects O). All sampled candidates for that input are then judged against the same contract, so rewards stay consistent across rollouts.
contract = vlm_build_contract(src, mask) # T, E, O (once per input)
for j in range(num_rollouts):
cand = policy.sample(src, mask)
scores = vlm_score(cand, src, mask, contract) # Removal, Recon, Preserve, Clean
psnr_out = outside_fidelity(cand, src, mask)
base = weighted_sum(scores, psnr_out)
n0 = count_zero_removal_subcriteria(scores)
reward[j] = (0.5 ** n0) * base # halve for each removal subcriterion that failed
update_policy_with_diffusionnft(reward)
The 0.5 ** n0 gate matters: strong reconstruction or preservation cannot paper over a failed removal, because each zero on a removal subcriterion halves the whole reward.
At inference, TraceFusion compares the generated output with the source, finds where real changes actually happened (combining the input mask with residual-difference detection), and restores source pixels outside that region. This keeps VAE round-trip noise from drifting unchanged background.
On the authors’ own LensErase-Bench (500 cases, VLM-judged across Removal, Inpainting, Preservation, Cleanliness), the stage-wise ablation tells a clean story. HiFi Removal (fraction of cases with perfect removal and solid quality on the other three axes) rises from 65.80% after pretraining, to 70.80% after SFT, to 81.60% after RL, and 82.80% after TraceFusion. TraceFusion’s main contribution is on pixel fidelity: mean PSNR across the six public benchmarks goes from 28.70 to 30.82 dB, and LPIPS from 0.0816 to 0.0684.
The mask-injection ablation supports the Mask-Aware RoPE design: it reaches 65.80% HiFi Removal, versus 58.80% for a channel-concatenation variant and 51.40% for a token-concatenation variant, all at the pretraining stage. Initializing the condition branch from the text branch instead of the image branch collapses Inpainting score to 1.30 and HiFi Removal to 0%, so the authors’ choice to drop the text-branch weights is load-bearing.
Against 12 published baselines on six public benchmarks (OBER-Test, RORD-Val-343, RemovalBench, OmniPaint-Bench captured subset, AnimeEraseBench, TextEraseBench), LensErase takes top scores on PSNR, FID, LPIPS, SSIM, DISTS, CLIP-I and DINO across all six. Reward-component ablation: removing the Removal reward and gate together drops Removal from 94.95 to 43.35 and HiFi Removal from 81.60% to 38.80%, by far the biggest hit.
Two observations worth separating from the authors’ framing. Reference-free VLM scoring uses the same model family (GPT-5.6 Sol) that supplied training-time rewards and data curation, so the benchmark and the reward are not independent. And the quantitative comparisons are against open-source baselines; closed commercial tools (GPT-Image 2.5-Sunburst, Flux2-Erase) appear only in qualitative figures.
•
If you are building an image editor that needs one erasure component instead of a routing stack, this paper is the recipe. The architectural piece most likely to transfer: Mask-Aware RoPE adds spatial-mask conditioning without extra tokens, parameters, or embeddings, and it beats concatenation baselines by meaningful margins. Worth testing if you condition a generation model on any binary spatial signal (bounding boxes, segmentation hints, inpainting masks).
•
The addition-then-reverse data trick solves a specific problem: your “clean” training target is a real photo instead of a model hallucination. If you have a removal or editing task and your current pipeline uses a teacher model to produce ground-truth outputs, consider whether you can invert an easier task to get real-image targets instead.
•
The contract-first reward design is reusable beyond erasure. Pre-committing a VLM to a per-input rubric before scoring any candidate reduces inconsistency across rollouts in RLHF-style post-training. The removal-aware gate (multiplicative penalty per failed subcriterion) is a cheap way to prevent reward hacking when one axis matters disproportionately.
•
Prerequisite: the paper is a Microsoft-led effort using a proprietary Lens backbone, roughly 2 weeks of pretraining on 96 A100s, plus SFT and RL. Weights and the 100M dataset are not announced as released in the supplied text. Treat the full pipeline as a reference architecture rather than a drop-in artifact.
•
Both the dataset curation (SFT filtering, RL difficulty scoring, erase-contract construction) and the LensErase-Bench evaluator use GPT-5.6 Sol. VLM-judged metrics should be read as internally consistent, not as an independent oracle.
•
The six public benchmark gains are strong, but comparisons to closed commercial erasers are qualitative only.
•
Watermark and large-foreground training data is synthetic (alpha compositing, copy-paste), which may undercount the appearance distribution of real-world watermarks and real large occluders.
•
The authors note that scaling to 4K resolution is limited because the teacher editing models used for data construction are themselves unreliable at that scale.
•
The paper does not specify any public release of code, weights, or the 100M dataset in the supplied text; downstream users cannot verify or reproduce without further information from the authors.