# Image compression benchmark 2026 — methodology

CompressMyPhotos is the publisher of this benchmark and is one of the tools tested. Every tool was tested using the same source files, target sizes and documented methodology. Raw results are provided so the findings can be reproduced independently.

Seven tools, four unchanged CC0 synthetic fixtures, 20 / 50 / 100 KB normal UI values: 84 planned, 82 actual approved outputs, two unavailable ExactKB minimum-scale refusals. Each case used the original fixture, never another tool's output. Unrelated options retained recorded defaults. No CAPTCHA/protection/rate-limit bypass or private API was used. Three SIR 20 KB outputs were collected separately on 2026-10-01 after ordinary visible ad Close dismissal; original failed download-event attempts on 2026-09-30 remain archived. They are later observations, not the missing original downloads. Browser version and OS details are in environment.json; image decoder versions are recorded per case. Egress country was not measured.

## Units

“KB” is ambiguous across software and websites: a decimal kilobyte is 1,000 bytes, while some interfaces use “KB” for 1,024 bytes (properly KiB). The normal UI received the numbers 20, 50 or 100. The strict decimal benchmark caps remain 20,000 / 50,000 / 100,000 bytes; the independent binary-equivalent caps are 20,480 / 51,200 / 102,400 bytes. A decimal overage does not by itself establish that a tool failed its own stated target. Exact measured bytes—not rounded UI sizes—let readers apply the limit required by their destination. Unspecified means no explicit definition was found in preserved evidence, not that a tool is decimal. Neither matching byte patterns nor rounded size displays establish a declared unit basis.

CompressMyPhotos explicitly documents decimal bytes. KBSize explicitly states 1 KB = 1024 bytes. Pi7, JPEG Optimizer, Simple Image Resizer, ExactKB and SmallJPG are unspecified in preserved evidence. That does not establish decimal semantics. All 16 strict decimal overages fit the binary-equivalent caps. No promised-target failure is inferred from those overages. Exact file bytes are in results.csv and results.json.

## Metrics

The actual downloaded bytes were hashed with SHA-256 and independently decoded using Sharp 0.35.4. Oriented format, dimensions and alpha were measured. Width/height retention is output/source × 100; pixel retention is output pixel count/source pixel count × 100.

SSIM uses the published single-scale Wang et al. formulation (2004), 11 × 11 Gaussian windows, sigma 1.5, K1 0.01, K2 0.03, peak 255, population moments and valid windows only. The original reference is auto-oriented and resized to the output's exact dimensions with Lanczos3 (fit fill, fastShrinkOnLoad false); the output is not resized. Both are decoded to sRGB and composited over #808080. Luminance is 0.299 R + 0.587 G + 0.114 B. RGB PSNR uses the three composited channels and peak 255; identity is null dB with an explicit identity flag. No first-frame scoring of animation. See METRIC-DEFINITIONS.json and metrics.mjs. Formula source: https://ece.uwaterloo.ca/~z70wang/publications/ssim.pdf

Alpha is measured separately: transparent and partial-alpha presence and mean absolute difference against the normalized reference. Preserved alpha does not imply an identical channel. Missing outputs retain null measurements. SSIM is not document readability or a calibrated human-quality scale; smaller reference dimensions can yield high similarity despite substantial loss of useful detail. Human review found the SIR document-20KB PNG unreadable as a normal document at 117 × 82, despite meeting the byte cap and scoring about 0.9927 normalized SSIM.

## Descriptive summaries and limitations

Coverage uses completed/planned. Cap compliance uses completed outputs only. Means/medians exclude unavailable and null measurements; quartiles use R type 7 interpolation at (n−1)×p. Tool/fixture order follows the fixed registry, never a quality ranking. Four synthetic fixtures, two input formats, differing tool defaults and mixed collection dates do not establish population-wide performance. No composite scores or privacy comparison. Third-party requests or marketing claims do not establish processing location/retention. Historical CMP screenshot/filename gaps remain disclosed. Repeated Pi7 byte patterns, reused filenames and the identical CMP/KBSize photo-50KB hash are confirmed file observations with no inferred cause.

## Reproduction

Use README.md and reproduce.mjs. Original fixtures and approved downloaded outputs are copied byte-for-byte. Comparison graphics are labeled derivatives; they do not replace measurements or exact files.
