Add low-memory JPEG decode path#1
Draft
mariusandra wants to merge 11 commits into
Draft
Conversation
- pixie/decodebudget: runtime per-decode memory budget (replaces the compile-time frameosEmbedded consts); decoders raise catchable PixieErrors instead of exhausting memory - jpeg: decode plan checked against the budget before any image-sized allocation; progressive JPEGs now use target-sized channel masks; sampling resolution clamps itself to the budget, trading sharpness for a successful decode - jpeg: streaming decode (decodeJpegStreamScaled/Into) pulls the compressed input through a 32K sliding window so files never need to be fully buffered; bit-identical output across the test suite - jpeg: decodeJpegInfo probe + jpegDecodeIntermediateBytes for pre-decode budget planning - jpeg/png: ScaledDecodeFit (stretch/cover/contain) on all scaled decode paths, enabling aspect-correct decode-into-canvas; cover inflates mask sampling density for the cropped region within budget - png: budget check at IHDR; inflated scanlines released before the pixel seq allocation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
seekEntropyMarker could set state.pos behind the sliding window start on a long 0xFF run in damaged entropy data; clamp to windowStart so recovery matches the buffered decoder instead of failing the whole decode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unfiltering allocated a second scanline-sized buffer, putting the decode plan for a canvas-sized RGBA PNG at pixels + 2x scanlines (4.5MB for 480x800). Compacting the [filter byte][row] stride in place drops the plan to pixels + scanlines (3.0MB), which fits the decode budget on ESP32-class devices. Interlaced images keep the per-pass copy and the old plan formula. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Non-interlaced PNGs now decode row by row through zippy's streaming inflate (pinned to the FrameOS fork): scanlines are unfiltered against a single previous-row buffer and written straight into the pixel data, so the whole-image inflate buffer disappears. Full decodes plan pixels + a fixed ~64KB; decodePngScaled/Into sample rows on the fly and never allocate the full-size pixel buffer at all, so a huge PNG can scale into display bounds like a streamed JPEG. Interlaced and 16-bit-scaled decodes keep the buffered path. Verified pixel-identical to the previous decoder across the whole pngsuite corpus, plus differential streamed-vs-buffered scaled decode tests and tightened budget assertions (480x800 RGBA: full decode within 2MB, scaled-into within 256KB). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A transitive URL requirement breaks nimble's CI resolution (two zippy sources for one package name); frameos.nimble pins the FrameOS zippy fork at the root instead, the same proven pattern used for pixie. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI's nimble cannot reliably resolve a second forked package (transitive or root URL requirements both produced incomplete nimble.paths), so the streaming inflate now lives inside pixie as a self-contained module (huffman machinery vendored from zippy 0.10.16, MIT) raising PixieError directly. The dependency graph returns to the CI-proven shape: one forked package (pixie), stock guzba zippy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The streaming inflate only took one contiguous buffer, so PNGs that split their compressed data across many IDAT chunks (libpng emits 8K chunks; a 2.6MB gallery PNG has 326) were concatenated into a second multi-MB allocation before decoding — the allocation that OOMed streamed decodes on fragmented ESP32 PSRAM. The bit reader now consumes a list of InflateSegments, crossing chunk boundaries byte-wise and keeping the fast word-load path within a segment. Verified: a real 2.6MB multi-IDAT PNG stream-decodes into a 1200x1600 target under a 1MB decode budget, pixel-identical to the buffered path; IDATs re-chunked at 1..8192 bytes decode identically; PNG fuzzer clean over 10k iterations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
decodePngScaledInto now accepts a list of InflateSegments, so a PNG whose bytes arrived in fixed-size download chunks decodes without ever being assembled into one contiguous buffer: the segmented parser validates every chunk CRC across boundaries (incremental crc32) and hands IDAT spans to the segmented inflater as sub-slices. Non-interlaced <=8-bit images stream scanlines into the target; interlaced/16-bit fall back to coalescing plus the buffered decode. streamIdatRows now takes inflater segments directly, shared by the contiguous and segmented paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Needed this to be able to load JPEGs on an ESP32...