emCompress-PRO
The allround compression solution
emCompress-PRO provides compression and decompression in any host application or embedded firmware, perfectly fitting most common use cases.
Overview
emCompress-PRO is a versatile compression library for most common use cases. It provides standard compression codecs and a zlib-like API for compression and decompression of input streams.
emCompress-PRO includes all compression and decompression codecs in source code, which enables usage in any host application and in any embedded firmware. In addition to the codecs, emCompress-PRO includes example application code for archive handling.
Key features
Compression and decompression of streams and files
Common codecs included: DEFLATE, LZMA, LZPJ, LZJU90
Easy-to-use zlib-like API
Source code for host and embedded applications
Codecs & performance values
emCompress-PRO includes well-defined, commonly used compression codecs. All codecs are implemented according to their specification or RFC, which enables interoperability with other tools and libraries.
DEFLATE
DEFLATE is the lossless compression stream format originally developed for and with the ZIP file format. It uses a combination of LZ77 and Huffman coding. DEFLATE is specified in RFC 1951.
| Codec | Compressed size [1] | Compression speed [2] | Decompression speed [2] | Compression RAM | Decompression RAM |
|---|---|---|---|---|---|
| DEFLATE(128,3,258) | 133.531 KB | 236 | 111 | 40.934 KB | 12.672 KB |
| DEFLATE(256,3,258) | 128.562 KB | 261 | 107 | 41.559 KB | 12.855 KB |
| DEFLATE(512,3,258) | 121.521 KB | 296 | 100 | 42.809 KB | 13.184 KB |
| DEFLATE(1k,3,258) | 116.601 KB | 375 | 96 | 45.309 KB | 13.738 KB |
| DEFLATE(2k,3,258) | 114.010 KB | 504 | 94 | 50.309 KB | 14.785 KB |
| DEFLATE(4k,3,258) | 112.430 KB | 742 | 92 | 60.309 KB | 16.832 KB |
| DEFLATE(8k,3,258) | 111.637 KB | 1,078 | 91 | 80.309 KB | 20.879 KB |
| DEFLATE(12k,3,258) | 111.396 KB | 1,768 | 92 | 100.309 KB | 24.902 KB |
| DEFLATE(16k,3,258) | 111.310 KB | 1,459 | 91 | 120.309 KB | 28.926 KB |
| DEFLATE(32k,3,258) | 111.313 KB | 1,011 | 90 | 200.309 KB | 44.969 KB |
LZMA
LZMA (Lempel–Ziv–Markov chain algorithm) is the lossless compression algorithm which was first used in the 7z file format. This algorithm uses a dictionary compression scheme similar to the LZ77 algorithm and features a high compression ratio and a variable compression-dictionary size, while still maintaining decompression speed similar to other commonly used compression algorithms.
| Codec | Compressed size [1] | Compression speed [2] | Decompression speed [2] | Compression RAM | Decompression RAM |
|---|---|---|---|---|---|
| LZMA(1k,0,0,0) | 108.676 KB | 282 | 87 | 6.010 MB | 5,060 B |
| LZMA(1k,3,1,1) | 103.675 KB | 289 | 91 | 6.010 MB | 27.613 KB |
| LZMA(2k,0,0,0) | 106.270 KB | 363 | 91 | 6.015 MB | 6,084 B |
| LZMA(2k,3,1,1) | 101.701 KB | 378 | 93 | 6.015 MB | 28.613 KB |
| LZMA(4k,0,0,0) | 104.640 KB | 582 | 85 | 6.025 MB | 8,132 B |
| LZMA(4k,3,1,1) | 100.450 KB | 593 | 90 | 6.025 MB | 30.613 KB |
| LZMA(8k,0,0,0) | 103.592 KB | 980 | 87 | 6.044 MB | 11.941 KB |
| LZMA(8k,3,1,1) | 99.742 KB | 992 | 87 | 6.044 MB | 34.613 KB |
| LZMA(16k,0,0,0) | 102.706 KB | 1,821 | 82 | 6.083 MB | 19.941 KB |
| LZMA(16k,3,1,1) | 99.198 KB | 1,785 | 85 | 6.083 MB | 42.613 KB |
| LZMA(32k,0,0,0) | 102.059 KB | 2,389 | 97 | 6.162 MB | 35.941 KB |
| LZMA(32k,3,1,1) | 98.808 KB | 2,550 | 86 | 6.162 MB | 58.613 KB |
| LZMA(64k,0,0,0) | 100.030 KB | 4,182 | 89 | 6.318 MB | 67.941 KB |
| LZMA(64k,3,1,1) | 97.180 KB | 4,274 | 83 | 6.318 MB | 90.613 KB |
| LZMA(128k,0,0,0) | 99.840 KB | 5,081 | 85 | 6.630 MB | 131.941 KB |
| LZMA(128k,3,1,1) | 97.067 KB | 5,026 | 86 | 6.630 MB | 154.613 KB |
| LZMA(256k,0,0,0) | 99.851 KB | 3,001 | 83 | 7.255 MB | 259.941 KB |
| LZMA(256k,3,1,1) | 97.080 KB | 3,212 | 87 | 7.255 MB | 282.613 KB |
LZJU90
LZJU90 defines the transfer encoding for MIME compliant mail systems. It uses prefix coding of a length and distance that leaves literals uncompressed.
| Codec | Compressed size [1] | Compression speed [2] | Decompression speed [2] | Compression RAM | Decompression RAM |
|---|---|---|---|---|---|
| LZJU(1535,2,255,9,1,10) | 121.459 KB | 363 | 56 | 9.898 KB | 1,616 B |
| LZJU(1535,3,256,9,1,10) | 127.421 KB | 291 | 48 | 9.906 KB | 1,616 B |
| LZJU(3583,2,255,9,1,11) | 117.644 KB | 684 | 54 | 19.898 KB | 3,664 B |
| LZJU(3583,3,256,9,1,11) | 122.245 KB | 539 | 48 | 19.906 KB | 3,664 B |
| LZJU(7679,2,255,9,1,12) | 116.253 KB | 1,139 | 54 | 39.898 KB | 7,760 B |
| LZJU(7679,3,256,9,1,12) | 119.478 KB | 878 | 49 | 39.906 KB | 7,760 B |
| LZJU(10751,2,255,9,2,13) | 117.183 KB | 1,568 | 53 | 54.898 KB | 10.578 KB |
| LZJU(10751,3,256,9,2,13) | 119.294 KB | 1,256 | 50 | 54.906 KB | 10.578 KB |
| LZJU(15871,2,255,9,1,13) | 115.639 KB | 2,521 | 55 | 79.898 KB | 15.578 KB |
| LZJU(15871,3,256,9,1,13) | 117.665 KB | 1,635 | 50 | 79.906 KB | 15.578 KB |
| LZJU(32255,2,255,9,1,14) | 115.902 KB | 4,308 | 55 | 159.898 KB | 31.578 KB |
| LZJU(32255,3,256,9,1,14) | 116.756 KB | 2,485 | 52 | 159.906 KB | 31.578 KB |
| LZJU(43519,3,256,9,2,15) | 117.021 KB | 3,944 | 49 | 214.906 KB | 42.578 KB |
| LZJU(43519,2,255,9,2,15) | 116.456 KB | 5,337 | 55 | 214.898 KB | 42.578 KB |
| LZJU(65023,2,255,9,1,15) | 115.133 KB | 6,736 | 58 | 319.898 KB | 63.578 KB |
| LZJU(65023,3,256,9,1,15) | 114.745 KB | 5,345 | 52 | 319.906 KB | 63.578 KB |
[1] Uncompressed input size: 182.695 KB
[2] ns/Byte on Intel i7 @ 4.6GHz, only for comparison between codecs
API use
emCompress-PRO provides a simple zlib-like API, for incremental process of input. This interface enables the use of compression and decompression of data on the fly. It can be used to process data in chunks as they are received from a remote system instead of requiring that all data be available at once. It can also be used to process data and output chunks into a small buffer for further processing instead of requiring a large output buffer that can hold all of the compressed or decompressed data at once. Chunk-wise input and output can be combined to ensure minimum memory buffer requirements.
Code example
/*********************************************************************
*
* Decompress()
*
* Function description
* Decompress DEFLATE or LZMA blocks.
*
* Parameters
* pFile - Pointer to input file.
* pAPI - Pointer to decompression API.
* pParas - Pointer to decompression parameters.
* Size - Size of data to decompress.
*/
void Decompress(FILE* pFile, const CX_DECODE_API* pAPI, const CX_PARAS* pParas, int Size) {
CX_DECODE_CONTEXT C;
CX_STREAM Stream;
int Status;
U8 ByteIn;
U8 ByteOut;
//
// Prepare decoder and stream.
//
Stream.AvailIn = 0;
Stream.AvailOut = 0;
Status = CX_DECODE_Init(&C, pAPI, &_StaticAllocator, pParas);
//
for (;;) {
//
// Read next byte from input.
//
if (Stream.AvailIn == 0 && Size > 0) {
fread(&ByteIn, 1, 1, pFile);
Stream.pIn = &ByteIn;
Stream.AvailIn = 1;
Size--;
}
//
// Reset output buffer.
//
Stream.pOut = &ByteOut;
Stream.AvailOut = 1;
//
// Process stream.
//
Status = CX_DECODE_Process(&C, &Stream, Size == 0 ? CX_FLUSH_END : CX_FLUSH_NONE);
if (Status < 0) {
printf("Failed to decode: %s\n", CX_GetErrorText(Status));
exit(100);
}
if (Status == 1) {
break;
}
}
//
CX_DECODE_Exit(&C);
}Use cases
emCompress-PRO is the allround solution for compression and decompression.
Speed up firmware updates
Modern products, in particular IoT devices, require the ability to update their firmware in the field. Most of these devices implement over-the-air (OTA) updates through their main communication channel. When a device is connected via WiFi, available bandwidth is not an issue, but other channels such as Bluetooth or Zigbee might have limited bandwidth. Also, updates via wired connections often have bandwidth restrictions, e.g. the popular CAN bus.
To speed up the update process, lossless compression of firmware images reduces the amount of data to be transferred to the device. emCompress-PRO enables firmware updates with compressed images. Its interface is suited to receive a compressed data stream and to decompress it on the fly for the update module, which processes the decompressed data afterward.
Learn more at:
https://casestudies.segger.com/size-matters-using-emcompress-to-accelerate-firmware-updates-via-can-bus/
Minimize the required memory footprint for asset storage
Most systems include digital assets for their user interface. Assets can be images, videos and fonts for a graphical user interface (GUI) on a display, sound files to be output on a speaker, web pages for a web or app interface, or any other static data. Storage space for static data, such as an external flash, can be limited. To fit more data into available memory, or even to design hardware with smaller memory, digital assets can be stored in compressed form.
emCompress-PRO enables on-demand decompression of data when it is needed. Data can be decompressed into small chunks so the entire asset doesn’t have to be held in memory before being deployed.
Process data provided by third-party compression tools
Connected devices might receive compressed data from external sources. When a device does not include its own software to manage such data, the system might need to be designed to process data created by third-party tools.
emCompress-PRO implements popular codecs, such as DEFLATE and LZMA, which provides compatibility with commonly used compression tools. A system receiving external data can select the decompressor on demand making it ready for most customer use cases.
Data is not always compressed in a single block but may be packaged in an archive. While emCompress-PRO implements compression codecs, it also comes with sample code to handle and process common archive formats. An input archive stream can be decompressed into structured files when a file system like emFile is used.
Compress data on different kinds of targets
Most use cases require decompression on the target system. Compression of data can be done with off-the-shelf tools, such as the popular zip file archiver, or integrated in a tool or framework which also creates the data, such as a firmware update utility.
Besides data decompression emCompress-PRO provides compression functionality as well and is not limited to embedded targets. Both parts of the codecs are written in plain C and can be integrated in firmware images as well as host applications.
Media gallery
emCompress — Example application | 2:43 min
emCompress-ToGo — Intro | 5:36 min