TL;DR: In this article, we explore how to compress data from JSON format to a tiny 2KB UintArray. By converting objects into arrays and using the msgpack-lite.js library, we achieve efficient data compression.

Link to my project on GitHub — https://github.com/xsfunc/chubriks. You can use it as template for your project on fxhash. The result of my project was a SVG file, but it can also be adapted to the p5.js for example.

Reactflow.js

Before we dive into data compression, let's briefly introduce ReactFlow.js. It is a powerful library for building node-based editors in React applications. With ReactFlow.js, you can create versatile editors with features like draggable nodes, connectors, and customizable node types. In this project, I have used it to design a unique node editor that includes various functionalities such as a color palette, gradient palette, SVG effect, head options, face options, and background settings.

Data Accumulation in JSON Format

I have organized the data into nodes that represent different design elements like colors, gradients, SVG effects, and more. As users manipulate these nodes, the data for each node accumulates in JSON format. This JSON data contains all the information required to recreate the user's design.

Data Compression Step 1: JSON to Array Conversion

To begin the compression process, we first convert our JSON data into arrays. Converting JSON objects into arrays reduces the overhead of property names and streamlines the data structure. This step is crucial for efficient compression.

Code example of simple functions from object to array and from array to object.

export const serializer = {
  base: (options)=> [
    options.patternType,
    options.id,
    options.rotate,
    options.scale,
    options.strokeWidth,
    fillingApi.serialize(options.color1),
    fillingApi.serialize(options.color2),
  ],
}

export const deserializer = {
  base: (data)=> ({
    patternType: data[0],
    id: data[1],
    rotate: data[2],
    scale: data[3],
    strokeWidth: data[4],
    color1: fillingApi.deserialize(data[5]),
    color2: fillingApi.deserialize(data[6]),
  }),
}

Data Compression Step 2: Utilizing msgpack-lite.js

Now comes the magic of data compression. We employ the msgpack-lite.js library, a lightweight implementation of the MessagePack binary serialization format. MessagePack is a binary format that efficiently represents data structures in a highly compressed form. It is especially useful for web applications where minimizing data size is essential.

// encode from JS Object to MessagePack (Buffer)
const buffer = msgpack.encode({"foo": "bar"});
 
// decode from MessagePack (Buffer) to JS Object
const data = msgpack.decode(buffer); // => {"foo": "bar"}

Using msgpack-lite.js, we encode our data into a compact binary format. This compression significantly reduces the size of our data payload. Then use byte fxparam to store all data.

The Results: Achieving a Tiny 2KB File Size

By implementing these data compression techniques, we can achieve astonishing results. What was once a sizable JSON object with all the design data has now been transformed into a minuscule binary representation, all while preserving the integrity of the information.