As an art platform serving image content to thousands of users daily, it is in our duty to serve and display it in the best possible way. We have already deployed our own IPFS infrastructure, but we wanted to take this one step further after observing some issues in the way we used to handle medias.

Demonstration of the update

Before diving into our process, following is a quick demonstration of the results of the update. Note the blurred placeholder while images are loading and how fast images are loaded now.

Problems observed before the update

So far, our strategy has not been optimal. Our signer module generates 2 assets per token:

• the high-quality one, captured from the code as specified by the artist
• a 300x300 downscaled version, so called thumbnail

Having only 2 assets to dispose of cannot possibly cover all the use-cases we face when displaying content.

Low thumbnail quality for some projects

When we want to display a small version such as in the explore feed, we must display the 300x300 thumbnail. This reduces the network footprint, but also reduces the output quality as demonstrated here on some high frequency images:

This gets even worse for retina displays, as low resolution artifacts are way more apparent due to the pixel density of those screens.

High costs for our users

On the contrary, when we want to display high-quality files, such as when browsing the iterations of a project, we must display the high-resolution image. This is fine in most cases, but can result in very high payloads for some projects and high CPU/GPU overhead to display such assets. Those big files have to get processed by the browser one way or the other before they can reach the screen, and so they have to be loaded somewhere in the memory. We noticed some lags when browsing some collections because of that.

Layout shifts

The website had no way to know an image details before downloading it entirely. This is why we could observe some layout shifts when images were loaded, particularly noticeable for projects with various output sizes.

Unfortunately we had no way to solve this, because the image resolution is not stored in the token metadata. So we had to compute this data at some point of our pipeline.

High costs for us

Serving very high-quality assets is very expensive. It is our biggest cost.

Imagine hundreds of people scrolling through the iterations of A Bugged Forest every day, we ended up serving terabytes of data through our CDN.

Our solution

We needed a solution which would solve all of the issues mentioned above, in the most optimal way. After considering different tools and services (cloudinary, imgix, imgproxy among others), we finally decided to opt for our own solution leveraging AWS infrastructure. We opted for a 2 step solution:

• an endpoint resizing images on-the-fly, cached on our CDN (cloudfront)
• a worker computing image informations (width, height, 12x12 placeholder) while indexing, seeding this data into our database

As soon as our indexer finds an image media, it saves it into our database as unprocessed (an IPFS pointer is saved with empty informations). Our image processing worker will keep searching the database for those unprocessed medias, and as soon as it finds some, it will look for them on our IPFS cluster, find their width/height/mime and compute a 12x12 placeholder we can display on our frontend while we fetch the actual resource.

The worker is very fast, it takes less than 6 seconds between an unprocessed image being added to the database and it being processed. It was written in Golang for efficiency.

We picked this strategy because it allows us to:

1) Know the image details as soon as we get the data from the API, allowing the front end to display a placeholder at the correct resolution while we fetch the actual media.

2) Request images at an optimal size based on the screen space, while hitting our cache as often as possible (ideally, we want to resize the image once and store it for fast retrieval).

Finally, we make sure to monitor changes in the viewport space allocated to an image to fetch a higher resolution if required, ensuring an ideal adaptative display quality when browsing the website, in any circumstance.

Notice how a higher resolution is loaded when increasing the card size, and stays loaded even if we resize it down (no need to fetch a lower res version):

Results

We will need some further monitoring of this optimisation, but we have already seen major improvements to the problems listed above.

Increase in thumbnail quality

We’ve greatly improved the quality of the content served when browsing the platform, especially for older tokens with a 256x256 thumbnail resolution. Our resize module uses Lanczos kernels with a = 3, known for producing high-quality image resampling.

This improvement can be observed easily for images with high frequency details, such as the thumbnail of Garden, Monoliths (what you see in the background is the actual thumbnail stretched to fill the space).

Network improvements

Payloads are now optimized for browsing on any device, and are highly improved for the pieces with very high-resolution outputs.

For loading 10 iterations of Bugged forest, we’ve reduced the payload by a factor of ~60

General improvements in image quality

When we used to send a 6000x6000 image to be displayed on a 500x500 area, browsers had to resize the images themselves. Usually, they use a bilinear filter as their default scaling strategy. This is because it's quite a fast filter, but also it produces outputs of a lesser quality than Lanczos3.

Now, a first high-quality resize is done by our services, and for a 500x500 area the browser will receive a 512x512 image. It will then apply its bilinear filter on the image, but the overall quality will be better because more details will be preserved by the first downscaling we have applied.

Notes

Source image file

On every project & token page, we will display the original file, ensuring that those highest resolution images can easily be retrieved if needed.

3rd party apps

Our API now exposes a new media field for entities which hold a media (articles for their thumbnail, users for their avatar and projects/tokens for their captures). All of these fields are exposing the same model, Media Image. For instance:

query TestMedias {
  user(name: "fxhash") {
    avatarMedia {
      cid
      placeholder
      width
      height
    }
  }
  generativeToken(id:0) {
    captureMedia {
      cid
      placeholder
      width
      height
    }
  }
  objkt(id:0) {
    captureMedia {
      cid
      placeholder
      width
      height
    }
  }
  article(id:0) {
    thumbnailMedia {
      cid
      placeholder
      width
      height
    }
  }
}

Api response:

{
  "data": {
    "user": {
      "id": "tz1fepn7jZsCYBqCDhpM63hzh9g2Ytqk4Tpv",
      "avatarMedia": {
        "cid": "QmURUAU4YPa6Wwco3JSVrcN7WfCrFBZH7hY51BLrc87WjM",
        "placeholder": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAIAAADZF8uwAAABA0lEQVR4nKyQoa6zMBTH+bpmazLRZWJubsteYGJ+FonhMXgCHgDFGxBSh8EgeA1EU1NFIEAIDmhaoF92mbm5N5m5P3PML+d//gdqrY1PgI/GN2kcx67rpJS/WPoLzrlt26ZpBkGgfwANw1BKpWlKKfV9/3q9Msb2+z1CqGma2+0GIXzFUUoJIWVZhmGolEqSxHEc13UJIe9aWmshhOd5j8eDcz5NU57n9/v9crlkWbbGvTbtdjuM8Xa7PZ1Om82mKAoAAEKIMbbeDdeBMT6fzwAAIUQURZZlHQ6HOI6fz+fxeHy36/u+bdtlWeZ5rut6GAYhRFVVUkqt9b8/+/j/AAAA//+gqZ6+Kr2MsgAAAABJRU5ErkJggg==",
        "width": 260,
        "height": 260
      }
    },
    "generativeToken": {
      "captureMedia": {
        "cid": "QmW6WBRx5M69kPx6CbSjsS4fcTVy21117pwwsU8iAMhBqX",
        "placeholder": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAIAAADZF8uwAAAAlElEQVR4nLSQO47CQBBEq6ZnPeNftA5WXg6AfP+zWBwBCRPhALCZIrAjIssSnbTU/aSnKocN8z3IgQ78ONq6zAjmeV74jMKkl3POm08pAfAL1B275rcxs8qH8/UiY4xxGIb+1EtadbdxjCFkPmP4aQ//7V9bl7WUlu+qJ1kVpSQA9+dDEkkC0zxvSba7gv3QOwAA//+G5CceURGWrgAAAABJRU5ErkJggg==",
        "width": 800,
        "height": 800
      }
    },
    "objkt": {
      "captureMedia": {
        "cid": "QmP1fDeWcy9jF4BcXLMyZvfAMjohtTCBCboJFEHhLGxvYU",
        "placeholder": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAIAAADZF8uwAAABKElEQVR4nCSQW2ocQQxF9awuu50HMdn/bvyTbQSHBPJiPDNdXdKV6Rl9SUgHjq59e3l5/fnr998/EZGZYwxTZdHWeibGGM/PX+zH99fT+S0z+Vb3RoiABGpdVyRk266qCqCqVLW1pqpEVAUR7n1pS7OIQBVVyW3XWnP3fd9vpyoizGTCosqeiYMutwNTtSrqvQExZ9hIYfAYk4RE9fp24YO+W4W5FcpofmbPh8eM83mLXVQyIUKmZuqP/sDFFvqvLy6kX2P973Sa26dlyUz3HkgCM8QmLnmVJ/M1JAgnFBcx8WVcDzn2D+1JROR4iERQMgKZiVSVZspMrBg0rPelqlK4qjiSrIho7juKCpjAtm3i7qpaTCODiT+SMR2ZZ4SZER3DewAAAP//x0a+ITln6I0AAAAASUVORK5CYII=",
        "width": 800,
        "height": 800
      }
    },
    "article": {
      "thumbnailMedia": {
        "cid": "QmSRNZmzpbvXPBu7d9wyBncBQJAWAEZSwBrLoceVUXeSqy",
        "placeholder": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAGCAIAAAB4jOjWAAAASUlEQVR4nKyOsQ3AQAjEgH8xBfuPxRR0CDlS0iTVN3FzzcnyBuSEHR8isp8BMlNVgbWW3kSEmX1M7j4zQFV199ukvzVdAQAA//96WxrdSaFUrwAAAABJRU5ErkJggg==",
        "width": 1280,
        "height": 720
      }
    }
  }
}

We hope this will help developers who want to improve the quality of 3rd party applications related to fxhash.

Final words

Although this may seem like a minor update, this will undoubtedly improve a lot the browsing experience and the quality of the representation of the pieces.

We've put a lot of efforts in researching and selecting the solution we thought would be the best to support the years to come.

Other global-platform improvements will come in the upcoming months.

Thank you for reading through this, and have a good day.