Otsu et al. (2018), part 1

Otsu et al. (2018) [1] is the second spectral upsampling method I’m adding to Colour. It’s a data-driven method, which not only produces reflectances matching the given color exactly, but also that are natural and resemble real, measured reflectances. This is in contrast with, for example, Jakob and Hanika (2019) [2], which I worked on in June and uses a mathematical model.

At the moment the code is split in two parts. The feature branch in my fork implements the method using the dataset published with the article. Creating new datasets is done by this program and will be covered in depth in a later article. Work is mostly done, I just need to integrate the program into Colour and complete this series of blog posts.

1. Reconstruction

Spectral reconstruction in this method is very simple and straightforward, thus also remarkably fast to compute. Let be the reconstructed reflectance vector:

where are three basis vectors and is the dataset mean. The original code used spectra discretized at 36 points (at intervals), making the vectors 36-dimensional. Since the four vectors are known (precomputed ahead of time), reconstruction is simply a matter of computing the weights . First rewrite the equation in a matrix form, treating vectors as column vectors:

In a previous article I’ve already shown how to compute CIE 1931 XYZ tristimulus values given a reflectance. Written in a matrix form, with being a vector of the tristimulus values:

The term appears because of discretization. Combining the two equations gives:

A quick note about the dimensions of . The matrix of cmfs is three 36-dimensional row vectors stacked vertically, a 3x36 matrix, while the basis matrix is three column vectors stacked horizontally, a 36x3 matrix. Multiplying the two yields a 3x3 matrix, which can be inverted (degenerate cases aside).

Solving this equation is very simple. Move to the left side and left-multiply both sides by the inverse of .

Knowing the weights is enough to do the reconstruction. An interesting thing to note is that and could be pre-computed (for a given illuminant and cmfs), making the reconstruction process potentially extremely fast. Also, an RGB-to-XYZ matrix could be pre-multiplied into to speed up upsampling of RGB data.

2. Notes

This method works on discrete spectra. The range and resolution are fixed and determined by the pre-computed dataset. On the other hand, this method could theoretically also be used for emissive spectra, so it can be quite versatile.

While reconstruction is very simple, coming up with good basis vectors is not. It’s a complex and time consuming process that I’ll cover in the next (at least) two parts.