GSoC Blog

Otsu et al. (2018), part 3

In the previous post I explained how PCA can be used to create a convenient basis for representing real-world reflectivities. The procedure uses a set of measured spectra to compute an orthonormal basis, sorted by variance or “statistical significance.” All but the three most significant vectors are then truncated. I called the difference between the actual spectrum and the reconstructed the error term $\pmb\epsilon$.

Naturally the smaller the error, the better. Because $\pmb\epsilon$ is a vector, the authors chose the Euclidean norm (2-norm or L2) to represent the (scalar) error:

$$\epsilon =  \sqrt{\sum_i \pmb\epsilon_i^2}$$

To get an idea of how well the method works in the general case, the authors would sum up reconstruction errors of all spectra in the dataset. I call this the total error $E$:

$$E = \sum_i \epsilon|_{\mathbf s_i}$$

where $\epsilon|_{\mathbf s_i}$ denotes the reconstruction error of $\mathbf s_i$, the $i$-th measured spectrum.

Using just one basis for everything isn't ideal. The picture below shows reconstructions of a few colorchecker spectra. While by definition XYZ tristimulus values (and perceived colors) match perfectly, the spectra aren't that close.

Reconstructions of a few colorchecker spectra.

→

2020/07/26 14:46 · Paweł Redman

Otsu et al. (2018), part 2

The previous post mentions three basis vectors and a “dataset mean.” Here I explain how to find a good basis and what “good” means in this context.

The idea is to find a basis so that any spectral reflectivity can be expressed in the form:

$$\mathbf s = \sum_i w_i\mathbf b_i + \mathbf b_\mu$$

where $i$ runs over all dimensions of the space (36 using Otsu et al.'s spectral data) and $\mathbf b_\mu$ is a constant term you can ignore for now. This is a very common problem and there are many general solutions. By far the most common one is the discrete Fourier transform, which turns arbitrary signals into weighted sums of sines and cosines.

Unfortunately, recalling that the goal is to upsample a color triplet (e.g. XYZ tristimulus values or RGB data), there are only three degrees of freedom to work with and so at most three weights can be determined. Reformulating the problem,

$$\mathbf s = w_1\mathbf b_1 + w_2\mathbf b_2 + w_3\mathbf b_3 + \pmb\epsilon + \mathbf b_\mu$$

where all but three terms are lumped together into an error term $\pmb\epsilon$. I think it's clear that three sines or cosines (and a constant term) added together won't accurately reconstruct most spectra. There are far better purely mathematical models, such as Jakob and Hanika (2019).

→

2020/07/24 16:47 · Paweł Redman

Otsu et al. (2018), part 1

Otsu et. al (2018) is the second spectral upsampling method I'm adding to Colour. It's a data-driven method, which not only produces reflectances matching the given color exactly, but also that are natural and resemble real, measured reflectances. This is in contrast with, for example, Jakob and Hanika (2019), which I worked on in June and uses a mathematical model.

At the moment the code is split in two parts. The feature branch in my fork implements the method using the dataset published with the article. Creating new datasets is done by this program and will be covered in depth in a later article. Work is mostly done, I just need to integrate the program into Colour and complete this series of blog posts.

→

2020/07/23 17:25 · Paweł Redman

First merge

I'm happy to announce the code I wrote in the first month of the Summer of Code was merged upstream. You can see the pull request here.

2020/07/23 11:02 · Paweł Redman

Differentiating code – addendum

This post describes how to the compute derivatives mentioned in an earlier post using Maxima, a computer algebra system. Code is listed in a monospaced font and answers are written directly below, in LaTeX. Ending lines with dollar signs in Maxima (instead of semicolons) surpresses the output, which is very useful for definitions.

Start by defining the reflectance model.

U: c_0 * lambda^2 + c_1 * lambda + c_2$
R: 1 / 2 + U / (2 * sqrt(1 + U^2))$

Differentiate R with respect to c_0. The expansion of U appears a few times in the result. Substitute U_ for U to simplify it. (The underscore is there so it doesn't expand back to the full expression.)

diff(R, c_0)$
subst(U_, U, %);

$$\frac{\mathrm{d}R}{\mathrm{d}c_0} = \frac{\lambda^2}{2\sqrt{U^2+1}} - \frac{U^2\lambda^2}{2(U^2+1)^\frac{3}{2}}}$$

This can be further simplified by substituting $t_1 = \sqrt{U^2 + 1}$. Unfortunately subst won't recognize $(U^2 + 1)^\frac{3}{2}$ as $t_1^3$ so I recommend doing this manually.

$$\frac{\mathrm{d}R}{\mathrm{d}c_0} = \frac{\lambda^2}{2t_1} - \frac{U^2\lambda^2}{2t_1^3}$$

Repeating the process for the other coefficients reveals a pattern. Let $t_2 = \frac{1}{2t_1} - \frac{U^2}{2t_1^3}$.

  \frac{\mathrm{d}R}{\mathrm{d}c_0} = \lambda^2 t_2 \\
  \frac{\mathrm{d}R}{\mathrm{d}c_1} = \lambda t_2 \\
  \frac{\mathrm{d}R}{\mathrm{d}c_2} = t_2

After skipping trivial derivatives, which can be easily done by hand, one will find themselves having to differentiate the final expression for $\Delta E$. Normally an expression like $\frac{\mathrm{d}L}{\mathrm{d}c_0}$ (diff(L, c_0)) would be treated as a derivative of a constant in Maxima and reduced to zero. One can use depends to tell the program that the L*a*b* coordinates are functions of the model parameters. The result will then contain derivatives of the coordinates themselves.

An alternative approach is typing out the coordinates as functions explicitly. Instead of using depends, replace L with L(c_0, c_1, c_2) (and so on for a and b) in the code below. The downside is that the function arguments will appear in the end result, cluttering it and making a mess.

depends([L, a, b], [c_0, c_1, c_2])$
delta_E: sqrt((L - L_target)^2 + (a - a_target)^2 + (b - b_target)^2)$

Use the same subst trick as before to simplify.

diff(delta_E, c_0)$
subst(delta_E_, delta_E, %);

$$\frac{\big(b - b_\text{target}\big)\frac{\mathrm{d}b}{\mathrm{d}c_0} + \big(a - a_\text{target}\big)\frac{\mathrm{d}a}{\mathrm{d}c_0} +\big(L - L_\text{target}\big)\frac{\mathrm{d}L}{\mathrm{d}c_0}}{\Delta E}$$

The other derivatives look the same. Simply substitute $c_1$ or $c_2$ for $c_0$ in the expression above to get obtain them.

2020/07/14 19:37 · Paweł Redman