Qm.n with m<0 (very small tensor elements)

[alexbramanti]

My question is about the Qm.n format when m<0.

Assume I'm quantizing a network including a tensor somewhere (weights, output…) which contains only small values (for example, all included between -0.499 and 0.499 or even smaller, between -0.1 and 0.1).

Using Qm.n format, we could set m = -1 for the range [-0.499, 0.499] and m = -3 for the range [-0.1, 0.1].
Is it possible to reproduce this in QKeras?

I tried and it seems that integer (which corresponds to the m of the Qm.n format) cannot be set equal to a negative value, is that correct? In this hypothesis, what are the correct values of integer and alpha (and other parameters) in these two cases (range [-0.499, 0.499] and range [-0.1, _0.1]), provided that I cannot retrain the network (because I don’t have the training set)?

The answer to this question could help us understand how QKeras works, and whether workarounds are necessary for this case.

[danielemoro]

In theory this is possible, but in QKeras we have certain assumptions in place that mean we cannot set m to a negative number with the current code. With some targeted refactoring of QKeras quantizers, I think this could be done and would be a useful feature. We may work on this in the near future.

Alternatively, you can get to the same range by using the scales in the quantizers. This is referred to as "alpha" in quantized_bits. See this documentation:

qkeras/qkeras/quantizers.py

Line 484 in 7ba2cb2

alpha: a tensor or None, the scaling factor per channel.

You can also set alpha to "auto" or "auto_po2" to automatically find the best scale for your data distribution.