Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2439
Index wrapper that performs rowwise normalization to [0,1], preserving the coefficients. This is a vector codec index only.
Basically, this index performs a rowwise scaling to [0,1] of every row in an input dataset before calling subindex::train() and subindex::sa_encode(). sa_encode() call stores the scaling coefficients (scaler and minv) in the very beginning of every output code. The format:
[scaler][minv][subindex::sa_encode() output]
The de-scaling in sa_decode() is done using:
output_rescaled = scaler * output + minv
An additional ::train_inplace() function is provided in order to do an inplace scaling before calling subindex::train() and, thus, avoiding the cloning of the input dataset, but modifying the input dataset because of the scaling and the scaling back.
Derived classes provide different data types for scaling coefficients. Currently, versions with fp16 and fp32 scaling coefficients are available.
* fp16 version adds 4 extra bytes per encoded vector
* fp32 version adds 8 extra bytes per encoded vector
Reviewed By: mdouze
Differential Revision: D38581012
fbshipit-source-id: d739878f1db62ac5ab9e0db3f84aeb2b70a1b6c0
1e4586a5a0