Summary:
Work in progress.
This PR is going to implement the following search methods for ProductAdditiveQuantizer, including index factory and I/O:
- [x] IndexProductAdditiveQuantizer
- [x] IndexIVFProductAdditiveQuantizer
- [x] IndexProductAdditiveQuantizerFastScan
- [x] IndexIVFProductAdditiveQuantizerFastScan
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2336
Test Plan:
buck test //faiss/tests/:test_fast_scan
buck test //faiss/tests/:test_fast_scan_ivf
buck test //faiss/tests/:test_local_search_quantizer
buck test //faiss/tests/:test_residual_quantizer
Reviewed By: alexanderguzhva
Differential Revision: D37172745
Pulled By: mdouze
fbshipit-source-id: 6ff18bfc462525478c90cd42e21805ab8605bd0f
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2251
the fast_scan and fast_scan_ivf tests are irregularly timing out on the FB test infra
This diff:
- breaks down more tests into sub-tests
- makes tests cheaper by reducing the test dataset sizes
- corrects a nasty local variable binding bug that prevented all cases of `implem` to be covered.
I also tried to fix the polysemous tests that also timeout but I could not reproduce the timeout.
https://www.internalfb.com/intern/test/562949978542309?ref_report_id=0
Reviewed By: beauby
Differential Revision: D34852254
fbshipit-source-id: b005ffb3723e7d9df75516a539540d9165249cea
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2245
This changeset makes the `heap_replace_top()` function of the FAISS heap implementation break distance ties by the element's ID, according to the heap's min/max property.
Reviewed By: mdouze
Differential Revision: D34669542
fbshipit-source-id: 0db24fd12442eedeee917fbb3e811ba4a070ce0f
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1905
This PR added some tests to make sure the building with AVX2 works as we expected in Linux.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1792
Test Plan: buck test //faiss/tests/:test_fast_scan -- test_PQ4_speed
Reviewed By: beauby
Differential Revision: D27435796
Pulled By: mdouze
fbshipit-source-id: 901a1d0abd9cb45ccef541bd7a570eb2bd8aac5b
Summary:
Fast-scan tests were disabled on windows because of a heap corruption. This diff enables them because the free_aligned bug was fixed in the meantime.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1663
Reviewed By: beauby
Differential Revision: D26201040
Pulled By: mdouze
fbshipit-source-id: 8d6223b4e42ccb1ce2da6e2c51d9e0833199bde7
Summary:
IndexPQ and IndexIVFPQ implementations with AVX shuffle instructions.
The training and computing of the codes does not change wrt. the original PQ versions but the code layout is "packed" so that it can be used efficiently by the SIMD computation kernels.
The main changes are:
- new IndexPQFastScan and IndexIVFPQFastScan objects
- simdib.h for an abstraction above the AVX2 intrinsics
- BlockInvertedLists for invlists that are 32-byte aligned and where codes are not sequential
- pq4_fast_scan.h/.cpp: for packing codes and look-up tables + optmized distance comptuation kernels
- simd_result_hander.h: SIMD version of result collection in heaps / reservoirs
Misc changes:
- added contrib.inspect_tools to access fields in C++ objects
- moved .h and .cpp code for inverted lists to an invlists/ subdirectory, and made a .h/.cpp for InvertedListsIOHook
- added a new inverted lists type with 32-byte aligned codes (for consumption by SIMD)
- moved Windows-specific intrinsics to platfrom_macros.h
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1542
Test Plan:
```
buck test mode/opt -j 4 //faiss/tests/:test_fast_scan_ivf //faiss/tests/:test_fast_scan
buck test mode/opt //faiss/manifold/...
```
Reviewed By: wickedfoo
Differential Revision: D25175439
Pulled By: mdouze
fbshipit-source-id: ad1a40c0df8c10f4b364bdec7172e43d71b56c34
Check Deng
Matthijs Douze
Ivan Sopin