Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2683
Adds a small function and a test to demonstrate how to crop a RCQ to its first quantizers.
Also adds a SearchParameters to set the beam_factor of a RCQ.
Reviewed By: alexanderguzhva
Differential Revision: D42842378
fbshipit-source-id: b522269ce983cddc50c3d0e18b541954c5d7001d
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2633
The core idea.
Instead of putting every element of the dataset into MaxHeap, split the dataset into buckets and let every bucket track elements min-1, min-2 or min-3 distances.
Applied to ResidualQuantizer class for vector codec purposes.
An example
```
rq.approx_topk_mode = faiss.APPROX_TOPK_BUCKETS_B16_D2
```
Reviewed By: mdouze
Differential Revision: D42044398
fbshipit-source-id: 43169026476650442806a31d1c1aa2d5d5028e65
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2568
Add a fused kernel for exhaustive_L2sqr_blas() call that combines a computation of dot product and the search for the nearest centroid. As a result, no temporary dot product values are written and read in RAM.
Speeds up the training of PQx[1] indices for dsub = 1, 2, 4, 8, and the effect is higher for higher values of [1]. AVX512 version provides additional overloads for dsub = 12, 16.
The speedup is also beneficial for higher values of pq.cp.max_points_per_centroid (which is 256 by default).
Speeds up IVFPQ training as well.
AVX512 kernel is not enabled, but I've seen it speeding up the training TWICE versus AVX2 version. So, please feel free to use it by enabling AVX512 manually.
Reviewed By: mdouze
Differential Revision: D41166766
fbshipit-source-id: 443014e2e59396b3a90b9171fec8c8191052bcf4
Summary:
This PR contains below changes:
- Conform C++11
- [`faiss` is written in C++11](https://github.com/facebookresearch/faiss/blob/main/CONTRIBUTING.md#coding-style), but [`faiss/cppcontrib/SaDecodeKernels-avx2-inl.h`](442d9f4a2d/faiss/cppcontrib/SaDecodeKernels-avx2-inl.h) and [the test](442d9f4a2d/tests/test_cppcontrib_sa_decode.cpp) use some C++17 features. This PR rewrites these codes to make them independent to C++17.
- Enable AVX2 on `faiss_test`
- Currently `faiss_test` is compiled without `-mavx2` even if `-DFAISS_OPT_LEVEL=avx2` , so **`tests/test_cppcontrib_sa_decode.cpp` hasn't checked `faiss/cppcontrib/SaDecodeKernels-avx2-inl.h` at all** . This PR adds `-mavx2` to `faiss_test` if `-DFAISS_OPT_LEVEL=avx2` , so now `tests/test_cppcontrib_sa_decode.cpp` confirms `faiss/cppcontrib/SaDecodeKernels-avx2-inl.h` if `-DFAISS_OPT_LEVEL=avx2` , and does `faiss/cppcontrib/SaDecodeKernels-inl.h` if not `-DFAISS_OPT_LEVEL=avx2` .
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2388
Reviewed By: mdouze
Differential Revision: D38005738
Pulled By: alexanderguzhva
fbshipit-source-id: b9319c585c6849e1c7a4782770f2d7ce8c0d8660
Summary:
Following up on issue https://github.com/facebookresearch/faiss/issues/2054 it seems that this code crashes Faiss (instead of just leaking memory).
Findings:
- when running in MT mode, each search in an indexflat used as coarse quantizer consumes some memory
- this mem consumption does not appear in single-thread mode or with few threads
- in gdb it appears that even when the nb of queries is 1, each search spawns max_threads threads (80 on the test machine)
This diff:
- adds a C++ test that checks how much mem is used when repeatedly searching a vector
- adjusts the number of search threads to the number of query vectors. This is especially useful for single-vector queries.
Reviewed By: beauby
Differential Revision: D31142383
fbshipit-source-id: 134ddaf141e7c52a854cea398f5dbf89951a7ff8
* add initial cmake support
* update cmake, add cmake instructions to INSTALL
* update findopenmp and INSTALL
* change FindOpenBLAS.cmake to cater for macports
- change cblas.h to openblas_config.h since macports does not ship
cblas.h with openblas.
* revise INSTALL for cmake
Gergely Szilvasy
Alexandr Guzhva
Matthijs Douze
Alexandr Guzhva
wx257osn2@yahoo.co.jp
Matthijs Douze
Lucas Hosseini
Adeykin
Tianwei Shen