Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3494
This tasks focus on the refine index construction tutorial with different index refinement on fp16/sq8 quantization. The python version was added a while ago.
Reviewed By: junjieqi
Differential Revision: D58161983
fbshipit-source-id: 1c598fe612b5dee3952c5f7398e6802e117f141d
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3480
This tutorial summarize the methods to construct different indexs for PQFastScan refinement. It shows how the choice can impact on accuracy.
Reviewed By: junjieqi
Differential Revision: D57799598
fbshipit-source-id: a75c52c60a5217366f3361676da8f03f0c4a9feb
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3474
This commit focus on the cpp version of PQfastscan tutorial with index refinement by defining the k factor.
Reviewed By: junjieqi
Differential Revision: D57680905
fbshipit-source-id: 980c2990172f24ec9a4f870685e706195883408f
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3468
This commit includes the tutorial for PQFastScan in the cpp environment.
Reviewed By: junjieqi
Differential Revision: D57631441
fbshipit-source-id: f5e17eee2a584ebfc9ff63868d741d0da6b3b413
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3465
This commit include python version of tutorial for FastScan. It includes all the parameters enabled within PQFastScan.
Reviewed By: junjieqi
Differential Revision: D57594044
fbshipit-source-id: cb12679b6fc241a654b9545c5bc7bd0517aa1813
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3439
LLVM-15 has a warning `-Wunused-but-set-variable` which we treat as an error because it's so often diagnostic of a code issue. Unused variables can compromise readability or, worse, performance.
This diff either (a) removes an unused variable and, possibly, it's associated code, or (b) qualifies the variable with `[[maybe_unused]]`, mostly in cases where the variable _is_ used, but, eg, in an `assert` statement that isn't present in production code.
- If you approve of this diff, please use the "Accept & Ship" button :-)
Reviewed By: palmje, junjieqi
Differential Revision: D57344013
fbshipit-source-id: adf410139d2e6ca69a26ccdbff8511c9b7620489
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2582
A few more or less cosmetic improvements
* Index::idx_t was in the Index object, which does not make much sense, this diff moves it to faiss::idx_t
* replace multiprocessing.dummy with multiprocessing.pool
* add Alexandr as a core contributor of Faiss in the README ;-)
```
for i in $( find . -name \*.cu -o -name \*.cuh -o -name \*.h -o -name \*.cpp ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
For the fbcode deps:
```
for i in $( fbgs Index::idx_t --exclude fbcode/faiss -l ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
Reviewed By: algoriddle
Differential Revision: D41437507
fbshipit-source-id: 8300f2a3ae97cace6172f3f14a9be3a83999fb89
Changelog:
- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
* Refactors Makefiles and add configure script.
* Give MKL higher priority in configure script.
* Clean up Linux example makefile.inc.
* Cleanup makefile.inc examples.
* Fix python clean Makefile target.
* Regen swig wrappers.
* Remove useless CUDAFLAGS variable.
* Fix python linking flags.
* Separate compile and link phase in python makefile.
* Add macro to look for swig.
* Add CUDA check in configure script.
* Cleanup make depend targets.
* Cleanup CUDA flags.
* Fix linking flags.
* Fix python GPU linking.
* Remove useless flags from python gpu module linking.
* Add check for cuda libs.
* Cleanup GPU targets.
* Clean up test target.
* Add cpu/gpu targets to python makefile.
* Clean up tutorial Makefile.
* Remove stale OS var from example makefiles.
* Clean up cuda example flags.
* Remove transitional print import.
* Add example for multiple GPUs in python.
* Add example on GPU in C++.
* Add example on multiple GPUs in C++.
* Add IVFFlat example on GPU in python.
* Add Makefile for C++ tutorial examples.
* Add IVF index on GPU example in C++.
* moved most FAISS_ASSERT calls to C++ exceptions, and adjusted
memory allocation to avoid mem leaks
* added an IndexIVFScalarQuantizer type that offers an
intermediate compression between IVFFlat and IVFPQ
* support removal of indices in IndexIDMap / IndexFlat combination
* various fixes in GPU code
* add initial cmake support
* update cmake, add cmake instructions to INSTALL
* update findopenmp and INSTALL
* change FindOpenBLAS.cmake to cater for macports
- change cblas.h to openblas_config.h since macports does not ship
cblas.h with openblas.
* revise INSTALL for cmake
Xiao Fu
Saarth Deshpande
Richard Barnes
Jayjeet Chakraborty
Matthijs Douze
Lucas Hosseini
Matthijs Douze
Lucas Hosseini
gensmusic
Xuhui Sun
Lizz
Tianwei Shen