Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2682
IndexShards normally sees the indexes as opaque, so there is no way to factrorize the coarse quantizer.
This diff introduces IndexIVFShards that handles IVF indexes with a common quantizer so that the quantization is computed only once.
Reviewed By: alexanderguzhva
Differential Revision: D42781513
fbshipit-source-id: 441316eff4c1ba0468501c456af9194ea5f042d6
Summary:
support merge for all IndexFlatCodes children
make merge_from and check_compatible_for_merge methods of Index and IndexIVF and IndexFlatCodes(the only supported types) inherit them from Index.
This is part 1 of 2 as merge_into still not updated
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2488
Test Plan:
cd build
make -j
make test
cd faiss/python && python setup.py build
cd ../../..
PYTHONPATH="$(ls -d ./build/faiss/python/build/lib*/)" pytest tests/test_*.py
# fbcode
buck test //faiss/tests/:test_index_merge
buck test //faiss/tests/:test_io
Reviewed By: mdouze
Differential Revision: D39726378
Pulled By: AbdelrahmanElmeniawy
fbshipit-source-id: 6739477fddcad3c7a990f3aae9be07c1b2b74fef
Summary:
- Disable problematic tests on OSX.
- Ensure compiler compatibility with CUDA builds.
- Fix path for Python extension libraries.
- Use CentOS for CUDA packaging.
- Update CUDA versions in CI (10.2 and 11.3).
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2121
Reviewed By: mdouze
Differential Revision: D32921117
Pulled By: beauby
fbshipit-source-id: 588c18add8084b8228ff5abc651eaa4567919cc6
Summary:
Need to add an ssh key to the circleci to be able to debug
For my own ref, how to connect to the job:
```
[matthijs@matthijs-mbp /Users/matthijs/Desktop/faiss_github/circleci_keys] ssh -p 54782 38.39.188.110 -i id_ed25519
```
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1849
Reviewed By: wickedfoo
Differential Revision: D28234897
Pulled By: mdouze
fbshipit-source-id: 6827fa45f24b3e4bf586315bd38f18608d07ecf9
Summary:
`long` is 32 bits on windows and so is the default int type for numpy (eg. the one used for `np.arange`).
This diff explicitly specifies 64-bit ints for all occurrences where it matters.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1381
Reviewed By: wickedfoo
Differential Revision: D23371232
Pulled By: mdouze
fbshipit-source-id: 220262cd70ee70379f83de93561a4eae71c94b04
Bugfixes:
- slow scanning of inverted lists (#836).
Features:
- add basic support for 6 new metrics in CPU `IndexFlat` and `IndexHNSW` (#848);
- add support for `IndexIDMap`/`IndexIDMap2` with binary indexes (#780).
Misc:
- throw python exception for OOM (#758);
- make `DistanceComputer` available for all random access indexes;
- gradually moving from `long` to `int64_t` for portability.
Changelog:
- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
Matthijs Douze
Abdelrahman Elmeniawy
Lucas Hosseini
Matthijs Douze
Lucas Hosseini