mirrors
/
faiss
mirror of https://github.com/facebookresearch/faiss.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
714 Commits
23 Branches
19 Tags
212 MiB
Commit Graph

3 Commits (60c850e2969c41a3a3597e70c7cb96393e574ff7)

Author SHA1 Message Date
Check Deng 838f85cb52 Implement search methods for ProductAdditiveQuantizer (#2336) 
Summary:
Work in progress.

This PR is going to implement the following search methods for ProductAdditiveQuantizer, including index factory and I/O:

- [x] IndexProductAdditiveQuantizer
- [x] IndexIVFProductAdditiveQuantizer
- [x] IndexProductAdditiveQuantizerFastScan
- [x] IndexIVFProductAdditiveQuantizerFastScan

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2336

Test Plan:
buck test //faiss/tests/:test_fast_scan
buck test //faiss/tests/:test_fast_scan_ivf
buck test //faiss/tests/:test_local_search_quantizer
buck test //faiss/tests/:test_residual_quantizer

Reviewed By: alexanderguzhva

Differential Revision: D37172745

Pulled By: mdouze

fbshipit-source-id: 6ff18bfc462525478c90cd42e21805ab8605bd0f
 2022-07-27 05:32:15 -07:00
Matthijs Douze f2a9324359 make tests cheaper 
Summary:
Many of the additive quantizer tests are recognized as flaky because the tests timeout in non-optimized stress mode.
This is probably because they don't import

https://www.internalfb.com/code/fbsource/fbcode/faiss/tests/common_faiss_tests.py

that sets the number of threads to 4. This diff fixes that and in addition declares the tests as "heavyweight" so that not too many of them are spawned in parallel in stress mode.

https://www.internalfb.com/intern/wiki/TAE/tpx/Timeouts_and_Sharded_Bundled_mode/#degree-of-parallelism

Hopefully it should fix the flaky tests

Reviewed By: alexanderguzhva

Differential Revision: D38111820

fbshipit-source-id: 7dd7c72e7e92b82384a170743cfd5c4aaf9a6960
 2022-07-25 06:58:39 -07:00
Check Deng 9b1982262a Add ProductAdditiveQuantizer (#2286) 
Summary:
This diff added ProductAdditiveQuantizer.

A Simple Algo description:

1. Divide the vector space into several orthogonal sub-spaces, just like PQ does.
2. Quantize each sub-space by an independent additive quantizer.

Usage:

Construct a ProductAdditiveQuantizer object:
- `d`: dimensionality of the input vectors
- `nsplits`: number of sub-spaces divided into
- `Msub`: `M` of each additive quantizer
- `nbits`: `nbits` of each additive quantizer

```python
d = 128
nsplits = 2
Msub = 4
nbits = 8
plsq = faiss.ProductLocalSearchQuantizer(d, nsplits, Msub, nbits)
prq = faiss.ProductResidualQuantizer(d, nsplits, Msub, nbits)
```

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2286

Test Plan:
```

buck test //faiss/tests/:test_local_search_quantizer -- TestProductLocalSearchQuantizer

buck test //faiss/tests/:test_residual_quantizer -- TestProductResidualQuantizer

```

Reviewed By: alexanderguzhva

Differential Revision: D35907702

Pulled By: mdouze

fbshipit-source-id: 7428a196e6bd323569caa585c57281dd70e547b1
 2022-05-05 15:14:07 -07:00