faiss/tests
Chengqi Deng eba1cb1a90 Support LSQ on GPU (#1978)
Summary:
## Description

This PR added support for LSQ on GPU. Only the encoding part is running on GPU and the others are still running on CPU.

Multi-GPU is also supported.

## Usage

``` python
lsq = faiss.LocalSearchQuantizer(d, M, nbits)
ngpus = faiss.get_num_gpus()
lsq.icm_encoder_factory = faiss.GpuIcmEncoderFactory(ngpus)  # we use all gpus

lsq.train(xt)
codes = lsq.compute_codes(xb)
decoded = lsq.decode(codes)
```

## Performance on SIFT1M

On 1 GPU:
```
===== lsq-gpu:
        mean square error = 17337.878528
        training time: 40.9857234954834 s
        encoding time: 27.12640070915222 s
```

On 2 GPUs:
```
===== lsq-gpu:
        mean square error = 17364.658176
        training time: 25.832106113433838 s
        encoding time: 14.879548072814941 s
```

On CPU:
```
===== lsq:
        mean square error = 17305.880576
        training time: 152.57522344589233 s
        encoding time: 110.01779270172119 s
```

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1978

Test Plan: buck test mode/dev-nosan //faiss/gpu/test/:test_gpu_index_py -- TestLSQIcmEncoder

Reviewed By: wickedfoo

Differential Revision: D29609763

Pulled By: mdouze

fbshipit-source-id: b6ffa2a3c02bf696a4e52348132affa0dd838870
2021-09-09 09:13:15 -07:00
..
CMakeLists.txt Add separate targets for libfaiss/libfaiss_avx2. (#1772) 2021-03-26 14:28:16 -07:00
common_faiss_tests.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_autotune.py
test_binary_factory.py
test_binary_flat.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_binary_hashindex.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_binary_io.py
test_build_blocks.py three small fixes (#1972) 2021-07-01 16:08:45 -07:00
test_clustering.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_contrib.py three small fixes (#1972) 2021-07-01 16:08:45 -07:00
test_dealloc_invlists.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_documentation.py Add docstrings for most of the replaced index methods 2021-03-05 17:19:17 -08:00
test_extra_distances.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_factory.py three small fixes (#1972) 2021-07-01 16:08:45 -07:00
test_fast_scan.py Add ARM to CI (#1914) 2021-06-08 14:53:02 -07:00
test_fast_scan_ivf.py Fix training of complex quantizer (#2035) 2021-09-06 08:53:29 -07:00
test_index.py Range search bug 2021-05-17 23:10:20 -07:00
test_index_accuracy.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_index_binary.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_index_binary_from_float.py Fix inefficient float / binary conversion 2021-03-01 21:43:31 -08:00
test_index_composite.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_io.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_ivflib.py
test_ivfpq_codec.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_ivfpq_indexing.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_lowlevel_ivf.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_lsq.py Support LSQ on GPU (#1978) 2021-09-09 09:13:15 -07:00
test_merge.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_meta_index.py Add manifold check for size 0 (#1867) 2021-05-09 22:30:31 -07:00
test_omp_threads.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_omp_threads_py.py
test_ondisk_ivf.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_oom_exception.py Portable SWIG Vectors (#1742) 2021-04-02 07:11:47 -07:00
test_pairs_decoding.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_params_override.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_partition.py
test_pq_encoding.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_product_quantizer.py Use BLAS to compute sdc table (#1809) 2021-04-16 00:17:51 -07:00
test_referenced_objects.py
test_refine.py
test_residual_quantizer.py fix centroids_norms storage for ResidualCoarseQuantizer (#2018) 2021-08-25 06:37:33 -07:00
test_sliding_ivf.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_standalone_codec.py Support for additive quantizer search (#1961) 2021-08-20 01:00:10 -07:00
test_threaded_index.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
test_transfer_invlists.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
torch_test_contrib.py