faiss/c_api
Matthijs Douze e1adde0d84 Faster brute force search (#1502)
Summary:
This diff streamlines the code that collects results for brute force distance computations for the L2 / IP and range search / knn search combinations.

It introduces a `ResultHandler` template class that abstracts what happens with the computed distances and ids. In addition to the heap result handler and the range search result handler, it introduces a reservoir result handler that improves the search speed for  large k (>=100).

Benchmark results (https://fb.quip.com/y0g1ACLEqJXx#OCaACA2Gm45) show that on small datasets (10k) search is 10-50% faster (improvements are larger for small k). There is room for improvement in the reservoir implementation, whose implementation is quite naive currently, but the diff is already useful in its current form.

Experiments on precomputed db vector norms for L2 distance computations were not very concluding performance-wise, so the implementation is removed from IndexFlatL2.

This diff also removes IndexL2BaseShift, which was never used.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1502

Test Plan:
```
buck test //faiss/tests/:test_product_quantizer
buck test //faiss/tests/:test_index -- TestIndexFlat
```

Reviewed By: wickedfoo

Differential Revision: D24705464

Pulled By: mdouze

fbshipit-source-id: 270e10b19f3c89ed7b607ec30549aca0ac5027fe
2020-11-04 22:16:23 -08:00
..
gpu [c_api] Fix GPU library build (#1296) 2020-07-27 17:26:17 +02:00
impl [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
AutoTune_c.cpp [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
AutoTune_c.h fix free parameterspace (#1243) 2020-07-09 09:32:03 +02:00
Clustering_c.cpp C API : ClusteringIterationStats integration (#1140) 2020-03-12 23:22:39 +01:00
Clustering_c.h C API : ClusteringIterationStats integration (#1140) 2020-03-12 23:22:39 +01:00
INSTALL.md
IndexFlat_c.cpp [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00
IndexFlat_c.h Faster brute force search (#1502) 2020-11-04 22:16:23 -08:00
IndexIVFFlat_c.cpp [C API] June 2019 update (#869) 2019-06-24 13:29:38 +02:00
IndexIVFFlat_c.h [C API] June 2019 update (#869) 2019-06-24 13:29:38 +02:00
IndexIVF_c.cpp c_api: faiss_IndexIVF_invlists_get_ids function (#1094) 2020-01-31 13:40:28 +01:00
IndexIVF_c.h c_api: faiss_IndexIVF_invlists_get_ids function (#1094) 2020-01-31 13:40:28 +01:00
IndexLSH_c.cpp [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00
IndexLSH_c.h [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00
IndexPreTransform_c.cpp [ C api ] Indexpretransform get underlying index (#1100) 2020-02-10 15:00:59 +01:00
IndexPreTransform_c.h [ C api ] Indexpretransform get underlying index (#1100) 2020-02-10 15:00:59 +01:00
IndexShards_c.cpp Add missing copyright headers. (#1460) 2020-10-13 11:15:59 -07:00
IndexShards_c.h Cleanup flag/data propagation for IndexShards and IndexReplicas 2020-09-29 10:25:46 -07:00
Index_c.cpp [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
Index_c.h [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
Makefile [ C api ] Indexpretransform get underlying index (#1100) 2020-02-10 15:00:59 +01:00
MetaIndexes_c.cpp [C API] June 2019 update (#869) 2019-06-24 13:29:38 +02:00
MetaIndexes_c.h [C API] June 2019 update (#869) 2019-06-24 13:29:38 +02:00
clone_index_c.cpp [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
clone_index_c.h [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
error_c.h [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00
error_impl.cpp [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00
error_impl.h [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00
example_c.c [c_api] Fix GPU library build (#1296) 2020-07-27 17:26:17 +02:00
faiss_c.h [C API] June 2019 update (#869) 2019-06-24 13:29:38 +02:00
index_factory_c.cpp [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
index_factory_c.h [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
index_io_c.cpp [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
index_io_c.h [C_API] Pass compilation in current state (#989) 2019-11-19 11:36:44 +01:00
macros_impl.h [C API] Update for compatibility with v1.5 (#858) 2019-06-12 15:46:08 +02:00