mirrors
/
faiss
mirror of https://github.com/facebookresearch/faiss.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
faiss/contrib
History
Matthijs Douze c7fd8d7ac3 improved range search evaluation functions 
Summary: For range search evaluation, this diff adds optimized functions for ground-truth generation (on GPU).

Reviewed By: beauby

Differential Revision: D25822716

fbshipit-source-id: c5278dfad0510d24c2a5c87c1f8c81161fa8c5d3
 		2021-01-11 08:12:10 -08:00
..
README.md Add range search accuracy evaluation 2020-12-17 17:17:09 -08:00
client_server.py Add missing copyright headers. (#1460) 2020-10-13 11:15:59 -07:00
datasets.py PQ4 fast scan benchmarks (#1555) 2020-12-16 01:18:58 -08:00
evaluation.py Add range search accuracy evaluation 2020-12-17 17:17:09 -08:00
exhaustive_search.py improved range search evaluation functions 2021-01-11 08:12:10 -08:00
factory_tools.py PQ4 fast scan benchmarks (#1555) 2020-12-16 01:18:58 -08:00
inspect_tools.py Implementation of PQ4 search with SIMD instructions (#1542) 2020-12-03 10:06:38 -08:00
ondisk.py Add missing copyright headers. (#1460) 2020-10-13 11:15:59 -07:00
rpc.py Add missing copyright headers. (#1460) 2020-10-13 11:15:59 -07:00
torch_utils.py PyTorch tensor / Faiss index interoperability (#1484) 2020-10-23 22:24:22 -07:00
vecs_io.py PQ4 fast scan benchmarks (#1555) 2020-12-16 01:18:58 -08:00

README.md

The contrib modules

The contrib directory contains helper modules for Faiss for various tasks.

Code structure

The contrib directory gets compiled in the module faiss.contrib. Note that although some of the modules may depend on additional modules (eg. GPU Faiss, pytorch, hdf5), they are not necessarily compiled in to avoid adding dependencies. It is the user's responsibility to provide them.

In contrib, we are progressively dropping python2 support.

List of contrib modules

rpc.py

A very simple Remote Procedure Call library, where function parameters and results are pickled, for use with client_server.py

client_server.py

The server handles requests to a Faiss index. The client calls the remote index. This is mainly to shard datasets over several machines, see Distributd index

ondisk.py

Encloses the main logic to merge indexes into an on-disk index. See On-disk storage

exhaustive_search.py

Computes the ground-truth search results for a dataset that possibly does not fit in RAM. Uses GPU if available. Tested in tests/test_contrib.TestComputeGT

torch_utils.py

Interoperability functions for pytorch and Faiss: Importing this will allow pytorch Tensors (CPU or GPU) to be used as arguments to Faiss indexes and other functions. Torch GPU tensors can only be used with Faiss GPU indexes. If this is imported with a package that supports Faiss GPU, the necessary stream synchronization with the current pytorch stream will be automatically performed.

Numpy ndarrays can continue to be used in the Faiss python interface after importing this file. All arguments must be uniformly either numpy ndarrays or Torch tensors; no mixing is allowed.

Tested in tests/test_contrib_torch.py (CPU) and gpu/test/test_contrib_torch_gpu.py (GPU).

inspect_tools.py

Functions to inspect C++ objects wrapped by SWIG. Most often this just means reading fields and converting them to the proper python array.

datasets.py

(may require h5py)

Defintion of how to access data for some standard datsets.

factory_tools.py

Functions related to factory strings.

evaluation.py

A few non-trivial evaluation functions for search results