e796f4f9df
Summary:
PyTorch GPU in general is free to use whatever stream it currently wants, based on `torch.cuda.current_stream()`. Due to C++/python language barrier issues, we couldn't previously pass the actual `cudaStream_t` that is currently in use on a given device from PyTorch C++ to Faiss C++ via python.
This diff adds conversion functions to convert a Python integer representing a pointer to a `cudaStream_t` (which is itself a `CUstream_st*`), so we can pass the stream specified in `torch.cuda.current_stream()` to `StandardGpuResources::setDefaultStream`. We thus guarantee that all Faiss work is ordered on the same stream that is in use in PyTorch.
For use in Python, there is now the `faiss.contrib.pytorch_tensors.using_stream` context object which automatically sets and unsets the current PyTorch stream within Faiss. This takes a `StandardGpuResources` object in Python, and an optional `torch.cuda.Stream` if one wants to use a different stream, otherwise it uses the current one. This is how it is used:
```
# Create a non-default stream
s = torch.cuda.Stream()
# Have Torch use it
with torch.cuda.stream(s):
# Have Faiss use the same stream as the above
with faiss.contrib.pytorch_tensors.using_stream(res):
# Do some work on the GPU
faiss.bfKnn(res, args)
```
`using_stream` uses the same pattern as the Pytorch `torch.cuda.stream` object.
This replaces any brute-force GPU/CPU synchronization work that was necessary before.
Other changes in this diff:
- cleans up the config objects in the GpuIndex subclasses, to distinguish between read-only parameters that can only be set upon index construction, versus those that can be changed at runtime.
- StandardGpuResources now more properly distinguishes between user-supplied streams (like the PyTorch one) which will not be destroyed upon resources destruction, versus internal streams.
- `search_index_pytorch` now needs to take a `StandardGpuResources` object as well, there is no way to get this from an index instance otherwise (or at least, I would have to return a `shared_ptr`, in which case we should just update the Python SWIG stuff to use `shared_ptr` for GpuResources or something.
Reviewed By: mdouze
Differential Revision: D24260026
fbshipit-source-id: b18bb0eb34eb012584b1c923088228776c10b720
The contrib modules
The contrib directory contains helper modules for Faiss for various tasks.
Code structure
The contrib directory gets compiled in the module faiss.contrib. Note that although some of the modules may depend on additional modules (eg. GPU Faiss, pytorch, hdf5), they are not necessarily compiled in to avoid adding dependencies. It is the user's responsibility to provide them.
In contrib, we are progressively dropping python2 support.
List of contrib modules
rpc.py
A very simple Remote Procedure Call library, where function parameters and results are pickled, for use with client_server.py
client_server.py
The server handles requests to a Faiss index. The client calls the remote index. This is mainly to shard datasets over several machines, see Distributd index
ondisk.py
Encloses the main logic to merge indexes into an on-disk index. See On-disk storage
exhaustive_search.py
Computes the ground-truth search results for a dataset that possibly does not fit in RAM. Uses GPU if available.
Tested in tests/test_contrib.TestComputeGT
gpu.py
(requires GPU Faiss)
Interoperability functions for pytorch and Faiss: pass GPU data without copying back to CPU.
Tested in gpu/test/test_pytorch_faiss
datasets.py
(may require h5py)
Defintion of how to access data for some standard datsets.