dd72e4121d
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3608 This is a straightforward implementation of QINCo in CPU Faiss, with encoding and decoding capabilities (not training). For this, we translate a simplified version of some torch classes: - tensors, restricted to 2D and int32 + float32 - Linear and Embedding layer Then the QINCoStep and QINCo can just be defined as C++ objects that are copy-constructable. There is some plumbing required in the wrapping layers to support the integration. Pytroch tensors are converted to numpy for getting / setting them in C++. Reviewed By: asadoughi Differential Revision: D59132952 fbshipit-source-id: eea4856507a5b7c5f219efcf8d19fe56944df088 |
||
|---|---|---|
| .. | ||
| offline_ivf | ||
| rocksdb_ivf | ||
| CMakeLists.txt | ||
| README.md | ||
| demo_auto_tune.py | ||
| demo_client_server_ivf.py | ||
| demo_imi_flat.cpp | ||
| demo_imi_pq.cpp | ||
| demo_ivfpq_indexing.cpp | ||
| demo_nndescent.cpp | ||
| demo_ondisk_ivf.py | ||
| demo_qinco.py | ||
| demo_residual_quantizer.cpp | ||
| demo_sift1M.cpp | ||
| demo_weighted_kmeans.cpp | ||
README.md
Demos for a few Faiss functionalities
demo_auto_tune.py
Demonstrates the auto-tuning functionality of Faiss
demo_ondisk_ivf.py
Shows how to construct a Faiss index that stores the inverted file data on disk, eg. when it does not fit in RAM. The script works on a small dataset (sift1M) for demonstration and proceeds in stages:
0: train on the dataset
1-4: build 4 indexes, each containing 1/4 of the dataset. This can be done in parallel on several machines
5: merge the 4 indexes into one that is written directly to disk (needs not to fit in RAM)
6: load and test the index