mirrors
/
faiss
mirror of https://github.com/facebookresearch/faiss.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
faiss/demos
History
Check Deng d6535a3d87 Add NNDescent to faiss (#1654) 
Summary:
As discussed in https://github.com/facebookresearch/faiss/issues/685, I'm going to add an NSG index to faiss. This PR which adds an NNDescent index is the first step as I commented [here ](https://github.com/facebookresearch/faiss/issues/685#issuecomment-760608431).

**Changes:**
1. Add an `IndexNNDescent` and an `IndexNNDescentFlat` which allow users to construct a KNN graph on a million scale dataset using CPU and search NN on it. The implementation part is put under `faiss/impl`.
2. Add compilation entries to `CMakeLists.txt` for C++ and `swigfaiss.swig` for Python. `IndexNNDescentFlat` could be directly called by users in C++ and Python.
3. `VisitedTable` struct in `HNSW.h` is moved into `AuxIndexStructures.h`.
3. Add a demo `demo_nndescent.cpp` to demonstrate the effectiveness.

**TODO**
1. Support index factor.
2. Implement `IndexNNDescentPQ` and `IndexNNDescentSQ`
3. More comments in the code.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1654

Test Plan:
buck test //faiss/tests/:test_index_accuracy -- TestNNDescent

buck test //faiss/tests/:test_build_blocks -- TestNNDescentKNNG

Reviewed By: wickedfoo

Differential Revision: D26309716

Pulled By: mdouze

fbshipit-source-id: 2abade9708d29023f8bccbf77143e8eea14f66c4
 		2021-02-25 16:48:28 -08:00
..
CMakeLists.txt Add NNDescent to faiss (#1654) 2021-02-25 16:48:28 -08:00
README.md sync with FB version 2018-02-23 (#347) 2018-02-23 07:49:45 -08:00
demo_auto_tune.py fbshipit-source-id: 4f3cfa59471d548af93fe118d1b73d45bc648edf 2020-08-04 12:00:38 -07:00
demo_client_server_ivf.py Fix faiss_contrib (#1478) 2020-10-20 04:35:19 -07:00
demo_imi_flat.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
demo_imi_pq.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
demo_ivfpq_indexing.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
demo_nndescent.cpp Add NNDescent to faiss (#1654) 2021-02-25 16:48:28 -08:00
demo_ondisk_ivf.py Fix faiss_contrib (#1478) 2020-10-20 04:35:19 -07:00
demo_sift1M.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00
demo_weighted_kmeans.cpp Enable clang-format + autofix. 2021-02-25 04:46:10 -08:00

README.md

Demos for a few Faiss functionalities

demo_auto_tune.py

Demonstrates the auto-tuning functionality of Faiss

demo_ondisk_ivf.py

Shows how to construct a Faiss index that stores the inverted file data on disk, eg. when it does not fit in RAM. The script works on a small dataset (sift1M) for demonstration and proceeds in stages:

0: train on the dataset

1-4: build 4 indexes, each containing 1/4 of the dataset. This can be done in parallel on several machines

5: merge the 4 indexes into one that is written directly to disk (needs not to fit in RAM)

6: load and test the index