Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2582
A few more or less cosmetic improvements
* Index::idx_t was in the Index object, which does not make much sense, this diff moves it to faiss::idx_t
* replace multiprocessing.dummy with multiprocessing.pool
* add Alexandr as a core contributor of Faiss in the README ;-)
```
for i in $( find . -name \*.cu -o -name \*.cuh -o -name \*.h -o -name \*.cpp ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
For the fbcode deps:
```
for i in $( fbgs Index::idx_t --exclude fbcode/faiss -l ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
Reviewed By: algoriddle
Differential Revision: D41437507
fbshipit-source-id: 8300f2a3ae97cace6172f3f14a9be3a83999fb89
Summary:
Following up on issue https://github.com/facebookresearch/faiss/issues/2054 it seems that this code crashes Faiss (instead of just leaking memory).
Findings:
- when running in MT mode, each search in an indexflat used as coarse quantizer consumes some memory
- this mem consumption does not appear in single-thread mode or with few threads
- in gdb it appears that even when the nb of queries is 1, each search spawns max_threads threads (80 on the test machine)
This diff:
- adds a C++ test that checks how much mem is used when repeatedly searching a vector
- adjusts the number of search threads to the number of query vectors. This is especially useful for single-vector queries.
Reviewed By: beauby
Differential Revision: D31142383
fbshipit-source-id: 134ddaf141e7c52a854cea398f5dbf89951a7ff8
Matthijs Douze
Matthijs Douze