Summary:
Following up on issue https://github.com/facebookresearch/faiss/issues/2054 it seems that this code crashes Faiss (instead of just leaking memory).
Findings:
- when running in MT mode, each search in an indexflat used as coarse quantizer consumes some memory
- this mem consumption does not appear in single-thread mode or with few threads
- in gdb it appears that even when the nb of queries is 1, each search spawns max_threads threads (80 on the test machine)
This diff:
- adds a C++ test that checks how much mem is used when repeatedly searching a vector
- adjusts the number of search threads to the number of query vectors. This is especially useful for single-vector queries.
Reviewed By: beauby
Differential Revision: D31142383
fbshipit-source-id: 134ddaf141e7c52a854cea398f5dbf89951a7ff8
Matthijs Douze