Commit Graph

10 Commits (dcbf33c525cd79e1cdd05d1a6fa5092829bd7cb4)

Author SHA1 Message Date
Lucas Hosseini 7492d23354 Fix packaging (#2121)
Summary:
- Disable problematic tests on OSX.
- Ensure compiler compatibility with CUDA builds.
- Fix path for Python extension libraries.
- Use CentOS for CUDA packaging.
- Update CUDA versions in CI (10.2 and 11.3).

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2121

Reviewed By: mdouze

Differential Revision: D32921117

Pulled By: beauby

fbshipit-source-id: 588c18add8084b8228ff5abc651eaa4567919cc6
2021-12-07 13:12:30 -08:00
Matthijs Douze 2d380e992b Add manifold check for size 0 (#1867)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1867

Merging code for the 1T photodna index seems to fail at

https://www.internalfb.com/phabricator/paste/view/P412975011?lines=174

with
```
terminate called after throwing an instance of 'facebook::manifold::blobstore::StorageException'
  what():  [400] Begin offset and/or length were invalid -- Begin offset must be positive and length must be non-negative. Received: offset = 2642410612, length = 0
Aborted (core dumped)
```
traces back to

https://www.internalfb.com/intern/diffusion/FBS/browsefile/master/fbcode/manifold/blobstore/BlobstoreThriftHandler.cpp?lines=671%2C700%2C732

There is a single case where we don't check if the read or write size is 0. So let's try this fix.

In the process I realized that the Manifold tests were non functional due to a name collision on common.py. Also fix this in all dependent files.

Differential Revision: D28231710

fbshipit-source-id: 700ffa6ca0c82c49e7d1eae9e76549ec5ff16332
2021-05-09 22:30:31 -07:00
Matthijs Douze 4dc5b27a38 Fix test failure in OSX with OpenMP called from multiple threads (#1849)
Summary:
Need to add an ssh key to the circleci to be able to debug

For my own ref, how to connect to the job:
```
[matthijs@matthijs-mbp /Users/matthijs/Desktop/faiss_github/circleci_keys] ssh -p 54782 38.39.188.110 -i id_ed25519
```

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1849

Reviewed By: wickedfoo

Differential Revision: D28234897

Pulled By: mdouze

fbshipit-source-id: 6827fa45f24b3e4bf586315bd38f18608d07ecf9
2021-05-06 05:18:34 -07:00
Matthijs Douze 27077c4627 Small fixes for compilation on ARM (#1655)
Summary:
This PR fixes a few small issues with compilation on ARM.
It has been tested on an AWS c6g.8xlarge machine with Ubuntu 18.04.5 LTS.
Compilation instructions are here: https://github.com/facebookresearch/faiss/wiki/Installing-Faiss#compiling-faiss-on-arm

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1655

Reviewed By: wickedfoo

Differential Revision: D26145921

Pulled By: mdouze

fbshipit-source-id: 007e57a610f489885e78ba22bc82605d67661c44
2021-01-29 10:06:45 -08:00
Matthijs Douze 6d73c2ff69 Fix int64 for python tests in windows (#1381)
Summary:
`long` is 32 bits on windows and so is the default int type for numpy (eg. the one used for `np.arange`).
This diff explicitly specifies 64-bit ints for all occurrences where it matters.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1381

Reviewed By: wickedfoo

Differential Revision: D23371232

Pulled By: mdouze

fbshipit-source-id: 220262cd70ee70379f83de93561a4eae71c94b04
2020-08-27 12:40:55 -07:00
Lucas Hosseini 22b7876ef5
Facebook sync (2020-03-10) (#1136) 2020-03-10 14:24:07 +01:00
Lucas Hosseini 3896b12c65
Facebook sync (Jun 2019) (#862)
Bugfixes:
- slow scanning of inverted lists (#836).

Features:
- add basic support for 6 new metrics in CPU `IndexFlat` and `IndexHNSW` (#848);
- add support for `IndexIDMap`/`IndexIDMap2` with binary indexes (#780).

Misc:
- throw python exception for OOM (#758);
- make `DistanceComputer` available for all random access indexes;
- gradually moving from `long` to `int64_t` for portability.
2019-06-19 15:59:06 +02:00
Lucas Hosseini a8118acbc5
Facebook sync (May 2019) + relicense (#838)
Changelog:

- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
2019-05-28 16:17:22 +02:00
Lucas Hosseini f417a53628
Fix CI tests (#687)
* Fix test_transfer_invlists.cpp

* Fix relative imports.

* Fix test_index_accuracy.py.

* Use default OSX version.

* Allow osx gcc6 build to fail.
2019-01-08 17:52:36 +01:00
Lucas Hosseini 323dbf3be3
Facebook sync (Dec 2018). (#660)
* Add GpuIndexBinaryFlat
* Add IndexBinaryHNSW
2018-12-19 17:48:35 +01:00