Commit Graph

44 Commits (1d4b1571e99eb6cf74b279d3f9df4e1bf44401d1)

Author SHA1 Message Date
Lucas Hosseini a97ebeeabd Fix format specifiers for size_t/idx_t. (#1338)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1338

Test Plan: Imported from OSS

Reviewed By: mdouze

Differential Revision: D23234964

Pulled By: beauby

fbshipit-source-id: 2222d90d8ca3f9a6ac3a9e624e7ec87072da3a25
2020-08-20 10:45:27 -07:00
Lucas Hosseini f3727a62f5 Remove python shebangs in tests.
Reviewed By: mdouze

Differential Revision: D23167048

fbshipit-source-id: 98196f489461bc922e6124e88e0bfb32dfe507ca
2020-08-17 11:46:26 -07:00
Lucas Hosseini b539a73e58 Linter auto-fix.
Summary: `arc lint faiss/**/*`

Reviewed By: LowikC

Differential Revision: D22891305

fbshipit-source-id: 45bab7294ccccf70898b4967b03683894b6ae4c4
2020-08-16 19:52:27 -07:00
Lucas Hosseini b7b261cad1 Move from TravisCI to CircleCI (#1315)
Summary:
Depends on https://github.com/facebookresearch/faiss/issues/1313.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1315

Reviewed By: mdouze

Differential Revision: D23148557

Pulled By: beauby

fbshipit-source-id: 0a35f17d22aa04db6bd1c16cfc5ff8eee28f1f74
2020-08-15 04:00:51 -07:00
Lucas Hosseini a8e4c5e2d5 Move build to CMake (#1313)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1313

Reviewed By: mdouze

Differential Revision: D22948267

Pulled By: beauby

fbshipit-source-id: ec16fa0342f37672d46fb7886ecc55c7996011c4
2020-08-14 15:03:10 -07:00
Matthijs Douze 4c3b5ad156 Add missing downcast (#1330)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1330

add missing downcast for Index (and test for github sync)

See github issue https://github.com/facebookresearch/faiss/issues/1278

Reviewed By: beauby

Differential Revision: D23053182

fbshipit-source-id: 1ce4c248342332ce632ecfd5074affa3ddf55b66
2020-08-13 16:37:58 -07:00
Lucas Hosseini ac74f576f7 fbshipit-source-id: 4f3cfa59471d548af93fe118d1b73d45bc648edf 2020-08-04 12:00:38 -07:00
Lucas Hosseini cd38e82f0c
Facebook sync 2020-07-31 (#1308) 2020-08-03 22:15:02 +02:00
Matthijs Douze e8c8db85ae fix #1213 2020-06-02 13:59:39 -07:00
Lucas Hosseini a17a631dc3
Sync 20200323. (#1157)
* Sync 20200323.

* Bump version.

* Remove warning filter.
2020-03-24 14:06:48 +01:00
Lucas Hosseini 22b7876ef5
Facebook sync (2020-03-10) (#1136) 2020-03-10 14:24:07 +01:00
Lucas Hosseini 2ba6985f81 Facebook sync 20191129 (#1048)
Looks good!
2019-12-04 07:21:02 +01:00
Lucas Hosseini 36ddba9196
Facebook sync (2019-09-10) (#943)
* Facebook sync (2019-09-10)

* Fix depends Makefile target.

* Add faiss symlink for new include directives.

* Fix missing header.

* Fix tests.

* Fix Makefile.

* Update depend.

* Fix include directives spacing.
2019-09-20 18:59:10 +02:00
Lucas Hosseini 3896b12c65
Facebook sync (Jun 2019) (#862)
Bugfixes:
- slow scanning of inverted lists (#836).

Features:
- add basic support for 6 new metrics in CPU `IndexFlat` and `IndexHNSW` (#848);
- add support for `IndexIDMap`/`IndexIDMap2` with binary indexes (#780).

Misc:
- throw python exception for OOM (#758);
- make `DistanceComputer` available for all random access indexes;
- gradually moving from `long` to `int64_t` for portability.
2019-06-19 15:59:06 +02:00
Lucas Hosseini a8118acbc5
Facebook sync (May 2019) + relicense (#838)
Changelog:

- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
2019-05-28 16:17:22 +02:00
Lucas Hosseini 7f5b22b0ff
Add conda packages metadata + tests. (#769)
+ Add conda packages metadata (now building Faiss using conda's toolchain);
+ add Dockerfile for building conda packages (for all CUDA versions);
+ add working Dockerfile building faiss on Centos7;
+ simplify GPU build;
+ avoid falling back to CPU-only version (python);
+ simplify TravisCI config;
+ update INSTALL.md;
+ add configure flag for specifying target architectures (--with-cuda-arch);
+ fix Makefile for gpu tests;
+ fix various Makefile issues;
+ remove stale file (gpu/utils/DeviceUtils.cpp).
2019-04-05 11:50:39 +02:00
Lucas Hosseini afe0fdc161
Facebook sync (Mar 2019) (#756)
Facebook sync (Mar 2019)

- MatrixStats object
- option to round coordinates during k-means optimization
- alternative option for search in HNSW
- moved stats and imbalance_factor of IndexIVF to InvertedLists object
- range search for IVFScalarQuantizer
- direct unit8 codec in ScalarQuantizer
- renamed IndexProxy to IndexReplicas and moved to main Faiss
- better support for PQ code assignment with external index
- support for IMI2x16 (4B virtual centroids!)
- support for k = 2048 search on GPU (instead of 1024)
- most CUDA mem alloc failures throw exceptions instead of terminating on an assertion
- support for renaming an ondisk invertedlists
- interrupt computations with ctrl-C in python
2019-03-29 16:32:28 +01:00
Lucas Hosseini f417a53628
Fix CI tests (#687)
* Fix test_transfer_invlists.cpp

* Fix relative imports.

* Fix test_index_accuracy.py.

* Use default OSX version.

* Allow osx gcc6 build to fail.
2019-01-08 17:52:36 +01:00
Lucas Hosseini ac7005b6ef Remove CMake. (#645) 2018-12-23 18:45:16 +01:00
matthijs daf589d9d2 add bench_all_ivf 2018-12-20 05:43:36 -08:00
Lucas Hosseini 323dbf3be3
Facebook sync (Dec 2018). (#660)
* Add GpuIndexBinaryFlat
* Add IndexBinaryHNSW
2018-12-19 17:48:35 +01:00
Tom Forbes a91a24e77a Fix #558 - Make `M` an integer (#589)
* Fix #558 - Make `d` an integer

* Use int()
2018-09-18 22:08:31 +02:00
Lucas Hosseini 76bec0b500
Facebook sync (#573)
Features:

- automatic tracking of C++ references in Python
- non-intel platforms supported -- some functions optimized for ARM
- override nprobe for concurrent searches
- support for floating-point quantizers in binary indexes
Bug fixes:

- no more segfaults in python (I know it's the same as the first feature but it's important!)
- fix GpuIndexIVFFlat issues for float32 with 64 / 128 dims
- fix sharding of flat indexes on GPU with index_cpu_to_gpu_multiple
2018-08-30 19:38:50 +02:00
Yohei Kikuta 67c6506bcc Replace os.tempnam with tempfile for python3 (#537)
os.tempnam is eliminated in python3.
Used tempfile.NamedTemporaryFile().name instead.
This is also valid for python2.7.
2018-07-24 10:25:40 +02:00
Lucas Hosseini 6880286ea0
Facebook sync (#504)
* Facebook sync

* Update swig wrappers.

* Fix comment.
2018-07-06 14:12:11 +02:00
Lucas Hosseini 6e40d6689f
Move python tests back together with C++ tests. (#479) 2018-06-04 12:20:44 +02:00
Lucas Hosseini cf18101f6d Refactor makefiles and add configure script (#466)
* Refactors Makefiles and add configure script.

* Give MKL higher priority in configure script.

* Clean up Linux example makefile.inc.

* Cleanup makefile.inc examples.

* Fix python clean Makefile target.

* Regen swig wrappers.

* Remove useless CUDAFLAGS variable.

* Fix python linking flags.

* Separate compile and link phase in python makefile.

* Add macro to look for swig.

* Add CUDA check in configure script.

* Cleanup make depend targets.

* Cleanup CUDA flags.

* Fix linking flags.

* Fix python GPU linking.

* Remove useless flags from python gpu module linking.

* Add check for cuda libs.

* Cleanup GPU targets.

* Clean up test target.

* Add cpu/gpu targets to python makefile.

* Clean up tutorial Makefile.

* Remove stale OS var from example makefiles.

* Clean up cuda example flags.
2018-06-02 08:35:30 +02:00
Ailing cd884114d0 Make tests compatible with py3 (#348) 2018-02-24 00:38:45 +01:00
Matthijs Douze 0c482e54eb sync with FB version 2018-02-23 (#347)
- support on-disk IVF
2018-02-23 07:49:45 -08:00
Lucas Hosseini db855a2146
Add Makefile for tests. (#345) 2018-02-23 16:08:40 +01:00
matthijs 9933892ec9 sync with FB version 2017-01-09
- adding HNSW indexing method

- simultaneous search and reconstruction for IndexIVFPQ
2018-01-09 06:42:06 -08:00
Matthijs Douze e652a6648f
Update demo_auto_tune.py 2017-11-23 16:52:31 +01:00
matthijs 5f684a4627 Merge branch 'master' of https://github.com/facebookresearch/faiss 2017-11-23 06:46:00 -08:00
matthijs fe446e41a2 bugfixes 2017-11-23 06:34:53 -08:00
Soumith Chintala de8ac33f2e fix test syntax (#260) 2017-11-23 10:40:06 +01:00
matthijs 250a3d3f18 sync with FB version 2017-11-22
various bugfixes from github issues
kmean with some frozen centroids
GPU better tiling for large flat datasets
default AVX for vector ops
2017-11-22 05:11:28 -08:00
matthijs a5ef16db89 sync with FB version 2017-08-09 2017-08-09 11:13:51 -07:00
matthijs 8e3dc6f2b0 changed license 2017-07-30 00:18:45 -07:00
matthijs 12f181ee44 forgotten 2017-07-18 02:55:11 -07:00
matthijs f7aedbdfc0 sync with FB version 2017-07-18
- implemented ScalarQuantizer (without IVF)
- implemented update for IndexIVFFlat
- implemented L2 normalization preproc
2017-07-18 02:51:27 -07:00
matthijs 784e2facd8 Synchronization with FB version 2017-06-21
* moved most FAISS_ASSERT calls to C++ exceptions, and adjusted
  memory allocation to avoid mem leaks

* added an IndexIVFScalarQuantizer type that offers an
  intermediate compression between IVFFlat and IVFPQ

* support removal of indices in IndexIDMap / IndexFlat combination

* various fixes in GPU code
2017-06-21 09:01:06 -07:00
Adeykin c056f1d320 mkl support in cmake (#123) 2017-05-31 15:41:42 +02:00
Tianwei Shen 80314d9f07 add initial cmake support (#75)
* add initial cmake support

* update cmake, add cmake instructions to INSTALL

* update findopenmp and INSTALL

* change FindOpenBLAS.cmake to cater for macports

- change cblas.h to openblas_config.h since macports does not ship
cblas.h with openblas.

* revise INSTALL for cmake
2017-05-02 11:04:50 +02:00
Matthijs Douze c670118a4e Initial commit 2017-02-22 23:26:44 +01:00