Summary:
In this commit ab2b7f5093, they changed format based on clang-format-18. However, we still use clang-format-11 in our circle ci job which caused the failure. In this PR, we are going to switch to clang-format-18
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3372
Reviewed By: kuarora
Differential Revision: D56280363
Pulled By: junjieqi
fbshipit-source-id: f832ab2112f762e6000b55a155e3e43fe99071d7
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3317
libraft packages are first published in rapidsai-nightly and moved to rapidsai after release, at which point they're removed from rapidsai-nightly
In this diff we enable both channels with a preference to rapidsai (since it's before rapidsai-nightly on the command line).
Reviewed By: mlomeli1
Differential Revision: D55310143
fbshipit-source-id: b85e0fda86a442f435d985ace1d7eb37209c74e1
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3295
In the past, we had build failure due to the latest swig version in conda-forge so we had to specify the version of swig. In this diff, we are going to change it to be the latest version always because the issue has gone.
Reviewed By: algoriddle
Differential Revision: D54975271
fbshipit-source-id: 7ca59fb58390edb0cc5ed52f6fd416f633dd7938
Summary:
libraft 24.02 is now in the main channel of rapidsai (and no longer in the nightly), switching channel we build against to prepare for the release of faiss-gpu-raft
cmake build will continue to use the nightly, but the conda packages will be built against the release
Reviewed By: mlomeli1
Differential Revision: D53712646
fbshipit-source-id: de60c3336e6a199b496fc3829d71a778e5ea8cbd
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3248
Build scripts can't find conda in the 'stable' image, trying to pin to a specific version.
Reviewed By: mlomeli1
Differential Revision: D53608347
fbshipit-source-id: 3f7c350608afc9c253cf0153e37ceeb4fceca92c
Summary:
Considering that only GPU classic works on Pascal and only if compiled with cuda 11.4, this sets up the conda builds as follows.
tl;dr: Pascal will only be supported by faiss-gpu on cuda 11.
## faiss-gpu
1. build on cuda 11.4.4, supports Pascal, works with pytorch-cuda=11 (including hosts with cuda 12 drivers)
2. build on cuda 12.1.1, does NOT support Pascal, works with pytorch-cuda=12
## faiss-gpu-raft
1. build on cuda 11.8.0, does NOT support Pascal, works with pytorch-cuda=11.8
2. build on cuda 12.1.1, does NOT support Pascal, works with pytorch-cuda=12
The reason faiss-gpu-raft is built with and supports only 11.8 is due to the risk of clobbering otherwise between libraft and pytorch-cuda dependencies.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3237
Reviewed By: mdouze
Differential Revision: D53354121
Pulled By: algoriddle
fbshipit-source-id: 86108e975168759572f84922bf0ccef55ae75ef8
Summary:
1. Split the GPU code into a separate static library to reduce build times.
2. faiss-gpu now depends on pytorch-cuda to stop clobbering of CUDA libraries when Faiss and Pytorch installed in the same environment
3. Add CUDA 12 (we support both 11.8 and 12.1)
4. Add Python 3.12
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3222
Reviewed By: mlomeli1
Differential Revision: D53215033
Pulled By: algoriddle
fbshipit-source-id: d48c5707af20b7e5ff72c3aa27b0e677259f22f8
Summary:
The CMake CUDA Architecture value of `60` means to generate both PTX and SASS for that arch. We only need SASS for the architectures we support, and one PTX version for future hardware versions.
So now we build on SASS for everything ( `60-real` ) and use 80 as the baseline for newer archs likes 90
By removing this unneeded PTX code we can reduce the libfaiss.a binary to 305MB from the current 484MB.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3083
Reviewed By: wickedfoo
Differential Revision: D49901896
Pulled By: algoriddle
fbshipit-source-id: 15e98f81e191a565319cf855debad33b24ebf10b
Summary:
Moving the raft build to a nightly, to remove the noise from the PR contbuilds.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2926
Reviewed By: mlomeli1
Differential Revision: D47016318
Pulled By: algoriddle
fbshipit-source-id: 3c60aa382b9aa68dcadb929e0e4afade13c9123e
Summary:
This fixes the build, except for MacOS, where there's a problem with cmake + OpenMP. We can fix it separately.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2856
Reviewed By: mlomeli1
Differential Revision: D45704458
Pulled By: algoriddle
fbshipit-source-id: 0c09036ae5fa34ab114b857f407a35603986613a
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2835
I don't think this provides much extra signal on top of Linux arm64, and we will still have the nightly. It's by far the most expensive to run, so let's save the $$$ and the planet.
Reviewed By: mlomeli1
Differential Revision: D45389825
fbshipit-source-id: 63fa6b37f3f7505f118c75f03605c065f1ad51f1
Summary:
RAFT requires cmake 3.23.1, pulling it from conda-forge. We continue to keep the dependency on conda-forge minimal, hence the ordering of the channels and the pinning of sysroot to a specific version.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2808
Test Plan:
contbuild
Imported from GitHub, without a `Test Plan:` line.
Reviewed By: mlomeli1
Differential Revision: D44746827
Pulled By: algoriddle
fbshipit-source-id: ad576b11b257203bd0cafd57c2c2e7fd8d10ca98
Summary:
1. GPU builds use CircleCI base image, no docker
2. Switched to CUDA 11.4 (used to be 11.3)
3. Merged all build jobs into two parameterized targets: `build_cmake` and `build_conda`.
4. Cleaned up test execution, fixed bug of Python GPU tests not running on PRs
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2803
Reviewed By: mlomeli1
Differential Revision: D44541714
Pulled By: algoriddle
fbshipit-source-id: aa09ae638ecb6ef556d42f27a4bfaddad7355e50
Summary:
# Refactor
- Extract executors and merge all conda builds (that run on diffs) and conda package builds (that run nightly and on release) into two parameterized targets (`build_conda`, `deploy_conda`), except for GPU builds (at least for now)
- Similarly, introduce a `build_cmake` target that can be parameterized with executor, although run it for Linux x86_64 only (for now)
- Keep GPU targets separate (both conda package build and cmake) for now. Introduce "v2" targets that will eventually replace the current GPU build targets (we need to resolve GPU test failures).
- Removed `beauby/faiss-circleci:cpu` docker container, use the miniconda docker for Linux and machine images everywhere else. v2 GPU targets use the latest circleci images (see https://discuss.circleci.com/t/cuda-11-8-gpu-cuda-image-any-plans/47240/3)
# New/changed functionality
- Dropped CUDA 10
- Support for Linux arm64 conda packages
- Workflows have a consistent naming scheme, `OSX arm64 (conda)`, `Linux x86_64 (cmake)` etc.
- No cmake build for Linux or OSX arm64, replaced both with a conda build target only. We can reintroduce arm64 cmake workflows for both if needed (via additional parameterized build_cmake workflows), but it seemed unnecessary to me.
# Next steps
- Make v2 GPU builds work, deprecate v1, get rid of all docker stuff
- Merge GPU builds into cmake/conda build targets
- Possibly further unify package build and conda build targets
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2798
Test Plan: contbuild
Reviewed By: mlomeli1
Differential Revision: D44469783
Pulled By: algoriddle
fbshipit-source-id: 8489942fb7a4e4de1dd2d4466790e550191d15a1
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2791
Removed building for 3.7 (D44373285 - not supported for M1), adding 3.10 with this diff.
The only change that matters is in `conda_build_config.yaml`, the others are about making the configs consistent between CPU and GPU.
Reviewed By: mlomeli1
Differential Revision: D44405573
fbshipit-source-id: ad933e08834593e55a35075c602e5f509a813e73
Summary:
Fixes OSX CI by pinning pytorch version for interop tests. The "real" fix is already landed in pytorch but has not been released yet.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2482
Reviewed By: alexanderguzhva
Differential Revision: D39891113
Pulled By: beauby
fbshipit-source-id: fa79bf9de1c93e056260ea64613e37625edfecc3
Summary:
It seems that [`xcode:12.4.0` has been retired from CircleCI](https://circleci.com/docs/en/using-macos#supported-xcode-versions), so this PR updates the image version and fixes stopping the pipeline at the spin-up stage.
This PR changes only `.circleci/config.yml` , and doesn't affect the software behavior.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2442
Reviewed By: beauby
Differential Revision: D39259341
Pulled By: mdouze
fbshipit-source-id: 8c7b0f8eb6f6f951329b4e2a2964672d0ee75ceb
Summary:
A recent CUDA driver is required for building packages for CUDA 11.3.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2146
Reviewed By: wickedfoo
Differential Revision: D33020204
Pulled By: beauby
fbshipit-source-id: 01257b1dcb4987f4866cc058c22d1dd5977d76ce
Summary:
- Disable problematic tests on OSX.
- Ensure compiler compatibility with CUDA builds.
- Fix path for Python extension libraries.
- Use CentOS for CUDA packaging.
- Update CUDA versions in CI (10.2 and 11.3).
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2121
Reviewed By: mdouze
Differential Revision: D32921117
Pulled By: beauby
fbshipit-source-id: 588c18add8084b8228ff5abc651eaa4567919cc6
Summary:
This is required for the renaming of the default branch from `master` to `main`, in accordance with the new Facebook OSS guidelines.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2029
Reviewed By: mdouze
Differential Revision: D30672862
Pulled By: beauby
fbshipit-source-id: 0b6458a4ff02a12aae14cf94057e85fdcbcbff96
Summary:
Currently CI jobs using conda are failed due to conflict packages.
This PR fixes this.
- use newer `numpy` to build `faiss-cpu`
- install `pytorch` when testing `faiss-cpu`
- to find correct `pytorch` package, `pytorch` channel is set at `conda build`
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1884
Reviewed By: mdouze
Differential Revision: D28777447
Pulled By: beauby
fbshipit-source-id: 82a1ce076abe6bbbba9415e8935ed57b6104b6c3
Summary:
Also remove support for deprecated compute capabilities 3.5 and 5.2 in
CUDA 11.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1899
Reviewed By: mdouze
Differential Revision: D28539826
Pulled By: beauby
fbshipit-source-id: 6e8265f2bfd991ff3d14a6a5f76f9087271f3f75
