A high-performance, zero-overhead, extensible Python compiler using LLVM
 
 
 
 
 
Go to file
A. R. Shajii 5613c1a84b
v0.16 (#335)
* Add Python extension lowering pass

* Add DocstringAttribute

* Add extension module codegen

* Handle different argument counts efficiently

* Add warnings to extension lowering

* Fix module name

* Fix extension codegen

* Fix argument check

* Auto-convert Codon exceptions to Python exceptions

* Fix #183

* Fix #162; Fix #135

* Fix #155

* Fix CPython interface in codegen

* Fix #191

* Fix #187

* Fix #189

* Generate object file in pyext mode

* Convert Codon exceptions to Python exceptions

* Fix vtable init; Fix failing tests on Linux

* Fix #190

* Fix #156

* Fix union routing

* Remove need for import python

* Automatic @export and wrapping for toplevel functions

* Reorganize API

* Add Python extension IR structs

* Add special calls for no-suspend yield-expr

* Add special calls for no-suspend yield-expr

* pyextension.h support [wip]

* pyextension.h support [wip]

* pyextension.h support

* pyextension.h support for toplevel functions

* clang-format

* Add PyFunction::nargs field

* Update pyextension codegen (WIP)

* SUpport nargs

* Add support for @pycapture

* PyType codegen (WIP)

* Py method codegen (WIP)

* Add type ptr hook

* Add getset codegen

* Add type alloc function

* Add type pointer hook codegen

* Re-organize codegen

* Add member codegen

* Update module init codegen

* Update module init codegen

* Add support for typePtrHook and new to/from_py hooks

* Fix extension codegen

* Fix init codegen

* Fix init codegen; add "tp_new" slot

* Fix type hook

* Add extra flags

* Specialized wrappers (PyType specs)

* Add static Python link option

* Fix C imports

* Add guards

* Remove unused field

* Python mode only when pyExt set

* Update python module

* Fix assert

* Update codegen/passes

* Fix tuple parsing in index expression

* Fix empty tuple unification

* Do not Cythonize underscore fns

* clang-format

* Fix switch

* Add Py support for cmp/setitem

* Add Py support for cmp/setitem

* Add type is support

* GetSet support

* clang-format

* GetSet support (fixes)

* Avoid useless vtable alloc

* Add iter support

* Fix size_t capture bug

* clang-format

* Fix POD type unification with tuples

* Add __try_from_py__ API

* Fix annotation

* Add static reflection methods (setattr; internal.static.*); refactor PyExt to python.codon; handle errors and kwargs in PyExt

* Python compat fixes

* Update Python object conversions

* Fix PyErrors

* clang-format; add copyright

* Add PyFunction::keywords field

* Fix JIT MRO handling; Refactor out Jupyter support

* Refactor out Jupyter support

* Add support for custom linking args (link=[]) to TOML plugins

* Fix tests

* Use g++ instead of gcc

* Fix Jupyter CMAKE

* Fix Jupyter CMAKE

* Add _PyArg_Parser definition

* Add complex64 type

* Add extra complex64 tests

* Fix Python calls; add staticenumerate

* Fix call

* Fix calls

* Update pyext wrappers

* Fix staticenumerate; Support static calls in tuple()

* Fix pyext routing

* Add add/mul for tuples

* clang-format

* Fix pyext codegen

* Fix wrap_multiple

* Add seq_alloc_atomic_uncollectable

* Fix default generics issue

* Add binary/ternary ops

* Fix missing generic issue

* Fix number slots

* Update pow

* Remove unnecessary pyobj

* Fix allocation

* Refactor errors

* Add test extension

* Fix formatting

* clang-format

* Fix getitem/setitem/delitem in pyext

* Fix pyext iterators

* Add builtin pow() (fix #294)

* Fix #244

* Fix #231

* Fix #229

* Fix #205

* Update docs

* Fix error message

* Add pyext tests

* Add pyext support for @property

* Add pyext support for toplevel fns and @tuple classes

* More pyext tests

* More pyext tests

* Fix file error checking

* More pyext tests

* Update pyext tests

* Update docs

* Add pyext test to CI

* Add pyext support for @tuple.__new__

* Add pyext support for @tuple.__new__

* Fix hetero-tuple issue with fn_overloads

* More pyext tests

* Bump versions

* Fix del magic in pyext

* Fix init magic for tuples in pyext

* Have test-pypi only run on develop branch

* Make exception type indices unnamed-addr

* Fix #316; Fix #317 (slash issue)

* Use uncollectible-alloc for vtable

* Fix #249

* Add pyext docs

* Fix #249; Fix clashing vtables; Fix super() and class_copy

* Add content-atomic type property instruction

* __contents_atomic__ support

* Update internal functions

* Use PIC when generating Python extension

* Cleanup

* Add Dockerfile & fix -fPIC

* Cleanup

* Fix setup.py

* Fix pyext fn iteration

* Fix CI

* clang-format

* Update long conversions in Py bridge

* Support wide-int to str conversions

* Fix test

* Add pow for arbitrary-width ints

* Fix Linux backtraces

* Cleanup

* Add more tests

* Fix docs; Remove tuple.__add__ for scalars

* Update docs

---------

Co-authored-by: Ibrahim Numanagić <ibrahimpasa@gmail.com>
2023-04-12 18:13:54 -04:00
.github v0.16 (#335) 2023-04-12 18:13:54 -04:00
bench Spelling (#276) 2023-03-20 19:13:39 -04:00
cmake v0.16 (#335) 2023-04-12 18:13:54 -04:00
codon v0.16 (#335) 2023-04-12 18:13:54 -04:00
docs v0.16 (#335) 2023-04-12 18:13:54 -04:00
extra/python v0.16 (#335) 2023-04-12 18:13:54 -04:00
jupyter v0.16 (#335) 2023-04-12 18:13:54 -04:00
scripts v0.16 (#335) 2023-04-12 18:13:54 -04:00
stdlib v0.16 (#335) 2023-04-12 18:13:54 -04:00
test v0.16 (#335) 2023-04-12 18:13:54 -04:00
.clang-format
.clang-tidy
.gitattributes
.gitignore v0.16 (#335) 2023-04-12 18:13:54 -04:00
CMakeLists.txt v0.16 (#335) 2023-04-12 18:13:54 -04:00
CODEOWNERS Dynamic Polymorphism (#58) 2022-12-04 19:45:21 -05:00
CONTRIBUTING.md Dynamic Polymorphism (#58) 2022-12-04 19:45:21 -05:00
LICENSE Dynamic Polymorphism (#58) 2022-12-04 19:45:21 -05:00
README.md Plugin loading fixes (#66) 2022-12-07 22:42:29 -05:00
book.json

README.md

Codon

Docs  |  FAQ  |  Blog  |  Forum  |  Chat  |  Benchmarks

Build Status

What is Codon?

Codon is a high-performance Python compiler that compiles Python code to native machine code without any runtime overhead. Typical speedups over Python are on the order of 10-100x or more, on a single thread. Codon's performance is typically on par with (and sometimes better than) that of C/C++. Unlike Python, Codon supports native multithreading, which can lead to speedups many times higher still. Codon grew out of the Seq project.

Install

Pre-built binaries for Linux (x86_64) and macOS (x86_64 and arm64) are available alongside each release. Download and install with:

/bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)"

Or you can build from source.

Examples

Codon is a Python-compatible language, and many Python programs will work with few if any modifications:

def fib(n):
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()
fib(1000)

The codon compiler has a number of options and modes:

# compile and run the program
codon run fib.py
# 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

# compile and run the program with optimizations enabled
codon run -release fib.py
# 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

# compile to executable with optimizations enabled
codon build -release -exe fib.py
./fib
# 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

# compile to LLVM IR file with optimizations enabled
codon build -release -llvm fib.py
# outputs file fib.ll

See the docs for more options and examples.

This prime counting example showcases Codon's OpenMP support, enabled with the addition of one line. The @par annotation tells the compiler to parallelize the following for-loop, in this case using a dynamic schedule, chunk size of 100, and 16 threads.

from sys import argv

def is_prime(n):
    factors = 0
    for i in range(2, n):
        if n % i == 0:
            factors += 1
    return factors == 0

limit = int(argv[1])
total = 0

@par(schedule='dynamic', chunk_size=100, num_threads=16)
for i in range(2, limit):
    if is_prime(i):
        total += 1

print(total)

Codon supports writing and executing GPU kernels. Here's an example that computes the Mandelbrot set:

import gpu

MAX    = 1000  # maximum Mandelbrot iterations
N      = 4096  # width and height of image
pixels = [0 for _ in range(N * N)]

def scale(x, a, b):
    return a + (x/N)*(b - a)

@gpu.kernel
def mandelbrot(pixels):
    idx = (gpu.block.x * gpu.block.dim.x) + gpu.thread.x
    i, j = divmod(idx, N)
    c = complex(scale(j, -2.00, 0.47), scale(i, -1.12, 1.12))
    z = 0j
    iteration = 0

    while abs(z) <= 2 and iteration < MAX:
        z = z**2 + c
        iteration += 1

    pixels[idx] = int(255 * iteration/MAX)

mandelbrot(pixels, grid=(N*N)//1024, block=1024)

GPU programming can also be done using the @par syntax with @par(gpu=True).

What isn't Codon?

While Codon supports nearly all of Python's syntax, it is not a drop-in replacement, and large codebases might require modifications to be run through the Codon compiler. For example, some of Python's modules are not yet implemented within Codon, and a few of Python's dynamic features are disallowed. The Codon compiler produces detailed error messages to help identify and resolve any incompatibilities.

Codon can be used within larger Python codebases via the @codon.jit decorator. Plain Python functions and libraries can also be called from within Codon via Python interoperability.

Documentation

Please see docs.exaloop.io for in-depth documentation.