mirror of
https://github.com/exaloop/codon.git
synced 2025-06-03 15:03:52 +08:00
* Add Python extension lowering pass * Add DocstringAttribute * Add extension module codegen * Handle different argument counts efficiently * Add warnings to extension lowering * Fix module name * Fix extension codegen * Fix argument check * Auto-convert Codon exceptions to Python exceptions * Fix #183 * Fix #162; Fix #135 * Fix #155 * Fix CPython interface in codegen * Fix #191 * Fix #187 * Fix #189 * Generate object file in pyext mode * Convert Codon exceptions to Python exceptions * Fix vtable init; Fix failing tests on Linux * Fix #190 * Fix #156 * Fix union routing * Remove need for import python * Automatic @export and wrapping for toplevel functions * Reorganize API * Add Python extension IR structs * Add special calls for no-suspend yield-expr * Add special calls for no-suspend yield-expr * pyextension.h support [wip] * pyextension.h support [wip] * pyextension.h support * pyextension.h support for toplevel functions * clang-format * Add PyFunction::nargs field * Update pyextension codegen (WIP) * SUpport nargs * Add support for @pycapture * PyType codegen (WIP) * Py method codegen (WIP) * Add type ptr hook * Add getset codegen * Add type alloc function * Add type pointer hook codegen * Re-organize codegen * Add member codegen * Update module init codegen * Update module init codegen * Add support for typePtrHook and new to/from_py hooks * Fix extension codegen * Fix init codegen * Fix init codegen; add "tp_new" slot * Fix type hook * Add extra flags * Specialized wrappers (PyType specs) * Add static Python link option * Fix C imports * Add guards * Remove unused field * Python mode only when pyExt set * Update python module * Fix assert * Update codegen/passes * Fix tuple parsing in index expression * Fix empty tuple unification * Do not Cythonize underscore fns * clang-format * Fix switch * Add Py support for cmp/setitem * Add Py support for cmp/setitem * Add type is support * GetSet support * clang-format * GetSet support (fixes) * Avoid useless vtable alloc * Add iter support * Fix size_t capture bug * clang-format * Fix POD type unification with tuples * Add __try_from_py__ API * Fix annotation * Add static reflection methods (setattr; internal.static.*); refactor PyExt to python.codon; handle errors and kwargs in PyExt * Python compat fixes * Update Python object conversions * Fix PyErrors * clang-format; add copyright * Add PyFunction::keywords field * Fix JIT MRO handling; Refactor out Jupyter support * Refactor out Jupyter support * Add support for custom linking args (link=[]) to TOML plugins * Fix tests * Use g++ instead of gcc * Fix Jupyter CMAKE * Fix Jupyter CMAKE * Add _PyArg_Parser definition * Add complex64 type * Add extra complex64 tests * Fix Python calls; add staticenumerate * Fix call * Fix calls * Update pyext wrappers * Fix staticenumerate; Support static calls in tuple() * Fix pyext routing * Add add/mul for tuples * clang-format * Fix pyext codegen * Fix wrap_multiple * Add seq_alloc_atomic_uncollectable * Fix default generics issue * Add binary/ternary ops * Fix missing generic issue * Fix number slots * Update pow * Remove unnecessary pyobj * Fix allocation * Refactor errors * Add test extension * Fix formatting * clang-format * Fix getitem/setitem/delitem in pyext * Fix pyext iterators * Add builtin pow() (fix #294) * Fix #244 * Fix #231 * Fix #229 * Fix #205 * Update docs * Fix error message * Add pyext tests * Add pyext support for @property * Add pyext support for toplevel fns and @tuple classes * More pyext tests * More pyext tests * Fix file error checking * More pyext tests * Update pyext tests * Update docs * Add pyext test to CI * Add pyext support for @tuple.__new__ * Add pyext support for @tuple.__new__ * Fix hetero-tuple issue with fn_overloads * More pyext tests * Bump versions * Fix del magic in pyext * Fix init magic for tuples in pyext * Have test-pypi only run on develop branch * Make exception type indices unnamed-addr * Fix #316; Fix #317 (slash issue) * Use uncollectible-alloc for vtable * Fix #249 * Add pyext docs * Fix #249; Fix clashing vtables; Fix super() and class_copy * Add content-atomic type property instruction * __contents_atomic__ support * Update internal functions * Use PIC when generating Python extension * Cleanup * Add Dockerfile & fix -fPIC * Cleanup * Fix setup.py * Fix pyext fn iteration * Fix CI * clang-format * Update long conversions in Py bridge * Support wide-int to str conversions * Fix test * Add pow for arbitrary-width ints * Fix Linux backtraces * Cleanup * Add more tests * Fix docs; Remove tuple.__add__ for scalars * Update docs --------- Co-authored-by: Ibrahim Numanagić <ibrahimpasa@gmail.com>
144 lines
7.5 KiB
Markdown
144 lines
7.5 KiB
Markdown
# Technical
|
|
|
|
## What is Codon?
|
|
|
|
Codon is a high-performance Python compiler that compiles Python code to native machine code
|
|
without any runtime overhead. Typical speedups over Python are on the order of 10-100x or more,
|
|
on a single thread. Codon's performance is typically on par with that of C/C++. Unlike Python,
|
|
Codon supports native multithreading, which can lead to speedups many times higher still.
|
|
Codon is extensible via a plugin infrastructure, which lets you incorporate new libraries,
|
|
compiler optimizations and even keywords.
|
|
|
|
## What isn't Codon?
|
|
|
|
While Codon supports nearly all of Python's syntax, it is not a drop-in replacement, and large
|
|
codebases might require modifications to be run through the Codon compiler. For example, some
|
|
of Python's modules are not yet implemented within Codon, and a few of Python's dynamic features
|
|
are disallowed. The Codon compiler produces detailed error messages to help identify and resolve
|
|
any incompatibilities. Codon supports seamless [Python interoperability](../interop/python.md) to
|
|
handle cases where specific Python libraries or dynamism are required.
|
|
|
|
## How does Codon compare to...
|
|
|
|
- **CPython?** Codon tries to follow CPython's syntax, semantics and APIs as
|
|
closely as possible, aside from a few cases where Codon differs from CPython for
|
|
performance reasons (one example being Codon's 64-bit `int` vs. CPython's arbitrary-
|
|
width `int`). Performance-wise, speedups over CPython are usually on the order of 10-100x.
|
|
|
|
- **Numba?** While Codon does offer a JIT decorator similar to Numba's, Codon is in
|
|
general an ahead-of-time compiler that compiles end-to-end programs to native code.
|
|
It also supports compilation of a much broader set of Python constructs and libraries.
|
|
|
|
- **PyPy?** PyPy strives to effectively be a drop-in replacement for CPython, whereas
|
|
Codon differs in a few places in order to eliminate any dynamic runtime or virtual
|
|
machine, and thereby attain much better performance.
|
|
|
|
- **Cython?** Like Cython, Codon has a [Python-extension build mode](../interop/pyext.md) that
|
|
compiles to Python extension modules, allowing Codon-compiled code to be imported and called
|
|
from plain Python.
|
|
|
|
- **C++?** Codon often generates the same code as an equivalent C or C++ program. Codon
|
|
can sometimes generate *better* code than C/C++ compilers for a variety of reasons, such
|
|
as better container implementations, the fact that Codon does not use object files and
|
|
inlines all library code, or Codon-specific compiler optimizations that are not performed
|
|
with C or C++.
|
|
|
|
- **Julia?** Codon's compilation process is actually much closer to C++ than to Julia. Julia
|
|
is a dynamically-typed language that performs type inference as an optimization, whereas
|
|
Codon type checks the entire program ahead of time. Codon also tries to circumvent the learning
|
|
curve of a new language by adopting Python's syntax and semantics.
|
|
|
|
You can see results from [Codon's benchmark suite](https://github.com/exaloop/codon/tree/develop/bench)
|
|
suite at [exaloop.io/benchmarks](https://exaloop.io/benchmarks).
|
|
More benchmarks can be found in the [2019 paper](https://dl.acm.org/doi/10.1145/3360551)
|
|
on bioinformatics-specific use cases (note that the name used in that paper is that of Codon's predecessor,
|
|
"Seq").
|
|
|
|
## I want to use Codon, but I have a large Python codebase I don't want to port.
|
|
|
|
You can use Codon on a per-function basis via the [`@codon.jit` decorator](../interop/decorator.md),
|
|
which can be used within Python codebases. This will compile only the annotated functions
|
|
and automatically handle data conversions to and from Codon. It also allows for
|
|
the use of any Codon-specific modules or extensions, such as multithreading.
|
|
|
|
## What about interoperability with other languages and frameworks?
|
|
|
|
Interoperability is and will continue to be a priority for Codon.
|
|
We don't want using Codon to render you unable to use all the other great frameworks and
|
|
libraries that exist. Codon supports full interoperability with Python and C/C++.
|
|
|
|
## Does Codon use garbage collection?
|
|
|
|
Yes, Codon uses the [Boehm garbage collector](https://github.com/ivmai/bdwgc).
|
|
|
|
## Codon doesn't support Python module X or function Y.
|
|
|
|
While Codon covers a sizeable subset of Python's standard library, it does not yet cover
|
|
every function from every module. Note that missing functions can still be called through
|
|
Python via `from python import`. Many of the functions that lack Codon-native implementations
|
|
(e.g. I/O or OS related functions) will generally also not see substantial speedups from Codon.
|
|
|
|
## Codon is no faster than Python for my application.
|
|
|
|
Applications that spend most of their time in C-implemented library code generally do not
|
|
see substantial performance improvements in Codon. Similarly, applications that are I/O or
|
|
network-bound will have the same bottlenecks in Codon.
|
|
|
|
## Codon is slower than Python for my application.
|
|
|
|
Please report any cases where Codon is noticeably slower than Python as bugs on our
|
|
[issue tracker](https://github.com/exaloop/codon/issues).
|
|
|
|
# Usage
|
|
|
|
## Is Codon free?
|
|
|
|
Codon is and always will be free for non-production use. That means you can use Codon
|
|
freely for personal, academic, or other non-commercial applications.
|
|
|
|
## Is Codon open source?
|
|
|
|
Codon is licensed under the [Business Source License (BSL)](https://mariadb.com/bsl11/), which
|
|
means its source code is publicly available and it's free for non-production use. The BSL
|
|
is technically *not* an "open source" license, although in many circumstances you can still treat
|
|
Codon as you would any other open source project. Importantly, as per the BSL, each version of
|
|
Codon converts to an actual open source license (specifically, [Apache](https://www.apache.org/licenses/LICENSE-2.0))
|
|
after 3 years.
|
|
|
|
## How can I use Codon for production or commercial use?
|
|
|
|
Please reach out to [info@exaloop.io](mailto:info@exaloop.io) to inquire about a
|
|
production-use license.
|
|
|
|
# Contributing
|
|
|
|
## Does Codon accept outside contributions?
|
|
|
|
Absolutely, we'd be delighted to accept any contributions in the form of issues, bug reports,
|
|
feature requests or pull requests.
|
|
|
|
## I want to contribute. Where do I start?
|
|
|
|
If you have a specific feature or use case in mind, here is a quick breakdown of the codebase
|
|
to help provide a sense of where to look first:
|
|
|
|
- [`codon/`](https://github.com/exaloop/codon/tree/develop/codon): compiler code
|
|
- [`codon/parser/`](https://github.com/exaloop/codon/tree/develop/codon/parser):
|
|
parser and type checker code: this is the first step of compilation
|
|
- [`codon/cir/`](https://github.com/exaloop/codon/tree/develop/codon/cir):
|
|
Codon IR and optimizations: the second step of compilation
|
|
- [`codon/cir/llvm/`](https://github.com/exaloop/codon/tree/develop/codon/cir/llvm):
|
|
conversion from Codon IR to LLVM IR and machine code: the last step of compilation
|
|
- [`codon/runtime/`](https://github.com/exaloop/codon/tree/develop/codon/runtime):
|
|
runtime library: used during execution
|
|
- [`stdlib/`](https://github.com/exaloop/codon/tree/develop/stdlib): standard library code
|
|
|
|
You can also take a look at some of the [open issues](https://github.com/exaloop/codon/issues). If you
|
|
have any question or suggestions, please feel free to ask in [the forum](https://github.com/exaloop/codon/discussions).
|
|
|
|
## Is there a Contributor License Agreement (CLA)?
|
|
|
|
Yes, there is a CLA that is required to be agreed to before any pull requests are merged.
|
|
Please see [exaloop.io/cla.txt](https://exaloop.io/cla.txt) for more information. To agree to
|
|
the CLA, send an email with your GitHub username to [info@exaloop.io](mailto:info@exaloop.io).
|