mirror of https://github.com/exaloop/codon.git
Update docs (#28)
* Update docs * Update docs * Update docs * GitBook: [#4] Add hint * Update primer * Re-organize docs * Fix table * Fix link * GitBook: [#5] No subject * GitBook: [#6] No subject * Cleanup and doc fix * Add IR docs * Add ir docs * Fix spelling error * More IR docs * Update README.md * Update README.md * Fix warning * Update intro * Update README.md * Update docs * Fix table * Don't build docs * Update docs * Add Jupyter docs * FIx snippet * Update README.md * Fix images * Fix code block * Update docs, update cmake * Break up tutorial * Update pipeline.svg * Update docs for new version * Add differences with Python docspull/40/head
A. R. Shajii
GitHub
parent
963ddb3b60
commit
d5ce1f8ff9
|
|
@ -6,8 +6,6 @@ project(
|
|||
DESCRIPTION "high-performance, extensible Python compiler")
|
||||
configure_file("${PROJECT_SOURCE_DIR}/cmake/config.h.in"
|
||||
"${PROJECT_SOURCE_DIR}/codon/config/config.h")
|
||||
configure_file("${PROJECT_SOURCE_DIR}/cmake/config.py.in"
|
||||
"${PROJECT_SOURCE_DIR}/docs/sphinx/config/config.py")
|
||||
|
||||
option(CODON_JUPYTER "build Codon Jupyter server" OFF)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
<p align="center">
|
||||
<img src="docs/sphinx/codon.png?raw=true" width="600" alt="Codon"/>
|
||||
<img src="docs/img/codon.png?raw=true" width="600" alt="Codon"/>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
|
|
@ -95,13 +95,13 @@ Pre-built binaries for Linux and macOS on x86_64 are available alongside [each r
|
|||
We also have a script for downloading and installing pre-built versions:
|
||||
|
||||
```bash
|
||||
/bin/bash -c "$(curl -fsSL https://codon.dev/install.sh)"
|
||||
/bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)"
|
||||
```
|
||||
|
||||
### Build from source
|
||||
|
||||
See [Building from Source](docs/sphinx/build.rst).
|
||||
See [Building from Source](docs/advanced/build.md).
|
||||
|
||||
## Documentation
|
||||
|
||||
Please check [docs.codon.dev](https://docs.codon.dev) for in-depth documentation.
|
||||
Please see [docs.exaloop.io](https://docs.exaloop.io/codon) for in-depth documentation.
|
||||
|
|
|
|||
|
|
@ -269,7 +269,6 @@ public:
|
|||
/// Gets or realizes a method.
|
||||
/// @param parent the parent class
|
||||
/// @param methodName the method name
|
||||
/// @param rType the return type
|
||||
/// @param args the argument types
|
||||
/// @param generics the generics
|
||||
/// @return the method or nullptr
|
||||
|
|
@ -279,7 +278,6 @@ public:
|
|||
|
||||
/// Gets or realizes a function.
|
||||
/// @param funcName the function name
|
||||
/// @param rType the return type
|
||||
/// @param args the argument types
|
||||
/// @param generics the generics
|
||||
/// @param module the module of the function
|
||||
|
|
|
|||
|
|
@ -0,0 +1,36 @@
|
|||
|
||||
|
||||
Codon is a high-performance Python compiler that compiles Python code to
|
||||
native machine code without any runtime overhead. Typical speedups over
|
||||
Python are on the order of 100x or more, on a single thread. Codon
|
||||
supports native multithreading which can lead to speedups many times higher still.
|
||||
|
||||
The Codon framework is fully modular and extensible, allowing for the
|
||||
seamless integration of new modules, compiler optimizations, domain-specific
|
||||
languages and so on. We actively develop Codon extensions for a number of
|
||||
domains such as bioinformatics and quantitative finance.
|
||||
|
||||
# Codon at a glance
|
||||
|
||||
A simple Python program `fib.py`...
|
||||
|
||||
``` python
|
||||
from time import time
|
||||
|
||||
def fib(n):
|
||||
return n if n < 2 else fib(n - 1) + fib(n - 2)
|
||||
|
||||
t0 = time()
|
||||
ans = fib(40)
|
||||
t1 = time()
|
||||
print(f'Computed fib(40) = {ans} in {t1 - t0} seconds.')
|
||||
```
|
||||
|
||||
... run through Python and Codon:
|
||||
|
||||
```
|
||||
$ python3 fib.py
|
||||
Computed fib(40) = 102334155 in 17.979357957839966 seconds.
|
||||
$ codon run -release fib.py
|
||||
Computed fib(40) = 102334155 in 0.275645 seconds.
|
||||
```
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# Table of contents
|
||||
|
||||
* [Welcome to Codon](README.md)
|
||||
|
||||
## Basics
|
||||
|
||||
* [Getting started](intro/intro.md)
|
||||
* [Frequently asked questions](intro/faq.md)
|
||||
* [Differences with Python](intro/differences.md)
|
||||
|
||||
## Language
|
||||
|
||||
* [Basics](language/basics.md)
|
||||
* [Collections](language/collections.md)
|
||||
* [Functions](language/functions.md)
|
||||
* [Classes](language/classes.md)
|
||||
* [Generators](language/generators.md)
|
||||
* [Statics](language/statics.md)
|
||||
* [Other types and features](language/extra.md)
|
||||
* [Foreign function interface](language/ffi.md)
|
||||
* [Inline LLVM IR](language/llvm.md)
|
||||
|
||||
## Interoperability
|
||||
|
||||
* [Python integration](interop/python.md)
|
||||
* [C/C++ integration](interop/cpp.md)
|
||||
* [Jupyter integration](interop/jupyter.md)
|
||||
|
||||
## Advanced
|
||||
|
||||
* [Parallelism and multithreading](advanced/parallel.md)
|
||||
* [Pipelines](advanced/pipelines.md)
|
||||
* [Intermediate representation](advanced/ir.md)
|
||||
* [Building from source](advanced/build.md)
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
Unless you really need to build Codon for whatever reason, we strongly
|
||||
recommend using pre-built binaries if possible.
|
||||
|
||||
# Dependencies
|
||||
|
||||
Codon depends on LLVM 12, which can be installed via most package
|
||||
managers. To build LLVM 12 yourself, you can do the following:
|
||||
|
||||
``` bash
|
||||
git clone --depth 1 -b release/12.x https://github.com/llvm/llvm-project
|
||||
mkdir -p llvm-project/llvm/build
|
||||
cd llvm-project/llvm/build
|
||||
cmake .. \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DLLVM_INCLUDE_TESTS=OFF \
|
||||
-DLLVM_ENABLE_RTTI=ON \
|
||||
-DLLVM_ENABLE_ZLIB=OFF \
|
||||
-DLLVM_ENABLE_TERMINFO=OFF \
|
||||
-DLLVM_TARGETS_TO_BUILD=host
|
||||
make
|
||||
make install
|
||||
```
|
||||
|
||||
# Build
|
||||
|
||||
The following can generally be used to build Codon. The build process
|
||||
will automatically download and build several smaller dependencies.
|
||||
|
||||
``` bash
|
||||
mkdir build
|
||||
(cd build && cmake .. -DCMAKE_BUILD_TYPE=Release \
|
||||
-DLLVM_DIR=$(llvm-config --cmakedir) \
|
||||
-DCMAKE_C_COMPILER=clang \
|
||||
-DCMAKE_CXX_COMPILER=clang++)
|
||||
cmake --build build --config Release
|
||||
```
|
||||
|
||||
This should produce the `codon` executable in the `build` directory, as
|
||||
well as `codon_test` which runs the test suite.
|
||||
|
|
@ -0,0 +1,315 @@
|
|||
After type checking but before native code generation, the Codon compiler
|
||||
makes use of a new [intermediate representation](https://en.wikipedia.org/wiki/Intermediate_representation)
|
||||
called SIR, where a number of higher-level optimizations, transformations and analyses take place.
|
||||
SIR offers a comprehensive framework for writing new optimizations or
|
||||
analyses without having to deal with cumbersome abstract syntax trees (ASTs).
|
||||
In this section we'll give an overview of SIR, discuss the types of things
|
||||
you might want to use it for, and give a few examples.
|
||||
|
||||
# At a glance
|
||||
|
||||
Here is a small (simplified) example showcasing SIR in action. Consider the code:
|
||||
|
||||
``` python
|
||||
def fib(n):
|
||||
if n < 2:
|
||||
return 1
|
||||
else:
|
||||
return fib(n - 1) + fib(n - 2)
|
||||
```
|
||||
|
||||
When instantiated with an `int` argument, the following IR gets produced (the
|
||||
names have been cleaned up for simplicity):
|
||||
|
||||
``` lisp
|
||||
(bodied_func
|
||||
'"fib[int]"
|
||||
(type '"fib[int]")
|
||||
(args (var '"n" (type '"int") (global false)))
|
||||
(vars)
|
||||
(series
|
||||
(if (call '"int.__lt__[int,int]" '"n" 2)
|
||||
(series (return 1))
|
||||
(series
|
||||
(return
|
||||
(call
|
||||
'"int.__add__[int,int]"
|
||||
(call
|
||||
'"fib[int]"
|
||||
(call '"int.__sub__[int,int]" '"n" 1))
|
||||
(call
|
||||
'"fib[int]"
|
||||
(call '"int.__sub__[int,int]" '"n" 2))))))))
|
||||
```
|
||||
|
||||
A few interesting points to consider:
|
||||
|
||||
- SIR is hierarchical like ASTS, but unlike ASTs it uses a vastly reduced
|
||||
set of nodes, making it much easier to work with and reason about.
|
||||
- Operators are expressed as function calls. In fact, SIR has no explicit
|
||||
concept of `+`, `-`, etc. and instead expresses these via their corresponding
|
||||
magic methods (`__add__`, `__sub__`, etc.).
|
||||
- SIR has no concept of generic types. By the time SIR is generated, all types
|
||||
need to have been resolved.
|
||||
|
||||
# Structure
|
||||
|
||||
SIR is comprised of a set of *nodes*, each with a specific semantic meaning.
|
||||
There are nodes for representing constants (e.g. `42`), instructions (e.g. `call`)
|
||||
control flow (e.g. `if`), types (e.g. `int`) and so on.
|
||||
|
||||
Here is a table showing the different types of nodes, LLVM IR equivalents,
|
||||
and some examples:
|
||||
|
||||
| Node | LLVM equivalent | Examples |
|
||||
|----------|-----------------|-------------------------------------------|
|
||||
| `Node` | n/a | all of the below |
|
||||
| `Module` | `Module` | n/a |
|
||||
| `Type` | `Type` | `IntType`, `FuncType`, `RefType` |
|
||||
| `Var` | `AllocaInst` | `Var`, `Func` |
|
||||
| `Func` | `Function` | `BodiedFunc`, `ExternalFunc`, `LLVMFunc` |
|
||||
| `Value` | `Value` | all of the below |
|
||||
| `Const` | `Constant` | `IntConst`, `FloatConst`, `StringConst` |
|
||||
| `Instr` | `Instruction` | `CallInstr`, `TernaryInstr`, `ThrowInstr` |
|
||||
| `Flow` | n/a | `IfFlow`, `WhileFlow`, `ForFlow` |
|
||||
|
||||
# Uses
|
||||
|
||||
SIR provides a framework for doing program optimizations, analyses and transformations.
|
||||
These operations are collectively known as IR *passes*.
|
||||
|
||||
A number of built-in passes and other functionalities are provided by SIR. These can be
|
||||
used as building blocks to create new passes. Examples include:
|
||||
|
||||
- Control-flow graph creation
|
||||
- Reaching definitions
|
||||
- Dominator analysis
|
||||
- Side effect analysis
|
||||
- Constant propagation and folding
|
||||
- Canonicalization
|
||||
- Inlining and outlining
|
||||
- Python-specific optimizations targeting several common Python idioms
|
||||
|
||||
We're regularly adding new standard passes, so this list is always growing.
|
||||
|
||||
## An example
|
||||
|
||||
Let's look at a real example. Imagine we want to write a pass that transforms expressions
|
||||
of the form `<int const> + <int const>` into a single `<int const>` denoting the result.
|
||||
In other words, a simple form of constant folding that only looks at addition on integers.
|
||||
The resulting pass would like this:
|
||||
|
||||
``` cpp
|
||||
#include "codon/sir/transform/pass.h"
|
||||
|
||||
using namespace codon::ir;
|
||||
|
||||
class MyAddFolder : public transform::OperatorPass {
|
||||
public:
|
||||
static const std::string KEY;
|
||||
std::string getKey() const override { return KEY; }
|
||||
|
||||
void handle(CallInstr *v) override {
|
||||
auto *f = util::getFunc(v->getCallee());
|
||||
if (!f || f->getUnmangledName() != "__add__" || v->numArgs() != 2)
|
||||
return;
|
||||
|
||||
auto *lhs = cast<IntConst>(v->front());
|
||||
auto *rhs = cast<IntConst>(v->back());
|
||||
|
||||
if (lhs && rhs) {
|
||||
auto sum = lhs->getVal() + rhs->getVal();
|
||||
v->replaceAll(v->getModule()->getInt(sum));
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const std::string MyAddFolder::KEY = "my-add-folder";
|
||||
```
|
||||
|
||||
So how does this actually work, and what do the different components mean? Here
|
||||
are some notable points:
|
||||
|
||||
- Most passes can inherit from `transform::OperatorPass`. `OperatorPass` is a combination
|
||||
of an `Operator` and a `Pass`. An `Operator` is a utility visitor that provides hooks for
|
||||
handling all the different node types (i.e. through the `handle()` methods). `Pass` is the
|
||||
base class representing a generic pass, which simply provides a `run()` method that takes
|
||||
a module.
|
||||
- Because of this, `MyAddFolder::handle(CallInstr *)` will be called on every call instruction
|
||||
in the module.
|
||||
- Within our `handle()`, we first check to see if the function being called is `__add__`, indicating
|
||||
addition (in practice there would be a more specific check to make sure this is *the* `__add__`),
|
||||
and if so we extract the first and second arguments.
|
||||
- We cast these arguments to `IntConst`. If the results are non-null, then both arguments were in fact
|
||||
integer constants, meaning we can replace the original call instruction with a new constant that
|
||||
represents the result of the addition. In SIR, all nodes are "replaceable" via a `replaceAll()` method.
|
||||
- Lastly, notice that all passes have a `KEY` field to uniquely identify them.
|
||||
|
||||
## Bidirectionality
|
||||
|
||||
An important and often very useful feature of SIR is that it is *bidirectional*, meaning it's possible
|
||||
to return to the type checking stage to generate new IR nodes that were not initially present in the
|
||||
module. For example, imagine that your pass needs to use a `List` with some new element type; that list's
|
||||
methods need to be instantiated by the type checker for use in SIR. In practice this bidirectionality
|
||||
often lets you write large parts of your optimization or transformation in Codon, and pull out the necessary
|
||||
functions or types as needed in the pass.
|
||||
|
||||
SIR's `Module` class has three methods to enable this feature:
|
||||
|
||||
``` cpp
|
||||
/// Gets or realizes a function.
|
||||
/// @param funcName the function name
|
||||
/// @param args the argument types
|
||||
/// @param generics the generics
|
||||
/// @param module the module of the function
|
||||
/// @return the function or nullptr
|
||||
Func *getOrRealizeFunc(const std::string &funcName, std::vector<types::Type *> args,
|
||||
std::vector<types::Generic> generics = {},
|
||||
const std::string &module = "");
|
||||
|
||||
/// Gets or realizes a method.
|
||||
/// @param parent the parent class
|
||||
/// @param methodName the method name
|
||||
/// @param rType the return type
|
||||
/// @param args the argument types
|
||||
/// @param generics the generics
|
||||
/// @return the method or nullptr
|
||||
Func *getOrRealizeMethod(types::Type *parent, const std::string &methodName,
|
||||
std::vector<types::Type *> args,
|
||||
std::vector<types::Generic> generics = {});
|
||||
|
||||
/// Gets or realizes a type.
|
||||
/// @param typeName the type name
|
||||
/// @param generics the generics
|
||||
/// @param module the module of the type
|
||||
/// @return the function or nullptr
|
||||
types::Type *getOrRealizeType(const std::string &typeName,
|
||||
std::vector<types::Generic> generics = {},
|
||||
const std::string &module = "");
|
||||
```
|
||||
|
||||
Let's see bidirectionality in action. Consider the following Codon code:
|
||||
|
||||
``` python
|
||||
def foo(x):
|
||||
return x*3 + x
|
||||
|
||||
def validate(x, y):
|
||||
assert y == x*4
|
||||
|
||||
a = foo(10)
|
||||
b = foo(1.5)
|
||||
c = foo('a')
|
||||
```
|
||||
|
||||
Assume we want our pass to insert a call to `validate()` after each assignment that takes the assigned variable
|
||||
and the argument passed to `foo()`. We would do something like the following:
|
||||
|
||||
``` cpp
|
||||
#include "codon/sir/transform/pass.h"
|
||||
|
||||
using namespace codon::ir;
|
||||
|
||||
class ValidateFoo : public transform::OperatorPass {
|
||||
public:
|
||||
static const std::string KEY;
|
||||
std::string getKey() const override { return KEY; }
|
||||
|
||||
void handle(AssignInstr *v) {
|
||||
auto *M = v->getModule();
|
||||
auto *var = v->getLhs();
|
||||
auto *call = cast<CallInstr>(v->getRhs());
|
||||
if (!call)
|
||||
return;
|
||||
|
||||
auto *foo = util::getFunc(call->getCallee());
|
||||
if (!foo || foo->getUnmangledName() != "foo")
|
||||
return;
|
||||
|
||||
auto *arg1 = call->front(); // argument of 'foo' call
|
||||
auto *arg2 = M->Nr<VarValue>(var); // result of 'foo' call
|
||||
auto *validate =
|
||||
M->getOrRealizeFunc("validate", {arg1->getType(), arg2->getType()});
|
||||
auto *validateCall = util::call(validate, {arg1, arg2});
|
||||
|
||||
insertAfter(validateCall); // call 'validate' after 'foo'
|
||||
}
|
||||
};
|
||||
|
||||
const std::string ValidateFoo::KEY = "validate-foo";
|
||||
```
|
||||
|
||||
Note that `insertAfter` is a conveience method of `Operator` that inserts the given node "after" the node
|
||||
being visited (along with `insertBefore` which inserts *before* the node being visited).
|
||||
|
||||
Running this pass on the snippet above, we would get:
|
||||
|
||||
``` python
|
||||
a = foo(10)
|
||||
validate(10, a)
|
||||
|
||||
b = foo(1.5)
|
||||
validate(1.5, b)
|
||||
|
||||
c = foo('a')
|
||||
validate('a', c)
|
||||
```
|
||||
|
||||
Notice that we used `getOrRealizeFunc` to create three different instances of `validate`: one for `int`
|
||||
arguments, one for `float` arguments and finally one for `str` arguments.
|
||||
|
||||
# Extending the IR
|
||||
|
||||
SIR is extensible, and it is possible to add new constants, instructions, flows and types. This can be
|
||||
done by subclassing the corresponding *custom* base class; to create a custom type, for example, you
|
||||
would subclass `CustomType`. Let's look at an example where we extend SIR to add a 32-bit float type:
|
||||
|
||||
``` cpp
|
||||
using namespace codon::ir;
|
||||
|
||||
#include "codon/sir/dsl/nodes.h"
|
||||
#include "codon/sir/llvm/llvisitor.h"
|
||||
|
||||
class Builder : public dsl::codegen::TypeBuilder {
|
||||
public:
|
||||
llvm::Type *buildType(LLVMVisitor *v) override {
|
||||
return v->getBuilder()->getFloatTy();
|
||||
}
|
||||
|
||||
llvm::DIType *buildDebugType(LLVMVisitor *v) override {
|
||||
auto *module = v->getModule();
|
||||
auto &layout = module->getDataLayout();
|
||||
auto &db = v->getDebugInfo();
|
||||
auto *t = buildType(v);
|
||||
return db.builder->createBasicType(
|
||||
"float_32",
|
||||
layout.getTypeAllocSizeInBits(t),
|
||||
llvm::dwarf::DW_ATE_float);
|
||||
}
|
||||
};
|
||||
|
||||
class Float32 : public dsl::CustomType {
|
||||
public:
|
||||
std::unique_ptr<TypeBuilder> getBuilder() const override {
|
||||
return std::make_unique<Builder>();
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
Notice that, in order to specify how to generate code for our `Float32` type, we create a `TypeBuilder`
|
||||
subclass with methods for building the corresponding LLVM IR type. There is also a `ValueBuilder` for
|
||||
new constants and converting them to LLVM IR, as well as a `CFBuilder` for new instructions and creating
|
||||
control-flow graphs out of them.
|
||||
|
||||
{% hint style="info" %}
|
||||
When subclassing nodes other than types (e.g. instructions, flows, etc.), be sure to use the `AcceptorExtend`
|
||||
[CRTP](https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern) class, as in
|
||||
`class MyNewInstr : public AcceptorExtend<MyNewInstr, dsl::CustomInstr>`.
|
||||
{% endhint %}
|
||||
|
||||
# Utilities
|
||||
|
||||
The `codon/ir/util/` directory has a number of utility and generally helpful functions, for things like
|
||||
cloning IR, inlining/outlining, matching and more. `codon/ir/util/irtools.h` in particular has many helpful
|
||||
functions for performing various common tasks. If you're working with SIR, be sure to take a look at these
|
||||
functions to make your life easier!
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
Codon supports parallelism and multithreading via OpenMP out of the box.
|
||||
Here\'s an example:
|
||||
|
||||
``` python
|
||||
@par
|
||||
for i in range(10):
|
||||
import threading as thr
|
||||
print('hello from thread', thr.get_ident())
|
||||
```
|
||||
|
||||
By default, parallel loops will use all available threads, or use the
|
||||
number of threads specified by the `OMP_NUM_THREADS` environment
|
||||
variable. A specific thread number can be given directly on the `@par`
|
||||
line as well:
|
||||
|
||||
``` python
|
||||
@par(num_threads=5)
|
||||
for i in range(10):
|
||||
import threading as thr
|
||||
print('hello from thread', thr.get_ident())
|
||||
```
|
||||
|
||||
`@par` supports several OpenMP parameters, including:
|
||||
|
||||
- `num_threads` (int): the number of threads to use when running the
|
||||
loop
|
||||
- `schedule` (str): either *static*, *dynamic*, *guided*, *auto* or
|
||||
*runtime*
|
||||
- `chunk_size` (int): chunk size when partitioning loop iterations
|
||||
- `ordered` (bool): whether the loop iterations should be executed in
|
||||
the same order
|
||||
|
||||
Other OpenMP parameters like `private`, `shared` or `reduction`, are
|
||||
inferred automatically by the compiler. For example, the following loop
|
||||
|
||||
``` python
|
||||
a = 0
|
||||
@par
|
||||
for i in range(N):
|
||||
a += foo(i)
|
||||
```
|
||||
|
||||
will automatically generate a reduction for variable `a`.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Modifying shared objects like lists or dictionaries within a parallel
|
||||
section needs to be done with a lock or critical section. See below
|
||||
for more details.
|
||||
{% endhint %}
|
||||
|
||||
Here is an example that finds the sum of prime numbers up to a
|
||||
user-defined limit, using a parallel loop on 16 threads with a dynamic
|
||||
schedule and chunk size of 100:
|
||||
|
||||
``` python
|
||||
from sys import argv
|
||||
|
||||
def is_prime(n):
|
||||
factors = 0
|
||||
for i in range(2, n):
|
||||
if n % i == 0:
|
||||
factors += 1
|
||||
return factors == 0
|
||||
|
||||
limit = int(argv[1])
|
||||
total = 0
|
||||
|
||||
@par(schedule='dynamic', chunk_size=100, num_threads=16)
|
||||
for i in range(2, limit):
|
||||
if is_prime(i):
|
||||
total += 1
|
||||
|
||||
print(total)
|
||||
```
|
||||
|
||||
Static schedules work best when each loop iteration takes roughly the
|
||||
same amount of time, whereas dynamic schedules are superior when each
|
||||
iteration varies in duration. Since counting the factors of an integer
|
||||
takes more time for larger integers, we use a dynamic schedule here.
|
||||
|
||||
`@par` also supports C/C++ OpenMP pragma strings. For example, the
|
||||
`@par` line in the above example can also be written as:
|
||||
|
||||
``` python
|
||||
# same as: @par(schedule='dynamic', chunk_size=100, num_threads=16)
|
||||
@par('schedule(dynamic, 100) num_threads(16)')
|
||||
```
|
||||
|
||||
# Different kinds of loops
|
||||
|
||||
`for`-loops can iterate over arbitrary generators, but OpenMP\'s
|
||||
parallel loop construct only applies to *imperative* for-loops of the
|
||||
form `for i in range(a, b, c)` (where `c` is constant). For general
|
||||
parallel for-loops of the form `for i in some_generator()`, a task-based
|
||||
approach is used instead, where each loop iteration is executed as an
|
||||
independent task.
|
||||
|
||||
The Codon compiler also converts iterations over lists
|
||||
(`for a in some_list`) to imperative for-loops, meaning these loops can
|
||||
be executed using OpenMP\'s loop parallelism.
|
||||
|
||||
# Custom reductions
|
||||
|
||||
Codon can automatically generate efficient reductions for `int` and
|
||||
`float` values. For other data types, user-defined reductions can be
|
||||
specified. A class that supports reductions must include:
|
||||
|
||||
- A default constructor that represents the *zero value*
|
||||
- An `__add__` method (assuming `+` is used as the reduction operator)
|
||||
|
||||
Here is an example for reducing a new `Vector` type:
|
||||
|
||||
``` python
|
||||
@tuple
|
||||
class Vector:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __new__():
|
||||
return Vector(0, 0)
|
||||
|
||||
def __add__(self, other: Vector):
|
||||
return Vector(self.x + other.x, self.y + other.y)
|
||||
|
||||
v = Vector()
|
||||
@par
|
||||
for i in range(100):
|
||||
v += Vector(i,i)
|
||||
print(v) # (x: 4950, y: 4950)
|
||||
```
|
||||
|
||||
# OpenMP constructs
|
||||
|
||||
All of OpenMP\'s API functions are accessible directly in Codon. For
|
||||
example:
|
||||
|
||||
``` python
|
||||
import openmp as omp
|
||||
print(omp.get_num_threads())
|
||||
omp.set_num_threads(32)
|
||||
```
|
||||
|
||||
OpenMP\'s *critical*, *master*, *single* and *ordered* constructs can be
|
||||
applied via the corresponding decorators:
|
||||
|
||||
``` python
|
||||
import openmp as omp
|
||||
|
||||
@omp.critical
|
||||
def only_run_by_one_thread_at_a_time():
|
||||
print('critical!', omp.get_thread_num())
|
||||
|
||||
@omp.master
|
||||
def only_run_by_master_thread():
|
||||
print('master!', omp.get_thread_num())
|
||||
|
||||
@omp.single
|
||||
def only_run_by_single_thread():
|
||||
print('single!', omp.get_thread_num())
|
||||
|
||||
@omp.ordered
|
||||
def run_ordered_by_iteration(i):
|
||||
print('ordered!', i)
|
||||
|
||||
@par(ordered=True)
|
||||
for i in range(100):
|
||||
only_run_by_one_thread_at_a_time()
|
||||
only_run_by_master_thread()
|
||||
only_run_by_single_thread()
|
||||
run_ordered_by_iteration(i)
|
||||
```
|
||||
|
||||
For finer-grained locking, consider using the locks from the `threading`
|
||||
module:
|
||||
|
||||
``` python
|
||||
from threading import Lock
|
||||
lock = Lock() # or RLock for re-entrant lock
|
||||
|
||||
@par
|
||||
for i in range(100):
|
||||
with lock:
|
||||
print('only one thread at a time allowed here')
|
||||
```
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
Codon extends the core Python language with a pipe operator. You can
|
||||
chain multiple functions and generators to form a pipeline. Pipeline
|
||||
stages can be regular functions or generators. In the case of standard
|
||||
functions, the function is simply applied to the input data and the
|
||||
result is carried to the remainder of the pipeline, akin to F#\'s
|
||||
functional piping. If, on the other hand, a stage is a generator, the
|
||||
values yielded by the generator are passed lazily to the remainder of
|
||||
the pipeline, which in many ways mirrors how piping is implemented in
|
||||
Bash. Note that Codon ensures that generator pipelines do not collect
|
||||
any data unless explicitly requested, thus allowing the processing of
|
||||
terabytes of data in a streaming fashion with no memory and minimal CPU
|
||||
overhead.
|
||||
|
||||
``` python
|
||||
def add1(x):
|
||||
return x + 1
|
||||
|
||||
2 |> add1 # 3; equivalent to add1(2)
|
||||
|
||||
def calc(x, y):
|
||||
return x + y**2
|
||||
2 |> calc(3) # 11; equivalent to calc(2, 3)
|
||||
2 |> calc(..., 3) # 11; equivalent to calc(2, 3)
|
||||
2 |> calc(3, ...) # 7; equivalent to calc(3, 2)
|
||||
|
||||
def gen(i):
|
||||
for i in range(i):
|
||||
yield i
|
||||
|
||||
5 |> gen |> print # prints 0 1 2 3 4 separated by newline
|
||||
range(1, 4) |> iter |> gen |> print(end=' ') # prints 0 0 1 0 1 2 without newline
|
||||
[1, 2, 3] |> print # prints [1, 2, 3]
|
||||
range(100000000) |> print # prints range(0, 100000000)
|
||||
range(100000000) |> iter |> print # not only prints all those numbers, but it uses almost no memory at all
|
||||
```
|
||||
|
||||
Codon will chain anything that implements `__iter__`, and the compiler
|
||||
will optimize out generators whenever possible. Combinations of pipes
|
||||
and generators can be used to implement efficient streaming pipelines.
|
||||
|
||||
{% hint style="warning" %}
|
||||
The Codon compiler may perform optimizations that change the order of
|
||||
elements passed through a pipeline. Therefore, it is best to not rely on
|
||||
order when using pipelines. If order needs to be maintained, consider
|
||||
using a regular loop or passing an index alongside each element sent
|
||||
through the pipeline.
|
||||
{% endhint %}
|
||||
|
||||
# Parallel pipelines
|
||||
|
||||
CPython and many other implementations alike cannot take advantage of
|
||||
parallelism due to the infamous global interpreter lock, a mutex that
|
||||
prevents multiple threads from executing Python bytecode at once. Unlike
|
||||
CPython, Codon has no such restriction and supports full multithreading.
|
||||
To this end, Codon supports a *parallel* pipe operator `||>`, which is
|
||||
semantically similar to the standard pipe operator except that it allows
|
||||
the elements sent through it to be processed in parallel by the
|
||||
remainder of the pipeline. Hence, turning a serial program into a
|
||||
parallel one often requires the addition of just a single character in
|
||||
Codon. Further, a single pipeline can contain multiple parallel pipes,
|
||||
resulting in nested parallelism.
|
||||
|
||||
``` python
|
||||
range(100000) |> iter ||> print # prints all these numbers, probably in random order
|
||||
range(100000) |> iter ||> process ||> clean # runs process in parallel, and then cleans data in parallel
|
||||
```
|
||||
|
||||
Codon will automatically schedule the `process` and `clean` functions to
|
||||
execute as soon as possible. You can control the number of threads via
|
||||
the `OMP_NUM_THREADS` environment variable.
|
||||
|
||||
Internally, the Codon compiler uses an OpenMP task backend to generate
|
||||
code for parallel pipelines. Logically, parallel pipe operators are
|
||||
similar to parallel-for loops: the portion of the pipeline after the
|
||||
parallel pipe is outlined into a new function that is called by the
|
||||
OpenMP runtime task spawning routines (as in `#pragma omp task` in C++),
|
||||
and a synchronization point (`#pragma omp taskwait`) is added after the
|
||||
outlined segment.
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
PROJECT_NAME = "Codon"
|
||||
XML_OUTPUT = xml
|
||||
INPUT = ../../codon
|
||||
EXCLUDE_PATTERNS = */codon/util/fmt/* */codon/sir/llvm/coro/*
|
||||
STRIP_FROM_PATH = ../..
|
||||
GENERATE_LATEX = NO
|
||||
GENERATE_MAN = NO
|
||||
GENERATE_RTF = NO
|
||||
CASE_SENSE_NAMES = NO
|
||||
GENERATE_XML = YES
|
||||
GENERATE_HTML = YES
|
||||
RECURSIVE = YES
|
||||
QUIET = YES
|
||||
JAVADOC_AUTOBRIEF = YES
|
||||
WARN_IF_UNDOCUMENTED = NO
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
*
|
||||
!.gitignore
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
*
|
||||
!.gitignore
|
||||
|
Before Width: | Height: | Size: 120 KiB After Width: | Height: | Size: 120 KiB |
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 442 KiB |
|
|
@ -0,0 +1,55 @@
|
|||
Calling C/C++ from Codon is quite easy with `from C import`, but Codon
|
||||
can also be called from C/C++ code. To make a Codon function externally
|
||||
visible, simply annotate it with `@export`:
|
||||
|
||||
``` python
|
||||
@export
|
||||
def foo(n: int):
|
||||
for i in range(n):
|
||||
print(i * i)
|
||||
return n * n
|
||||
```
|
||||
|
||||
Note that only top-level, non-generic functions can be exported. Now we
|
||||
can create a shared library containing `foo` (assuming source file
|
||||
*foo.codon*):
|
||||
|
||||
``` bash
|
||||
codon build -o libfoo.so foo.codon
|
||||
```
|
||||
|
||||
Now we can call `foo` from a C program:
|
||||
|
||||
``` c
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
|
||||
int64_t foo(int64_t);
|
||||
|
||||
int main() {
|
||||
printf("%llu\n", foo(10));
|
||||
}
|
||||
```
|
||||
|
||||
Compile:
|
||||
|
||||
``` bash
|
||||
gcc -o foo -L. -lfoo foo.c
|
||||
```
|
||||
|
||||
Now running `./foo` should invoke `foo()` as defined in Codon, with an
|
||||
argument of `10`.
|
||||
|
||||
# Converting types
|
||||
|
||||
The following table shows the conversions between Codon and C/C++ types:
|
||||
|
||||
| Codon | C/C++ |
|
||||
|-----------|--------------------------------------|
|
||||
| `int` | `long` or `int64_t` |
|
||||
| `float` | `double` |
|
||||
| `bool` | `bool` |
|
||||
| `byte` | `char` or `int8_t` |
|
||||
| `str` | `{int64_t, char*}` (length and data) |
|
||||
| `class` | Pointer to corresponding tuple |
|
||||
| `@tuple` | Struct of fields |
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
Codon ships with a kernel that can be used by Jupyter, invoked
|
||||
with the `codon jupyter ...` subcommand.
|
||||
|
||||
To add the Codon kernel, add the following `kernel.json` file to
|
||||
the directory `/path/to/jupyter/kernels/codon/`:
|
||||
|
||||
``` json
|
||||
{
|
||||
"display_name": "Codon",
|
||||
"argv": [
|
||||
"/path/to/codon",
|
||||
"jupyter",
|
||||
"{connection_file}"
|
||||
],
|
||||
"language": "python"
|
||||
}
|
||||
```
|
||||
|
||||
Plugins can also optionally be specified, as in:
|
||||
|
||||
``` json
|
||||
{
|
||||
"display_name": "Codon",
|
||||
"argv": [
|
||||
"/path/to/codon",
|
||||
"jupyter",
|
||||
"-plugin", "/path/to/plugin",
|
||||
"{connection_file}"
|
||||
],
|
||||
"language": "python"
|
||||
}
|
||||
```
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
Calling Python from Codon is possible in two ways:
|
||||
|
||||
- `from python import` allows importing and calling Python functions
|
||||
from existing Python modules.
|
||||
- `@python` allows writing Python code directly in Codon.
|
||||
|
||||
In order to use these features, the `CODON_PYTHON` environment variable
|
||||
must be set to the appropriate Python shared library:
|
||||
|
||||
``` bash
|
||||
export CODON_PYTHON=/path/to/libpython.X.Y.so
|
||||
```
|
||||
|
||||
For example, with a `brew`-installed Python 3.9 on macOS, this might be
|
||||
|
||||
``` bash
|
||||
/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib
|
||||
```
|
||||
|
||||
Note that only Python versions 3.6 and later are supported.
|
||||
|
||||
# `from python import`
|
||||
|
||||
Let\'s say we have a Python function defined in *mymodule.py*:
|
||||
|
||||
``` python
|
||||
def multiply(a, b):
|
||||
return a * b
|
||||
```
|
||||
|
||||
We can call this function in Codon using `from python import` and
|
||||
indicating the appropriate call and return types:
|
||||
|
||||
``` python
|
||||
from python import mymodule.multiply(int, int) -> int
|
||||
print(multiply(3, 4)) # 12
|
||||
```
|
||||
|
||||
(Be sure the `PYTHONPATH` environment variable includes the path of
|
||||
*mymodule.py*!)
|
||||
|
||||
# `@python`
|
||||
|
||||
Codon programs can contain functions that will be executed by Python via
|
||||
`pydef`:
|
||||
|
||||
``` python
|
||||
@python
|
||||
def multiply(a: int, b: int) -> int:
|
||||
return a * b
|
||||
|
||||
print(multiply(3, 4)) # 12
|
||||
```
|
||||
|
||||
This makes calling Python modules like NumPy very easy:
|
||||
|
||||
``` python
|
||||
@python
|
||||
def myrange(n: int) -> List[int]:
|
||||
from numpy import arange
|
||||
return list(arange(n))
|
||||
|
||||
print(myrange(5)) # [0, 1, 2, 3, 4]
|
||||
```
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
While Codon's syntax and semantics are virtually identical
|
||||
to Python's, there are some notable differences that are
|
||||
worth mentioning. Most of these design decisions were made
|
||||
with the trade-off between performance and Python compatibility
|
||||
in mind.
|
||||
|
||||
# Data types
|
||||
|
||||
- **Integers:** Codon's `int` is a 64-bit signed integer,
|
||||
whereas Python's (after version 3) can be arbitrarily large.
|
||||
However Codon does support larger integers via `Int[N]` where
|
||||
`N` is the bit width.
|
||||
|
||||
- **Strings:** Codon currently uses ASCII strings unlike
|
||||
Python's unicode strings.
|
||||
|
||||
- **Dictionaries:** Codon's dictionary type is not sorted
|
||||
internally, unlike Python's.
|
||||
|
||||
# Type checking
|
||||
|
||||
Since Codon performs static type checking ahead of time, a
|
||||
few of Python's dynamic features are disallowed. For example,
|
||||
monkey patching classes at runtime (although Codon supports a
|
||||
form of this at compile time) or adding objects of different
|
||||
types to a collection.
|
||||
|
||||
These few restrictions are ultimately what allow Codon to
|
||||
compile to native code without any runtime performance overhead.
|
||||
Future versions of Codon will lift some of these restrictions
|
||||
by the introduction of e.g. implicit union types.
|
||||
|
||||
# Modules
|
||||
|
||||
While most of the commonly used builtin modules have Codon-native
|
||||
implementations, a few are not yet implemented. However these can
|
||||
still be used within Codon via `from python import`.
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
> **What is the goal of Codon?**
|
||||
|
||||
One of the main focuses of Codon is to bridge the gap between usability
|
||||
and performance. Codon aims to make writing high-performance software
|
||||
substantially easier, and to provide a common, unified framework for the
|
||||
development of such software across a range of domains.
|
||||
|
||||
> **I want to use Codon, but I have a large Python codebase I don't want to port.**
|
||||
|
||||
You can use Codon on a per-function basis via the `@codon` annotation, which
|
||||
can be used within Python codebases. This will compile only the annotated functions
|
||||
and automatically handle data conversions to and from Codon. It also allows for
|
||||
the use of any Codon-specific modules or extensions.
|
||||
|
||||
> **How does Codon compare to other Python implementations?**
|
||||
|
||||
Unlike many other performance-oriented Python implementations, such as
|
||||
PyPy or Numba, Codon is a standalone system implemented entirely
|
||||
independently of regular Python or any dynamic runtime, and therefore has
|
||||
far greater flexibility to generate optimized code. In fact, Codon will
|
||||
frequently generate the same code as that from an equivalent C or C++ program.
|
||||
This design choice also allows Codon to circumvent issues like Python's global
|
||||
interpretter lock, and thereby to take full advantage of parallelism and multithreading.
|
||||
|
||||
> **What about interoperability with other languages and frameworks?**
|
||||
|
||||
Interoperability is and will continue to be a priority for the Codon
|
||||
project. We don't want using Codon to render you unable to use all the
|
||||
other great frameworks and libraries that exist. Codon supports full
|
||||
interoperability with Python and C/C++.
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
# Using `codon`
|
||||
|
||||
The `codon` program can directly `run` Codon source in JIT mode:
|
||||
|
||||
```bash
|
||||
codon run myprogram.codon
|
||||
```
|
||||
|
||||
The default compilation and run mode is _debug_ (`-debug`).
|
||||
Compile and run with optimizations with the `-release` option:
|
||||
|
||||
```bash
|
||||
codon run -release myprogram.codon
|
||||
```
|
||||
|
||||
`codon` can also `build` executables:
|
||||
|
||||
```bash
|
||||
# generate 'myprogram' executable
|
||||
codon build -exe myprogram.codon
|
||||
|
||||
# generate 'foo' executable
|
||||
codon build -o foo myprogram.codon
|
||||
```
|
||||
|
||||
`codon` can produce object files:
|
||||
|
||||
```bash
|
||||
# generate 'myprogram.o' object file
|
||||
codon build -obj myprogram.codon
|
||||
|
||||
# generate 'foo.o' object file
|
||||
codon build -o foo.o myprogram.codon
|
||||
```
|
||||
|
||||
`codon` can produce LLVM IR:
|
||||
|
||||
```bash
|
||||
# generate 'myprogram.ll' object file
|
||||
codon build -llvm myprogram.codon
|
||||
|
||||
# generate 'foo.ll' object file
|
||||
codon build -o foo.ll myprogram.codon
|
||||
```
|
||||
|
||||
## Compile-time definitions
|
||||
|
||||
`codon` allows for compile-time definitions via the `-D` flag.
|
||||
For example, in the following code:
|
||||
|
||||
```python
|
||||
print(Int[BIT_WIDTH]())
|
||||
```
|
||||
|
||||
`BIT_WIDTH` can be specified on the command line as such:
|
||||
`codon run -DBIT_WIDTH=10 myprogram.codon`.
|
||||
|
|
@ -0,0 +1,254 @@
|
|||
If you know Python, you already know 99% of Codon. This section
|
||||
covers the Codon language as well as some of the key differences
|
||||
and additional features on top of Python.
|
||||
|
||||
# Printing
|
||||
|
||||
``` python
|
||||
print('hello world')
|
||||
|
||||
from sys import stderr
|
||||
print('hello world', end='', file=stderr)
|
||||
```
|
||||
|
||||
# Comments
|
||||
|
||||
``` python
|
||||
# Codon comments start with "# 'and go until the end of the line
|
||||
|
||||
"""
|
||||
Multi-line comments are
|
||||
possible like this.
|
||||
"""
|
||||
```
|
||||
|
||||
# Literals
|
||||
|
||||
``` python
|
||||
# Booleans
|
||||
True # type: bool
|
||||
False
|
||||
|
||||
# Numbers
|
||||
a = 1 # type: int; a signed 64-bit integer
|
||||
b = 1.12 # type: float; a 64-bit float (just like "double" in C)
|
||||
c = 5u # unsigned int; an unsigned 64-bit int
|
||||
d = Int[8](12) # 8-bit signed integer; you can go all the way to Int[2048]
|
||||
e = UInt[8](200) # 8-bit unsigned integer
|
||||
f = byte(3) # Codon's byte is equivalent to C's char; equivalent to Int[8]
|
||||
|
||||
h = 0x12AF # hexadecimal integers are also welcome
|
||||
g = 3.11e+9 # scientific notation is also supported
|
||||
g = .223 # and this is also float
|
||||
g = .11E-1 # and this as well
|
||||
|
||||
# Strings
|
||||
s = 'hello! "^_^" ' # type: str
|
||||
t = "hello there! \t \\ '^_^' " # \t is a tab character; \\ stands for \
|
||||
raw = r"hello\n" # raw strings do not escape slashes; this would print "hello\n"
|
||||
fstr = f"a is {a + 1}" # an f-string; prints "a is 2"
|
||||
fstr = f"hi! {a+1=}" # an f-string; prints "hi! a+1=2"
|
||||
t = """
|
||||
hello!
|
||||
multiline string
|
||||
"""
|
||||
|
||||
# The following escape sequences are supported:
|
||||
# \\, \', \", \a, \b, \f, \n, \r, \t, \v,
|
||||
# \xHHH (HHH is hex code), \OOO (OOO is octal code)
|
||||
```
|
||||
|
||||
# Assignments and operators
|
||||
|
||||
``` python
|
||||
a = 1 + 2 # this is 3
|
||||
a = (1).__add__(2) # you can use a function call instead of an operator; this is also 3
|
||||
a = int.__add__(1, 2) # this is equivalent to the previous line
|
||||
b = 5 / 2.0 # this is 2.5
|
||||
c = 5 // 2 # this is 2; // is an integer division
|
||||
a *= 2 # a is now 6
|
||||
```
|
||||
|
||||
Here is the list of binary operators and each one's associated magic method:
|
||||
|
||||
| Operator | Magic method | Description |
|
||||
|--------------------|----------------|------------------------------|
|
||||
| `+` | `__add__` | addition |
|
||||
| `-` | `__sub__` | subtraction |
|
||||
| `*` | `__mul__` | multiplication |
|
||||
| `/` | `__truediv__` | float (true) division |
|
||||
| `//` | `__floordiv__` | integer (floor) division |
|
||||
| `**` | `__pow__` | exponentiation |
|
||||
| `%` | `__mod__` | modulo |
|
||||
| `@` | `__matmul__` | matrix multiplication |
|
||||
| `&` | `__and__` | bitwise and |
|
||||
| <code>|<code> | `__or__` | bitwise or |
|
||||
| `^` | `__xor__` | bitwise xor |
|
||||
| `<<` | `__lshift__` | left bit shift |
|
||||
| `>>` | `__rshift__` | right bit shift |
|
||||
| `<` | `__lt__` | less than |
|
||||
| `<=` | `__le__` | less than or equal to |
|
||||
| `>` | `__gt__` | greater than |
|
||||
| `>=` | `__ge__` | greater than or equal to |
|
||||
| `==` | `__eq__` | equal to |
|
||||
| `!=` | `__ne__` | not equal to |
|
||||
| `in` | `__contains__` | belongs to |
|
||||
| `and` | none | boolean and (short-circuits) |
|
||||
| `or` | none | boolean or (short-circuits) |
|
||||
|
||||
Codon also has the following unary operators:
|
||||
|
||||
| Operator | Magic method | Description |
|
||||
|----------|--------------|------------------|
|
||||
| `~` | `__invert__` | bitwise not |
|
||||
| `+` | `__pos__` | unary positive |
|
||||
| `-` | `__neg__` | unary negation |
|
||||
| `not` | none | boolean negation |
|
||||
|
||||
# Control flow
|
||||
|
||||
## Conditionals
|
||||
|
||||
Codon supports the standard Python conditional syntax:
|
||||
|
||||
``` python
|
||||
if a or b or some_cond():
|
||||
print(1)
|
||||
elif whatever() or 1 < a <= b < c < 4: # chained comparisons are supported
|
||||
print('meh...')
|
||||
else:
|
||||
print('lo and behold!')
|
||||
|
||||
a = b if sth() else c # ternary conditional operator
|
||||
```
|
||||
|
||||
Codon extends the Python conditional syntax with a `match` statement,
|
||||
which is inspired by Rust's:
|
||||
|
||||
``` python
|
||||
match a + some_heavy_expr(): # assuming that the type of this expression is int
|
||||
case 1: # is it 1?
|
||||
print('hi')
|
||||
case 2 ... 10: # is it 2, 3, 4, 5, 6, 7, 8, 9 or 10?
|
||||
print('wow!')
|
||||
case _: # "default" case
|
||||
print('meh...')
|
||||
|
||||
match bool_expr(): # now it's a bool expression
|
||||
case True:
|
||||
print('yay')
|
||||
case False:
|
||||
print('nay')
|
||||
|
||||
match str_expr(): # now it's a str expression
|
||||
case 'abc': print("it's ABC time!")
|
||||
case 'def' | 'ghi': # you can chain multiple rules with the "|" operator
|
||||
print("it's not ABC time!")
|
||||
case s if len(s) > 10: print("so looong!") # conditional match expression
|
||||
case _: assert False
|
||||
|
||||
match some_tuple: # assuming type of some_tuple is Tuple[int, int]
|
||||
case (1, 2): ...
|
||||
case (a, _) if a == 42: # you can do away with useless terms with an underscore
|
||||
print('hitchhiker!')
|
||||
case (a, 50 ... 100) | (10 ... 20, b): # you can nest match expressions
|
||||
print('complex!')
|
||||
|
||||
match list_foo():
|
||||
case []: # [] matches an empty list
|
||||
print('A')
|
||||
case [1, 2, 3]: # make sure that list_foo() returns List[int] though!
|
||||
print('B')
|
||||
case [1, 2, ..., 5]: # matches any list that starts with 1 and 2 and ends with 5
|
||||
print('C')
|
||||
case [..., 6] | [6, ...]: # matches a list that starts or ends with 6
|
||||
print('D')
|
||||
case [..., w] if w < 0: # matches a list that ends with a negative integer
|
||||
print('E')
|
||||
case [...]: # any other list
|
||||
print('F')
|
||||
```
|
||||
|
||||
You can mix, match and chain match rules as long as the match type
|
||||
matches the expression type.
|
||||
|
||||
## Loops
|
||||
|
||||
Standard fare:
|
||||
|
||||
``` python
|
||||
a = 10
|
||||
while a > 0: # prints even numbers from 9 to 1
|
||||
a -= 1
|
||||
if a % 2 == 1:
|
||||
continue
|
||||
print(a)
|
||||
|
||||
for i in range(10): # prints numbers from 0 to 7, inclusive
|
||||
print(i)
|
||||
if i > 6:
|
||||
break
|
||||
```
|
||||
|
||||
`for` construct can iterate over any generator, which means any object
|
||||
that implements the `__iter__` magic method. In practice, generators,
|
||||
lists, sets, dictionaries, homogenous tuples, ranges, and many more
|
||||
types implement this method. If you need to implement one yourself,
|
||||
just keep in mind that `__iter__` is a generator and not a function.
|
||||
|
||||
# Imports
|
||||
|
||||
You can import functions and classes from another Codon module by doing:
|
||||
|
||||
``` python
|
||||
# Create foo.codon with a bunch of useful methods
|
||||
import foo
|
||||
|
||||
foo.useful1()
|
||||
p = foo.FooType()
|
||||
|
||||
# Create bar.codon with a bunch of useful methods
|
||||
from bar import x, y
|
||||
x(y)
|
||||
|
||||
from bar import z as bar_z
|
||||
bar_z()
|
||||
```
|
||||
|
||||
`import foo` looks for `foo.codon` or `foo/__init__.codon` in the
|
||||
current directory.
|
||||
|
||||
# Exceptions
|
||||
|
||||
Again, if you know how to do this in Python, you know how to do it in
|
||||
Codon:
|
||||
|
||||
``` python
|
||||
def throwable():
|
||||
raise ValueError("doom and gloom")
|
||||
|
||||
try:
|
||||
throwable()
|
||||
except ValueError as e:
|
||||
print("we caught the exception")
|
||||
except:
|
||||
print("ouch, we're in deep trouble")
|
||||
finally:
|
||||
print("whatever, it's done")
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
Right now, Codon cannot catch multiple exceptions in one statement. Thus
|
||||
`catch (Exc1, Exc2, Exc3) as var` will not compile, since the type of `var`
|
||||
needs to be known ahead of time.
|
||||
{% endhint %}
|
||||
|
||||
If you have an object that implements `__enter__` and `__exit__` methods
|
||||
to manage its lifetime (say, a `File`), you can use a `with` statement
|
||||
to make your life easier:
|
||||
|
||||
``` python
|
||||
with open('foo.txt') as f, open('foo_copy.txt', 'w') as fo:
|
||||
for l in f:
|
||||
fo.write(l)
|
||||
```
|
||||
|
|
@ -0,0 +1,176 @@
|
|||
Codon supports classes just like Python. However, you must declare
|
||||
class members and their types in the preamble of each class (like
|
||||
you would do with Python's dataclasses):
|
||||
|
||||
``` python
|
||||
class Foo:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __init__(self, x: int, y: int): # constructor
|
||||
self.x, self.y = x, y
|
||||
|
||||
def method(self):
|
||||
print(self.x, self.y)
|
||||
|
||||
f = Foo(1, 2)
|
||||
f.method() # prints "1 2"
|
||||
```
|
||||
|
||||
Unlike Python, Codon supports method overloading:
|
||||
|
||||
``` python
|
||||
class Foo:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __init__(self): # constructor
|
||||
self.x, self.y = 0, 0
|
||||
|
||||
def __init__(self, x: int, y: int): # another constructor
|
||||
self.x, self.y = x, y
|
||||
|
||||
def __init__(self, x: int, y: float): # yet another constructor
|
||||
self.x, self.y = x, int(y)
|
||||
|
||||
def method(self: Foo):
|
||||
print(self.x, self.y)
|
||||
|
||||
Foo().method() # prints "0 0"
|
||||
Foo(1, 2).method() # prints "1 2"
|
||||
Foo(1, 2.3).method() # prints "1 2"
|
||||
Foo(1.1, 2.3).method() # error: there is no Foo.__init__(float, float)
|
||||
```
|
||||
|
||||
Classes can also be generic:
|
||||
|
||||
``` python
|
||||
class Container[T]:
|
||||
elements: List[T]
|
||||
|
||||
def __init__(self, elements: List[T]):
|
||||
self.elements = elements
|
||||
```
|
||||
|
||||
Classes create objects that are passed by reference:
|
||||
|
||||
``` python
|
||||
class Point:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
p = Point(1, 2)
|
||||
q = p # this is a reference!
|
||||
p.x = 2
|
||||
print((p.x, p.y), (q.x, q.y)) # (2, 2), (2, 2)
|
||||
```
|
||||
|
||||
If you need to copy an object's contents, implement the `__copy__`
|
||||
magic method and use `q = copy(p)` instead.
|
||||
|
||||
Classes can inherit from other classes:
|
||||
|
||||
```python
|
||||
class NamedPoint(Point):
|
||||
name: str
|
||||
|
||||
def __init__(self, x: int, y: int, name: str):
|
||||
super().__init__(x, y)
|
||||
self.name = name
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
Currently, inheritance in Codon is static and polymorphism is not supported.
|
||||
{% endhint %}
|
||||
|
||||
# Named tuples
|
||||
|
||||
Codon also supports pass-by-value types via the `@tuple` annotation, which are
|
||||
effectively named tuples (equivalent to Python's `collections.namedtuple`):
|
||||
|
||||
``` python
|
||||
@tuple
|
||||
class Point:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
p = Point(1, 2)
|
||||
q = p # this is a copy!
|
||||
print((p.x, p.y), (q.x, q.y)) # (1, 2), (1, 2)
|
||||
```
|
||||
|
||||
However, named tuples are immutable. The following code will not compile:
|
||||
|
||||
``` python
|
||||
p = Point(1, 2)
|
||||
p.x = 2 # error: immutable type
|
||||
```
|
||||
|
||||
You can also add methods to named tuples:
|
||||
|
||||
``` python
|
||||
@tuple
|
||||
class Point:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __new__(): # named tuples are constructed via __new__, not __init__
|
||||
return Point(0, 1)
|
||||
|
||||
def some_method(self):
|
||||
return self.x + self.y
|
||||
|
||||
p = Point() # p is (0, 1)
|
||||
print(p.some_method()) # 1
|
||||
```
|
||||
|
||||
# Type extensions
|
||||
|
||||
Suppose you have a class that lacks a method or an operator that might
|
||||
be really useful. Codon provides an `@extend` annotation that allows
|
||||
programmers to add and modify methods of various types at compile time,
|
||||
including built-in types like `int` or `str`. This actually allows much
|
||||
of the functionality of built-in types to be implemented in Codon as type
|
||||
extensions in the standard library.
|
||||
|
||||
``` python
|
||||
class Foo:
|
||||
...
|
||||
|
||||
f = Foo(...)
|
||||
|
||||
# We need foo.cool() but it does not exist... not a problem for Codon
|
||||
@extend
|
||||
class Foo:
|
||||
def cool(self: Foo):
|
||||
...
|
||||
|
||||
f.cool() # works!
|
||||
|
||||
# Let's add support for adding integers and strings:
|
||||
@extend
|
||||
class int:
|
||||
def __add__(self: int, other: str):
|
||||
return self + int(other)
|
||||
|
||||
print(5 + '4') # 9
|
||||
```
|
||||
|
||||
Note that all type extensions are performed strictly at compile time and
|
||||
incur no runtime overhead.
|
||||
|
||||
# Magic methods
|
||||
|
||||
Here is a list of useful magic methods that you might want to add and
|
||||
overload:
|
||||
|
||||
| Magic method | Description |
|
||||
|---------------|-------------------------------------------------------------------------------------|
|
||||
| `__copy__` | copy-constructor for `copy` method |
|
||||
| `__len__` | for `len` method |
|
||||
| `__bool__` | for `bool` method and condition checking |
|
||||
| `__getitem__` | overload `obj[key]` |
|
||||
| `__setitem__` | overload `obj[key] = value` |
|
||||
| `__delitem__` | overload `del obj[key]` |
|
||||
| `__iter__` | support iterating over the object |
|
||||
| `__repr__` | support printing and `str` conversion |
|
||||
|
|
@ -0,0 +1,134 @@
|
|||
Collections are largely the same as in Python:
|
||||
|
||||
``` python
|
||||
l = [1, 2, 3] # type: List[int]; a list of integers
|
||||
s = {1.1, 3.3, 2.2, 3.3} # type: Set[float]; a set of floats
|
||||
d = {1: 'hi', 2: 'ola', 3: 'zdravo'} # type: Dict[int, str]; a dictionary of int to str
|
||||
|
||||
ln = [] # an empty list whose type is inferred based on usage
|
||||
ln = List[int]() # an empty list with explicit element type
|
||||
dn = {} # an empty dict whose type is inferred based on usage
|
||||
dn = Dict[int, float]() # an empty dictionary with explicit element types
|
||||
sn = set() # an empty set whose type is inferred based on usage
|
||||
sn = Set[str]() # an empty set with explicit element type
|
||||
```
|
||||
|
||||
Lists also take an optional `capacity` constructor argument, which can be useful
|
||||
when creating large lists:
|
||||
|
||||
``` python
|
||||
squares = list(capacity=1_000_000) # list with room for 1M elements
|
||||
|
||||
# Fill the list
|
||||
for i in range(1_000_000):
|
||||
squares.append(i ** 2)
|
||||
```
|
||||
|
||||
{% hint style="info" %}
|
||||
Dictionaries and sets are unordered and are based on [klib](https://github.com/attractivechaos/klib).
|
||||
{% endhint %}
|
||||
|
||||
# Comprehensions
|
||||
|
||||
Comprehensions are a nifty, Pythonic way to create collections, and are fully
|
||||
supported by Codon:
|
||||
|
||||
``` python
|
||||
l = [i for i in range(5)] # type: List[int]; l is [0, 1, 2, 3, 4]
|
||||
l = [i for i in range(15) if i % 2 == 1 if i > 10] # type: List[int]; l is [11, 13]
|
||||
l = [i * j for i in range(5) for j in range(5) if i == j] # l is [0, 1, 4, 9, 16]
|
||||
|
||||
s = {abs(i - j) for i in range(5) for j in range(5)} # s is {0, 1, 2, 3, 4}
|
||||
d = {i: f'item {i+1}' for i in range(3)} # d is {0: "item 1", 1: "item 2", 2: "item 3"}
|
||||
```
|
||||
|
||||
You can also use generators to create collections:
|
||||
|
||||
``` python
|
||||
g = (i for i in range(10))
|
||||
print(list(g)) # prints list of integers from 0 to 9, inclusive
|
||||
```
|
||||
|
||||
# Tuples
|
||||
|
||||
``` python
|
||||
t = (1, 2.3, 'hi') # type: Tuple[int, float, str]
|
||||
t[1] # type: float
|
||||
u = (1, ) # type: Tuple[int]
|
||||
```
|
||||
|
||||
As all types must be known at compile time, tuple indexing works only if
|
||||
a tuple is homogenous (all types are the same) or if the value of the
|
||||
index is known at compile time.
|
||||
|
||||
You can, however, iterate over heterogenous tuples in Codon. This is
|
||||
achieved behind the scenes by unrolling the loop to accommodate the
|
||||
different types.
|
||||
|
||||
``` python
|
||||
t = (1, 2.3, 'hi')
|
||||
t[1] # works because 1 is a constant int
|
||||
|
||||
x = int(argv[1])
|
||||
t[x] # compile error: x is not known at compile time
|
||||
|
||||
# This is a homogenous tuple (all member types are the same)
|
||||
u = (1, 2, 3) # type: Tuple[int, int, int]
|
||||
u[x] # works because tuple members share the same type regardless of x
|
||||
for i in u: # works
|
||||
print(i)
|
||||
|
||||
# Also works
|
||||
v = (42, 'x', 3.14)
|
||||
for i in v:
|
||||
print(i)
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
Just like in Python, tuples are immutable, so `a = (1, 2); a[1] = 1` will not compile.
|
||||
{% endhint %}
|
||||
|
||||
Codon supports most of Python's tuple unpacking syntax:
|
||||
|
||||
``` python
|
||||
x, y = 1, 2 # x is 1, y is 2
|
||||
(x, (y, z)) = 1, (2, 3) # x is 1, y is 2, z is 3
|
||||
[x, (y, z)] = (1, [2, 3]) # x is 1, y is 2, z is 3
|
||||
|
||||
l = range(1, 8) # l is [1, 2, 3, 4, 5, 6, 7]
|
||||
a, b, *mid, c = l # a is 1, b is 2, mid is [3, 4, 5, 6], c is 7
|
||||
a, *end = l # a is 1, end is [2, 3, 4, 5, 6, 7]
|
||||
*beg, c = l # c is 7, beg is [1, 2, 3, 4, 5, 6]
|
||||
(*x, ) = range(3) # x is [0, 1, 2]
|
||||
*x = range(3) # error: this does not work
|
||||
|
||||
*sth, a, b = (1, 2, 3, 4) # sth is (1, 2), a is 3, b is 4
|
||||
*sth, a, b = (1.1, 2, 3.3, 4) # error: this only works on homogenous tuples for now
|
||||
|
||||
(x, y), *pff, z = [1, 2], 'this'
|
||||
print(x, y, pff, z) # x is 1, y is 2, pff is an empty tuple --- () ---, and z is "this"
|
||||
|
||||
s, *q = 'XYZ' # works on strings as well; s is "X" and q is "YZ"
|
||||
```
|
||||
|
||||
# Strong typing
|
||||
|
||||
Because Codon is strongly typed, these won't compile:
|
||||
|
||||
``` python
|
||||
l = [1, 's'] # is it a List[int] or List[str]? you cannot mix-and-match types
|
||||
d = {1: 'hi'}
|
||||
d[2] = 3 # d is a Dict[int, str]; the assigned value must be a str
|
||||
|
||||
t = (1, 2.2) # Tuple[int, float]
|
||||
lt = list(t) # compile error: t is not homogenous
|
||||
|
||||
lp = [1, 2.1, 3, 5] # compile error: Codon will not automatically cast a float to an int
|
||||
```
|
||||
|
||||
This will work, though:
|
||||
|
||||
``` python
|
||||
u = (1, 2, 3)
|
||||
lu = list(u) # works: u is homogenous
|
||||
```
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
Codon supports a number of additional types that are not present
|
||||
in plain Python.
|
||||
|
||||
# Arbitrary-width integers
|
||||
|
||||
Codon's `int` type is a 64-bit signed integer. However, Codon
|
||||
supports arbitrary-width signed and unsigned integers:
|
||||
|
||||
``` python
|
||||
a = Int[16](42) # signed 16-bit integer 42
|
||||
b = UInt[128](99) # unsigned 128-bit integer 99
|
||||
```
|
||||
|
||||
The Codon standard library provides shorthands for the common
|
||||
variants:
|
||||
|
||||
- `i8`/`u8`: signed/unsigned 8-bit integer
|
||||
- `i16`/`u16`: signed/unsigned 16-bit integer
|
||||
- `i32`/`u32`: signed/unsigned 32-bit integer
|
||||
- `i64`/`u64`: signed/unsigned 64-bit integer
|
||||
|
||||
# Pointers
|
||||
|
||||
Codon has a `Ptr[T]` type that represents a pointer to an object
|
||||
of type `T`. Pointers can be useful when interfacing with C. The
|
||||
`__ptr__` keyword can also be used to obtain a pointer to a variable:
|
||||
|
||||
``` python
|
||||
p = Ptr[int](100) # allocate a buffer of 100 ints
|
||||
p = Ptr[int]() # null pointer
|
||||
|
||||
x = 42
|
||||
p = __ptr__(x) # pointer to x, like "&x" in C
|
||||
|
||||
from C import foo(Ptr[int])
|
||||
foo(p) # pass pointer to C function
|
||||
```
|
||||
|
||||
The `cobj` alias corresponds to `void*` in C and represents a generic
|
||||
C or C++ object.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Using pointers directly circumvents any runtime checks, so dereferencing a
|
||||
null pointer, for example, will cause a segmentation fault just like in C.
|
||||
{% endhint %}
|
||||
|
||||
# Static arrays
|
||||
|
||||
The `__array__` keyword can be used to allocate static arrays on the stack:
|
||||
|
||||
``` python
|
||||
def foo(n):
|
||||
arr = __array__[int](5) # similar to "long arr[5]" in C
|
||||
arr[0] = 11
|
||||
arr[1] = arr[0] + 1
|
||||
...
|
||||
```
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
Codon can seamlessly call functions from C and Python:
|
||||
|
||||
``` python
|
||||
from C import pow(float, float) -> float
|
||||
pow(2.0, 2.0) # 4.0
|
||||
|
||||
# Import and rename function
|
||||
# cobj is a C pointer (void*, char*, etc.)
|
||||
from C import puts(cobj) -> void as print_line
|
||||
print_line("hello".c_str()) # prints "hello"; c_str() converts Codon str to C string
|
||||
```
|
||||
|
||||
`from C import` only works if the symbol is available to the program. If
|
||||
you are running your programs via `codon`, you can link dynamic
|
||||
libraries with `-l`: `codon run -l /path/to/library.so ...`.
|
||||
|
||||
You can also load shared libraries with `dlopen`:
|
||||
|
||||
``` python
|
||||
LIBRARY = "somelib.so"
|
||||
from C import LIBRARY.mymethod(int, float) -> cobj
|
||||
from C import LIBRARY.myothermethod(int, float) -> cobj as my2
|
||||
foo = mymethod(1, 2.2)
|
||||
foo2 = my2(4, 3.2)
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
When importing external non-Codon functions, you must explicitly specify
|
||||
argument and return types.
|
||||
{% endhint %}
|
||||
|
||||
How about Python? If you have set the `CODON_PYTHON` environment
|
||||
variable to point to the Python library, you can do:
|
||||
|
||||
``` python
|
||||
from python import mymodule.myfunction(str) -> int as foo
|
||||
print(foo("bar"))
|
||||
```
|
||||
|
||||
You might want to execute more complex Python code within Codon. To that
|
||||
end, you can use Codon's `@python` annotation:
|
||||
|
||||
``` python
|
||||
@python
|
||||
def scipy_eigenvalues(i: List[List[float]]) -> List[float]:
|
||||
# Code within this block is executed by the Python interpreter,
|
||||
# so it must be valid Python code.
|
||||
import scipy.linalg
|
||||
import numpy as np
|
||||
data = np.array(i)
|
||||
eigenvalues, _ = scipy.linalg.eig(data)
|
||||
return list(eigenvalues)
|
||||
print(scipy_eigenvalues([[1.0, 2.0], [3.0, 4.0]])) # [-0.372281, 5.37228]
|
||||
```
|
||||
|
||||
Codon will automatically bridge any object that implements the
|
||||
`__to_py__` and `__from_py__` magic methods. All standard Codon types
|
||||
already implement these methods.
|
||||
|
|
@ -0,0 +1,119 @@
|
|||
Functions are defined as follows:
|
||||
|
||||
``` python
|
||||
def foo(a, b, c):
|
||||
return a + b + c
|
||||
|
||||
print(foo(1, 2, 3)) # prints 6
|
||||
```
|
||||
|
||||
Functions don't have to return a value:
|
||||
|
||||
``` python
|
||||
def proc(a, b):
|
||||
print(a, 'followed by', b)
|
||||
|
||||
proc(1, 's')
|
||||
|
||||
|
||||
def proc2(a, b):
|
||||
if a == 5:
|
||||
return
|
||||
print(a, 'followed by', b)
|
||||
|
||||
proc2(1, 's')
|
||||
proc2(5, 's') # this prints nothing
|
||||
```
|
||||
|
||||
Codon is a strongly-typed language, so you can restrict argument and
|
||||
return types:
|
||||
|
||||
``` python
|
||||
def fn(a: int, b: float):
|
||||
return a + b # this works because int implements __add__(float)
|
||||
|
||||
fn(1, 2.2) # 3.2
|
||||
fn(1.1, 2) # error: 1.1. is not an int
|
||||
|
||||
|
||||
def fn2(a: int, b):
|
||||
return a - b
|
||||
|
||||
fn2(1, 2) # -1
|
||||
fn2(1, 1.1) # -0.1; works because int implements __sub__(float)
|
||||
fn2(1, 's') # error: there is no int.__sub__(str)!
|
||||
|
||||
|
||||
def fn3(a, b) -> int:
|
||||
return a + b
|
||||
|
||||
fn3(1, 2) # works, since 1 + 2 is an int
|
||||
fn3('s', 'u') # error: 's'+'u' returns 'su' which is str
|
||||
# but the signature indicates that it must return int
|
||||
```
|
||||
|
||||
Default and named arguments are also supported:
|
||||
|
||||
``` python
|
||||
def foo(a, b: int, c: float = 1.0, d: str = 'hi'):
|
||||
print(a, b, c, d)
|
||||
|
||||
foo(1, 2) # prints "1 2 1 hi"
|
||||
foo(1, d='foo', b=1) # prints "1 1 1 foo"
|
||||
```
|
||||
|
||||
As are optional arguments:
|
||||
|
||||
``` python
|
||||
# type of b promoted to Optional[int]
|
||||
def foo(a, b: int = None):
|
||||
print(a, b + 1)
|
||||
|
||||
foo(1, 2) # prints "1 3"
|
||||
foo(1) # raises ValueError, since b is None
|
||||
```
|
||||
|
||||
# Generics
|
||||
|
||||
Codon emulates Python's lax runtime type checking using a technique known as
|
||||
*monomorphization*. If a function has an argument without a type definition,
|
||||
Codon will treat it as a *generic* function, and will generate different instantiations
|
||||
for each different invocation:
|
||||
|
||||
``` python
|
||||
def foo(x):
|
||||
print(x) # print relies on typeof(x).__repr__(x) method to print the representation of x
|
||||
|
||||
foo(1) # Codon automatically generates foo(x: int) and calls int.__repr__ when needed
|
||||
foo('s') # Codon automatically generates foo(x: str) and calls str.__repr__ when needed
|
||||
foo([1, 2]) # Codon automatically generates foo(x: List[int]) and calls List[int].__repr__ when needed
|
||||
```
|
||||
|
||||
But what if you need to mix type definitions and generic types? Say,
|
||||
your function can take a list of *anything*? You can use generic
|
||||
type parameters:
|
||||
|
||||
``` python
|
||||
def foo(x: List[T], T: type):
|
||||
print(x)
|
||||
|
||||
foo([1, 2]) # prints [1, 2]
|
||||
foo(['s', 'u']) # prints [s, u]
|
||||
foo(5) # error: 5 is not a list!
|
||||
foo(['s', 'u'], int) # fails: T is int, so foo expects List[int] but it got List[str]
|
||||
|
||||
|
||||
def foo(x, R: type) -> R:
|
||||
print(x)
|
||||
return 1
|
||||
|
||||
foo(4, int) # prints 4, returns 1
|
||||
foo(4, str) # error: return type is str, but foo returns int!
|
||||
```
|
||||
|
||||
{% hint style="info" %}
|
||||
Coming from C++? `foo(x: List[T], T: type): ...` is roughly the same as
|
||||
`template<typename T, typename U> U foo(T x) { ... }`.
|
||||
{% endhint %}
|
||||
|
||||
Generic type parameters are an optional way to enforce various typing constraints.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Codon supports generators, and in fact they are heavily optimized in
|
||||
the compiler so as to typically eliminate any overhead:
|
||||
|
||||
``` python
|
||||
def gen(i):
|
||||
while i < 10:
|
||||
yield i ** 2
|
||||
i += 1
|
||||
|
||||
print(list(gen(10))) # prints [0, 1, 4, ..., 81]
|
||||
print(list(gen(0))) # prints []
|
||||
```
|
||||
|
||||
You can also use `yield` to implement coroutines: `yield` suspends the
|
||||
function, while `(yield)` (i.e. with parenthesis) receives a value, as
|
||||
in Python.
|
||||
|
||||
``` python
|
||||
def mysum(start):
|
||||
m = start
|
||||
while True:
|
||||
a = (yield) # receives the input of coroutine.send() call
|
||||
if a == -1:
|
||||
break # exits the coroutine
|
||||
m += a
|
||||
yield m
|
||||
|
||||
iadder = mysum(0) # assign a coroutine
|
||||
next(iadder) # activate it
|
||||
for i in range(10):
|
||||
iadder.send(i) # send a value to coroutine
|
||||
print(iadder.send(-1)) # prints 45
|
||||
```
|
||||
|
||||
Generator expressions are also supported:
|
||||
|
||||
``` python
|
||||
squares = (i ** 2 for i in range(10))
|
||||
for i,s in enumerate(squares):
|
||||
print(i, 'x', i, '=', s)
|
||||
```
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
Codon allows inline LLVM IR via the `@llvm` annotation:
|
||||
|
||||
``` python
|
||||
@llvm
|
||||
def llvm_add(a: int, b: int) -> int:
|
||||
%res = add i64 %a, %b
|
||||
ret i64 %res
|
||||
|
||||
print(llvm_add(3, 4)) # 7
|
||||
```
|
||||
|
||||
Note that LLVM functions must explicitly specify argument
|
||||
and return types.
|
||||
|
||||
LLVM functions can also be generic, and a format specifier
|
||||
in the body will be replaced by the appropriate LLVM type:
|
||||
|
||||
``` python
|
||||
@llvm
|
||||
def llvm_add[T](a: T, b: T) -> T:
|
||||
%res = add {=T} %a, %b
|
||||
ret {=T} %res
|
||||
|
||||
print(llvm_add(3, 4)) # 7
|
||||
print(llvm_add(i8(5), i8(6))) # 11
|
||||
```
|
||||
|
||||
You can also access LLVM intrinsics with `declare`:
|
||||
|
||||
``` python
|
||||
@llvm
|
||||
def popcnt(n: int) -> int:
|
||||
declare i64 @llvm.ctpop.i64(i64)
|
||||
%0 = call i64 @llvm.ctpop.i64(i64 %n)
|
||||
ret i64 %0
|
||||
|
||||
print(popcnt(42)) # 3
|
||||
```
|
||||
|
||||
# Annotations
|
||||
|
||||
Sometimes it can be helpful to annotate `@llvm` functions to give
|
||||
the compiler more information as to how they behave. Codon has
|
||||
a number of default annotations for LLVM functions (all of
|
||||
which also apply to external/C functions):
|
||||
|
||||
- `@pure`: Function does not capture arguments (aside from
|
||||
return value capturing as in `def foo(x): return x`), does not
|
||||
modify arguments, and has no side effects. This is a
|
||||
mathematically "pure" function.
|
||||
|
||||
- `@no_side_effect`: Very similar to `@pure` but function may
|
||||
return different results on different calls, such as the C
|
||||
function `time()`.
|
||||
|
||||
- `@nocapture`: Function does not capture any of its arguments
|
||||
(again excluding return value capturing).
|
||||
|
||||
- `@self_captures`: Function's first (`self`) argument captures
|
||||
the other arguments, an example being `List.__setitem__()`.
|
||||
|
||||
These are mutually-exclusive annotations. Another complementary
|
||||
annotation `@derives` can be used to indicate that the return
|
||||
value of the function captures its arguments.
|
||||
|
||||
These annotations are completely optional and do not affect
|
||||
program semantics.
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
Sometimes, certain values or conditions need to be known
|
||||
at compile time. For example, the bit width `N` of an
|
||||
integer type `Int[N]`, or the size `M` of a static array
|
||||
`__array__[int](M)` need to be compile time constants.
|
||||
|
||||
To accomodate this, Codon uses *static values*, i.e.
|
||||
values that are known and can be operated on at compile
|
||||
time. `Static[T]` represents a static value of type `T`.
|
||||
Currently, `T` can only be `int` or `str`.
|
||||
|
||||
For example, we can parameterize the bit width of an
|
||||
integer type as follows:
|
||||
|
||||
``` python
|
||||
N: Static[int] = 32
|
||||
|
||||
a = Int[N](10) # 32-bit integer 10
|
||||
b = Int[2 * N](20) # 64-bit integer 20
|
||||
```
|
||||
|
||||
All of the standard arithmetic operations can be applied
|
||||
to static integers to produce new static integers.
|
||||
|
||||
Statics can also be passed to the `codon` compiler via the
|
||||
`-D` flag, as in `-DN=32`.
|
||||
|
||||
Classes can also be parameterized by statics:
|
||||
|
||||
``` python
|
||||
class MyInt[N: Static[int]]:
|
||||
n: Int[N]
|
||||
|
||||
x = MyInt[16](i16(42))
|
||||
```
|
||||
|
||||
# Static evaluation
|
||||
|
||||
In certain cases a program might need to check a particular
|
||||
type and perform different actions based on it. For example:
|
||||
|
||||
``` python
|
||||
def flatten(x):
|
||||
if isinstance(x, list):
|
||||
for a in x:
|
||||
flatten(a)
|
||||
else:
|
||||
print(x)
|
||||
|
||||
flatten([[1,2,3], [], [4, 5], [6]])
|
||||
```
|
||||
|
||||
Standard static typing on this program would be problematic
|
||||
since, if `x` is an `int`, it would not be iterable and hence
|
||||
would produce an error on `for a in x`. Codon solves this problem
|
||||
by evaluating certain conditions at compile time, such as
|
||||
`isinstance(x, list)`, and avoiding type checking blocks that it
|
||||
knows will never be reached. In fact, this program works and flattens
|
||||
the argument list.
|
||||
|
||||
Static evaluation works with plain static types as well as general
|
||||
types used in conjunction with `type`, `isinstance` or `hasattr`.
|
||||
|
|
@ -1,24 +0,0 @@
|
|||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line.
|
||||
SPHINXOPTS =
|
||||
SPHINXBUILD = sphinx-build
|
||||
SPHINXPROJ = codon
|
||||
SOURCEDIR = .
|
||||
BUILDDIR = _build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile clean
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
clean:
|
||||
rm -rf ../doxygen/xml/* api/
|
||||
@$(SPHINXBUILD) -M clean "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
|
@ -1,157 +0,0 @@
|
|||
from collections import defaultdict
|
||||
|
||||
import re
|
||||
|
||||
from typing import Any,Dict,Iterable,Iterator,List,NamedTuple,Tuple
|
||||
from typing import cast
|
||||
|
||||
from docutils.parsers.rst import directives
|
||||
from sphinx import addnodes
|
||||
from sphinx.directives import ObjectDescription
|
||||
from sphinx.domains import Domain
|
||||
from sphinx.domains import Index
|
||||
from sphinx.roles import XRefRole
|
||||
from sphinx.util.nodes import make_refnode
|
||||
from sphinx.locale import _,__
|
||||
from docutils import nodes
|
||||
from docutils.nodes import Element,Node
|
||||
from docutils.parsers.rst import directives
|
||||
|
||||
from sphinx import addnodes
|
||||
from sphinx.addnodes import pending_xref,desc_signature
|
||||
from sphinx.application import Sphinx
|
||||
from sphinx.builders import Builder
|
||||
from sphinx.directives import ObjectDescription
|
||||
from sphinx.domains import Domain,ObjType,Index,IndexEntry
|
||||
from sphinx.environment import BuildEnvironment
|
||||
from sphinx.locale import _,__
|
||||
from sphinx.pycode.ast import ast,parse as ast_parse
|
||||
from sphinx.roles import XRefRole
|
||||
from sphinx.util import logging
|
||||
from sphinx.util.docfields import Field,GroupedField,TypedField
|
||||
from sphinx.util.docutils import SphinxDirective
|
||||
from sphinx.util.inspect import signature_from_str
|
||||
from sphinx.util.nodes import make_id,make_refnode
|
||||
from sphinx.util.typing import TextlikeNode
|
||||
|
||||
import sphinx.domains.python
|
||||
|
||||
codon_sig_re=re.compile(r'''^([\w.]*\.)?(\w+)\s*(?:\[\s*(.*)\s*\])?\s*(?:\(\s*(.*)\s*\)(?:\s*->\s*(.*))?)?$''',re.VERBOSE)
|
||||
|
||||
|
||||
def handle_signature(self,sig: str,signode: desc_signature) -> Tuple[str,str]:
|
||||
m=codon_sig_re.match(sig)
|
||||
if m is None:
|
||||
raise ValueError
|
||||
prefix,name,generics,arglist,retann=m.groups()
|
||||
|
||||
# determine module and class name (if applicable), as well as full name
|
||||
modname=self.options.get('module',self.env.ref_context.get('py:module'))
|
||||
classname=self.env.ref_context.get('py:class')
|
||||
isextension=self.objtype=='extension'
|
||||
if classname:
|
||||
add_module=False
|
||||
if prefix and (prefix==classname or prefix.startswith(classname+".")):
|
||||
fullname=prefix+name
|
||||
# class name is given again in the signature
|
||||
prefix=prefix[len(classname):].lstrip('.')
|
||||
elif prefix:
|
||||
# class name is given in the signature, but different
|
||||
# (shouldn't happen)
|
||||
fullname=classname+'.'+prefix+name
|
||||
else:
|
||||
# class name is not given in the signature
|
||||
fullname=classname+'.'+name
|
||||
else:
|
||||
add_module=True
|
||||
if prefix:
|
||||
classname=prefix.rstrip('.')
|
||||
fullname=prefix+name
|
||||
else:
|
||||
classname=''
|
||||
fullname=name
|
||||
|
||||
signode['module']=modname
|
||||
if modname.startswith('..'): # HACK: no idea why this happens
|
||||
modname=modname[2:]
|
||||
signode['class']=classname
|
||||
signode['fullname']=fullname
|
||||
|
||||
sig_prefix=self.get_signature_prefix(sig)
|
||||
if sig_prefix:
|
||||
signode+=addnodes.desc_annotation(sig_prefix,sig_prefix)
|
||||
|
||||
if not isextension:
|
||||
if prefix:
|
||||
signode+=addnodes.desc_addname(prefix,prefix)
|
||||
elif add_module and self.env.config.add_module_names:
|
||||
if modname and modname!='exceptions':
|
||||
# exceptions are a special case, since they are documented in the
|
||||
# 'exceptions' module.
|
||||
nodetext=modname+'.'
|
||||
signode+=addnodes.desc_addname(nodetext,nodetext)
|
||||
signode+=addnodes.desc_name(name,name)
|
||||
else:
|
||||
ref=type_to_xref(str(name),self.env)
|
||||
signode+=addnodes.desc_name(name,name)
|
||||
|
||||
if generics:
|
||||
signode+=addnodes.desc_name("generics","["+generics+"]")
|
||||
if arglist:
|
||||
try:
|
||||
signode+=sphinx.domains.python._parse_arglist(arglist,self.env)
|
||||
except SyntaxError:
|
||||
# fallback to parse arglist original parser.
|
||||
# it supports to represent optional arguments (ex. "func(foo [, bar])")
|
||||
sphinx.domains.python._pseudo_parse_arglist(signode,arglist)
|
||||
except NotImplementedError as exc:
|
||||
logger.warning("could not parse arglist (%r): %s",arglist,exc,location=signode)
|
||||
sphinx.domains.python._pseudo_parse_arglist(signode,arglist)
|
||||
else:
|
||||
if self.needs_arglist():
|
||||
# for callables, add an empty parameter list
|
||||
signode+=addnodes.desc_parameterlist()
|
||||
|
||||
if retann:
|
||||
children=sphinx.domains.python._parse_annotation(retann,self.env)
|
||||
signode+=addnodes.desc_returns(retann,'',*children)
|
||||
|
||||
anno=self.options.get('annotation')
|
||||
if anno:
|
||||
signode+=addnodes.desc_annotation(' '+anno,' '+anno)
|
||||
|
||||
return fullname,prefix
|
||||
|
||||
|
||||
sphinx.domains.python.PyObject.handle_signature=handle_signature
|
||||
|
||||
from sphinx.domains.python import *
|
||||
|
||||
logger=logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CodonDomain(PythonDomain):
|
||||
name='codon'
|
||||
label='Codon'
|
||||
object_types={'function':ObjType(_('function'),'func','obj'),# 'data': ObjType(_('data'), 'data', 'obj'),
|
||||
'class':ObjType(_('class'),'class','exc','obj'),'type':ObjType(_('class'),'exc','class','obj'),'extension':ObjType(_('class'),'exc','class','obj'),
|
||||
'exception':ObjType(_('exception'),'exc','class','obj'),'method':ObjType(_('method'),'meth','obj'),# 'classmethod': ObjType(_('class method'), 'meth', 'obj'),
|
||||
# 'staticmethod': ObjType(_('static method'), 'meth', 'obj'),
|
||||
# 'attribute': ObjType(_('attribute'), 'attr', 'obj'),
|
||||
'module':ObjType(_('module'),'mod','obj'),} # type: Dict[str, ObjType]
|
||||
|
||||
directives={'function':PyFunction,'data':PyVariable,'class':PyClasslike,'exception':PyClasslike,'type':PyClasslike,'extension':PyClasslike,'method':PyMethod,'classmethod':PyClassMethod,
|
||||
'staticmethod':PyStaticMethod,'attribute':PyAttribute,'module':PyModule,'currentmodule':PyCurrentModule,'decorator':PyDecoratorFunction,'decoratormethod':PyDecoratorMethod,}
|
||||
roles={'data':PyXRefRole(),'exc':PyXRefRole(),'func':PyXRefRole(fix_parens=True),'class':PyXRefRole(),# 'const': PyXRefRole(),
|
||||
'attr':PyXRefRole(),'meth':PyXRefRole(fix_parens=True),'mod':PyXRefRole(),'obj':PyXRefRole(),}
|
||||
initial_data={'objects':{}, # fullname -> docname, objtype
|
||||
'modules':{}, # modname -> docname, synopsis, platform, deprecated
|
||||
} # type: Dict[str, Dict[str, Tuple[Any]]]
|
||||
indices=[PythonModuleIndex,]
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_domain(CodonDomain)
|
||||
# app.connect('object-description-simplify', filter_meta_fields)
|
||||
|
||||
return {'version':'0.1','parallel_read_safe':True,'parallel_write_safe':True,}
|
||||
|
|
@ -1,44 +0,0 @@
|
|||
Building from Source
|
||||
====================
|
||||
|
||||
Unless you really need to build Codon for whatever reason, we strongly
|
||||
recommend using pre-built binaries if possible.
|
||||
|
||||
Dependencies
|
||||
------------
|
||||
|
||||
Codon depends on LLVM 12, which can be installed via most package managers. To
|
||||
build LLVM 12 yourself, you can do the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone --depth 1 -b release/12.x https://github.com/llvm/llvm-project
|
||||
mkdir -p llvm-project/llvm/build
|
||||
cd llvm-project/llvm/build
|
||||
cmake .. \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DLLVM_INCLUDE_TESTS=OFF \
|
||||
-DLLVM_ENABLE_RTTI=ON \
|
||||
-DLLVM_ENABLE_ZLIB=OFF \
|
||||
-DLLVM_ENABLE_TERMINFO=OFF \
|
||||
-DLLVM_TARGETS_TO_BUILD=host
|
||||
make
|
||||
make install
|
||||
|
||||
Build
|
||||
-----
|
||||
|
||||
The following can generally be used to build Codon. The build process will automatically
|
||||
download and build several smaller dependencies.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
mkdir build
|
||||
(cd build && cmake .. -DCMAKE_BUILD_TYPE=Release \
|
||||
-DLLVM_DIR=$(llvm-config --cmakedir) \
|
||||
-DCMAKE_C_COMPILER=clang \
|
||||
-DCMAKE_CXX_COMPILER=clang++)
|
||||
cmake --build build --config Release
|
||||
|
||||
This should produce the ``codon`` executable in the ``build`` directory, as well as
|
||||
``codon_test`` which runs the test suite.
|
||||
|
|
@ -1,208 +0,0 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file does only contain a selection of the most common options. For a
|
||||
# full list see the documentation:
|
||||
# http://www.sphinx-doc.org/en/stable/config
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.abspath("./config"))
|
||||
sys.path.insert(0, os.path.abspath("./_ext"))
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'Codon'
|
||||
copyright = '2019-2021, Exaloop, Inc.'
|
||||
author = 'exaloop'
|
||||
|
||||
from config import CODON_VERSION_MAJOR, CODON_VERSION_MINOR, CODON_VERSION_PATCH
|
||||
ver_major = CODON_VERSION_MAJOR
|
||||
ver_minor = CODON_VERSION_MINOR
|
||||
ver_patch = CODON_VERSION_PATCH
|
||||
# The short X.Y version
|
||||
version = '%d.%d' % (ver_major, ver_minor)
|
||||
# The full version, including alpha/beta/rc tags
|
||||
release = '%d.%d.%d' % (ver_major, ver_minor, ver_patch)
|
||||
|
||||
# Logo path
|
||||
html_logo = 'logo.png'
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#
|
||||
# needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'sphinx.ext.todo',
|
||||
'sphinx.ext.mathjax',
|
||||
'sphinx.ext.githubpages',
|
||||
'sphinx.ext.intersphinx',
|
||||
# 'breathe',
|
||||
# 'exhale',
|
||||
'codon',
|
||||
]
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
#
|
||||
# source_suffix = ['.rst', '.md']
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path .
|
||||
exclude_patterns = [u'_build', 'Thumbs.db', '.DS_Store']
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = 'sphinx'
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = 'sphinx_book_theme'
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#
|
||||
html_theme_options = {
|
||||
"repository_url": "https://github.com/exaloop/codon",
|
||||
"repository_branch": "develop",
|
||||
"path_to_docs": "docs/sphinx/",
|
||||
"use_repository_button": True,
|
||||
}
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['_static']
|
||||
|
||||
# Custom sidebar templates, must be a dictionary that maps document names
|
||||
# to template names.
|
||||
#
|
||||
# The default sidebars (for documents that don't match any pattern) are
|
||||
# defined by theme itself. Builtin themes are using these templates by
|
||||
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
|
||||
# 'searchbox.html']``.
|
||||
#
|
||||
html_sidebars = { '**': ['globaltoc.html', 'relations.html', 'searchbox.html'], }
|
||||
|
||||
|
||||
# -- Options for HTMLHelp output ---------------------------------------------
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'codondoc'
|
||||
|
||||
|
||||
# -- Options for LaTeX output ------------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#
|
||||
# 'papersize': 'letterpaper',
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#
|
||||
# 'pointsize': '10pt',
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#
|
||||
# 'preamble': '',
|
||||
|
||||
# Latex figure (float) alignment
|
||||
#
|
||||
# 'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(master_doc, 'codon.tex', u'Codon Documentation',
|
||||
u'arshajii', 'manual'),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for manual page output ------------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [
|
||||
(master_doc, 'codon', u'Codon Documentation',
|
||||
[author], 1)
|
||||
]
|
||||
|
||||
|
||||
# -- Options for Texinfo output ----------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(master_doc, 'codon', u'Codon Documentation',
|
||||
author, 'codon', 'a high-performance language',
|
||||
'Miscellaneous'),
|
||||
]
|
||||
|
||||
|
||||
# -- Extension configuration -------------------------------------------------
|
||||
breathe_projects = {
|
||||
'Codon': '../doxygen/xml'
|
||||
}
|
||||
|
||||
breathe_default_project = 'Codon'
|
||||
|
||||
exhale_args = {
|
||||
# These arguments are required
|
||||
'containmentFolder': 'api',
|
||||
'rootFileName': 'doxygen.rst',
|
||||
'rootFileTitle': 'Compiler API',
|
||||
'doxygenStripFromPath': '../..',
|
||||
# Suggested optional arguments
|
||||
'createTreeView': True,
|
||||
# TIP: if using the sphinx-bootstrap-theme, you need
|
||||
# 'treeViewIsBootstrap': True,
|
||||
'exhaleExecutesDoxygen': True,
|
||||
'exhaleDoxygenStdin': 'INPUT = ../../compiler/include ../../compiler/lang ../../compiler/types ../../compiler/util',
|
||||
}
|
||||
|
||||
# Tell sphinx what the primary language being documented is.
|
||||
primary_domain = 'cpp'
|
||||
|
||||
# Tell sphinx what the pygments highlight language should be.
|
||||
highlight_language = 'cpp'
|
||||
|
||||
|
||||
# -- Options for todo extension ----------------------------------------------
|
||||
|
||||
# If true, `todo` and `todoList` produce output, else they produce nothing.
|
||||
# todo_include_todos = True
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
*
|
||||
*/
|
||||
!.gitignore
|
||||
|
|
@ -1,59 +0,0 @@
|
|||
Calling Codon from C/C++
|
||||
========================
|
||||
|
||||
Calling C/C++ from Codon is quite easy with ``from C import``, but Codon can also be called from C/C++ code. To make a Codon function externally visible, simply annotate it with ``@export``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@export
|
||||
def foo(n: int):
|
||||
for i in range(n):
|
||||
print(i * i)
|
||||
return n * n
|
||||
|
||||
Note that only top-level, non-generic functions can be exported. Now we can create a shared library containing ``foo`` (assuming source file *foo.codon*):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
codon build -o foo.o foo.codon
|
||||
gcc -shared -lcodonrt -lomp foo.o -o libfoo.so
|
||||
|
||||
(The last command might require an additional ``-L/path/to/codonrt/lib/`` argument if ``libcodonrt`` is not installed on a standard path.)
|
||||
|
||||
Now we can call ``foo`` from a C program:
|
||||
|
||||
.. code-block:: C
|
||||
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
|
||||
int64_t foo(int64_t);
|
||||
|
||||
int main() {
|
||||
printf("%llu\n", foo(10));
|
||||
}
|
||||
|
||||
Compile:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
gcc -o foo -L. -lfoo foo.c
|
||||
|
||||
Now running ``./foo`` should invoke ``foo()`` as defined in Codon, with an argument of ``10``.
|
||||
|
||||
Converting types
|
||||
----------------
|
||||
|
||||
The following table shows the conversions between Codon and C/C++ types:
|
||||
|
||||
============ ============
|
||||
**Codon** **C/C++**
|
||||
------------ ------------
|
||||
``int`` ``long`` or ``int64_t``
|
||||
``float`` ``double``
|
||||
``bool`` ``bool``
|
||||
``byte`` ``char`` or ``int8_t``
|
||||
``str`` ``{int64_t, char*}`` (length and data)
|
||||
``class`` Pointer to corresponding tuple
|
||||
``@tuple`` Struct of fields
|
||||
============ ============
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
**Codon** - a high-performance Python compiler
|
||||
==============================================
|
||||
|
||||
What is Codon?
|
||||
--------------
|
||||
|
||||
Codon is a high-performance Python compiler that compiles Python code to native machine code without any runtime overhead.
|
||||
The Codon framework grew out of the `Seq <https://seq-lang.org>`_ project; while Seq focuses on genomics and bioinformatics, Codon
|
||||
can be applied in a number of different areas and is extensible via a plugin system.
|
||||
|
||||
Typical speedups over Python are on the order of 10-100x or more, on a single thread. Codon supports native multithreading which can lead
|
||||
to speedups many times higher still.
|
||||
|
||||
What isn't Codon?
|
||||
-----------------
|
||||
|
||||
While Codon supports nearly all of Python's syntax, it is not a drop-in replacement, and large codebases might require modifications
|
||||
to be run through the Codon compiler. For example, some of Python's modules are not yet implemented within Codon, and a few of Python's
|
||||
dynamic features are disallowed. The Codon compiler produces detailed error messages to help identify and resolve any incompatibilities.
|
||||
|
||||
Questions, comments or suggestions? Visit our `Discord server <https://discord.com/invite/8aKr6HEN?utm_source=Discord%20Widget&utm_medium=Connect>`_.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
intro
|
||||
primer
|
||||
parallel
|
||||
pipelines
|
||||
python
|
||||
embed
|
||||
build
|
||||
stdlib/index
|
||||
|
||||
|
||||
Frequently Asked Questions
|
||||
--------------------------
|
||||
|
||||
*What is the goal of Codon?*
|
||||
|
||||
One of the main focuses of Codon is to bridge the gap between usability and performance. Codon aims to make writing high-performance software
|
||||
substantially easier, and to provide a common, unified framework for the development of such software across a range of domains.
|
||||
|
||||
*How does Codon compare to other Python implementations?*
|
||||
|
||||
Unlike other performance-oriented Python implementations, such as PyPy or Numba, Codon is a standalone system implemented entirely independently
|
||||
of regular Python. Since it does not need to interoperate with CPython's runtime, Codon has far greater flexibility to generate optimized code.
|
||||
In fact, Codon will frequently generate the same code as that from an equivalent C or C++ program. This design choice also allows Codon to circumvent
|
||||
issues like Python's global interpretter lock, and thereby to take full advantage of parallelism and multithreading.
|
||||
|
||||
*What about interoperability with other languages and frameworks?*
|
||||
|
||||
Interoperability is and will continue to be a priority for the Codon project. We don't want using Codon to render you unable to use all the other great
|
||||
frameworks and libraries that exist. Codon already supports interoperability with C/C++ and Python (see :ref:`interop`).
|
||||
|
||||
*I want to contribute! How do I get started?*
|
||||
|
||||
Great! Check out our `contribution guidelines <https://github.com/exaloop/codon/blob/master/CONTRIBUTING.md>`_ and `open issues <https://github.com/exaloop/codon/issues>`_
|
||||
to get started. Also don't hesitate to drop by our `Discord server <https://discord.com/invite/8aKr6HEN?utm_source=Discord%20Widget&utm_medium=Connect>`_
|
||||
if you have any questions.
|
||||
|
||||
*What is planned for the future?*
|
||||
|
||||
See the `roadmap <https://github.com/exaloop/codon/wiki/Roadmap>`_ for information about this.
|
||||
|
|
@ -1,77 +0,0 @@
|
|||
Getting Started
|
||||
===============
|
||||
|
||||
Install
|
||||
-------
|
||||
|
||||
Pre-built binaries
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Pre-built binaries for Linux and macOS on x86_64 are available alongside `each release <https://github.com/exaloop/codon/releases>`_. We also have a script for downloading and installing pre-built versions:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
/bin/bash -c "$(curl -fsSL https://codon.dev/install.sh)"
|
||||
|
||||
This will install Codon in a new ``.codon`` directory within your home directory.
|
||||
|
||||
Building from source
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
See `Building from Source <build.html>`_.
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
The ``codon`` program can either directly ``run`` Codon source in JIT mode:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
codon run myprogram.codon
|
||||
|
||||
The default compilation and run mode is *debug* (``-debug``). Compile and run with optimizations with the ``-release`` option:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
codon run -release myprogram.codon
|
||||
|
||||
``codon`` can also ``build`` executables (ensure you have ``clang`` installed, as it is used for linking):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# generate 'myprogram' executable
|
||||
codon build -exe myprogram.codon
|
||||
|
||||
# generate 'foo' executable
|
||||
codon build -o foo myprogram.codon
|
||||
|
||||
``codon`` can produce object files:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# generate 'myprogram.o' object file
|
||||
codon build -obj myprogram.codon
|
||||
|
||||
# generate 'foo.o' object file
|
||||
codon build -o foo.o myprogram.codon
|
||||
|
||||
``codon`` can produce LLVM IR:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# generate 'myprogram.ll' object file
|
||||
codon build -llvm myprogram.codon
|
||||
|
||||
# generate 'foo.ll' object file
|
||||
codon build -o foo.ll myprogram.codon
|
||||
|
||||
Compile-time definitions
|
||||
------------------------
|
||||
|
||||
``codon`` allows for compile-time definitions via the ``-D`` flag. For example, in the following code:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
print(Int[BIT_WIDTH]())
|
||||
|
||||
``BIT_WIDTH`` can be specified on the command line as such: ``codon run -DBIT_WIDTH=10 myprogram.codon``.
|
||||
|
|
@ -1,36 +0,0 @@
|
|||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=.
|
||||
set BUILDDIR=_build
|
||||
set SPHINXPROJ=codon
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.http://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
|
||||
|
||||
:end
|
||||
popd
|
||||
|
|
@ -1,174 +0,0 @@
|
|||
.. _parallelism:
|
||||
|
||||
Parallelism and Multithreading
|
||||
==============================
|
||||
|
||||
Codon supports parallelism and multithreading via OpenMP out of the box. Here's an example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@par
|
||||
for i in range(10):
|
||||
import threading as thr
|
||||
print('hello from thread', thr.get_ident())
|
||||
|
||||
By default, parallel loops will use all available threads, or use the number of threads
|
||||
specified by the ``OMP_NUM_THREADS`` environment variable. A specific thread number can
|
||||
be given directly on the ``@par`` line as well:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@par(num_threads=5)
|
||||
for i in range(10):
|
||||
import threading as thr
|
||||
print('hello from thread', thr.get_ident())
|
||||
|
||||
``@par`` supports several OpenMP parameters, including:
|
||||
|
||||
- ``num_threads`` (int): the number of threads to use when running the loop
|
||||
- ``schedule`` (str): either *static*, *dynamic*, *guided*, *auto* or *runtime*
|
||||
- ``chunk_size`` (int): chunk size when partitioning loop iterations
|
||||
- ``ordered`` (bool): whether the loop iterations should be executed in the same order
|
||||
|
||||
Other OpenMP parameters like ``private``, ``shared`` or ``reduction``, are inferred
|
||||
automatically by the compiler. For example, the following loop
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
total = 0
|
||||
@par
|
||||
for i in range(N):
|
||||
a += foo(i)
|
||||
|
||||
will automatically generate a reduction for variable ``a``.
|
||||
|
||||
Here is an example that finds the sum of prime numbers up to a user-defined limit, using
|
||||
a parallel loop on 16 threads with a dynamic schedule and chunk size of 100:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from sys import argv
|
||||
|
||||
def is_prime(n):
|
||||
factors = 0
|
||||
for i in range(2, n):
|
||||
if n % i == 0:
|
||||
factors += 1
|
||||
return factors == 0
|
||||
|
||||
limit = int(argv[1])
|
||||
total = 0
|
||||
|
||||
@par(schedule='dynamic', chunk_size=100, num_threads=16)
|
||||
for i in range(2, limit):
|
||||
if is_prime(i):
|
||||
total += 1
|
||||
|
||||
print(total)
|
||||
|
||||
Static schedules work best when each loop iteration takes roughly the same amount of time,
|
||||
whereas dynamic schedules are superior when each iteration varies in duration. Since counting
|
||||
the factors of an integer takes more time for larger integers, we use a dynamic schedule here.
|
||||
|
||||
``@par`` also supports C/C++ OpenMP pragma strings. For example, the ``@par`` line in the
|
||||
above example can also be written as:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# same as: @par(schedule='dynamic', chunk_size=100, num_threads=16)
|
||||
@par('schedule(dynamic, 100) num_threads(16)')
|
||||
|
||||
Different kinds of loops
|
||||
------------------------
|
||||
|
||||
``for``-loops can iterate over arbitrary generators, but OpenMP's parallel loop construct only
|
||||
applies to *imperative* for-loops of the form ``for i in range(a, b, c)`` (where ``c`` is constant).
|
||||
For general parallel for-loops of the form ``for i in some_generator()``, a task-based approach is
|
||||
used instead, where each loop iteration is executed as an independent task.
|
||||
|
||||
The Codon compiler also converts iterations over lists (``for a in some_list``) to imperative
|
||||
for-loops, meaning these loops can be executed using OpenMP's loop parallelism.
|
||||
|
||||
Custom reductions
|
||||
-----------------
|
||||
|
||||
Codon can automatically generate efficient reductions for ``int`` and ``float`` values. For other
|
||||
data types, user-defined reductions can be specified. A class that supports reductions must
|
||||
include:
|
||||
|
||||
- A default constructor that represents the *zero value*
|
||||
- An ``__add__`` method (assuming ``+`` is used as the reduction operator)
|
||||
|
||||
Here is an example for reducing a new ``Vector`` type:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@tuple
|
||||
class Vector:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __new__():
|
||||
return Vector(0, 0)
|
||||
|
||||
def __add__(self, other: Vector):
|
||||
return Vector(self.x + other.x, self.y + other.y)
|
||||
|
||||
v = Vector()
|
||||
@par
|
||||
for i in range(100):
|
||||
v += Vector(i,i)
|
||||
print(v) # (x: 4950, y: 4950)
|
||||
|
||||
OpenMP constructs
|
||||
-----------------
|
||||
|
||||
All of OpenMP's API functions are accessible directly in Codon. For example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import openmp as omp
|
||||
print(omp.get_num_threads())
|
||||
omp.set_num_threads(32)
|
||||
|
||||
OpenMP's *critical*, *master*, *single* and *ordered* constructs can be applied via the
|
||||
corresponding decorators:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import openmp as omp
|
||||
|
||||
@omp.critical
|
||||
def only_run_by_one_thread_at_a_time():
|
||||
print('critical!', omp.get_thread_num())
|
||||
|
||||
@omp.master
|
||||
def only_run_by_master_thread():
|
||||
print('master!', omp.get_thread_num())
|
||||
|
||||
@omp.single
|
||||
def only_run_by_single_thread():
|
||||
print('single!', omp.get_thread_num())
|
||||
|
||||
@omp.ordered
|
||||
def run_ordered_by_iteration(i):
|
||||
print('ordered!', i)
|
||||
|
||||
@par(ordered=True)
|
||||
for i in range(100):
|
||||
only_run_by_one_thread_at_a_time()
|
||||
only_run_by_master_thread()
|
||||
only_run_by_single_thread()
|
||||
run_ordered_by_iteration(i)
|
||||
|
||||
For finer-grained locking, consider using the locks from the ``threading`` module:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from threading import Lock
|
||||
lock = Lock() # or RLock for re-entrant lock
|
||||
|
||||
@par
|
||||
for i in range(100):
|
||||
with lock:
|
||||
print('only one thread at a time allowed here')
|
||||
|
|
@ -1,63 +0,0 @@
|
|||
Pipelines
|
||||
=========
|
||||
|
||||
Codon extends the core Python language with a pipe operator. You can chain multiple functions and generators to form a pipeline.
|
||||
Pipeline stages can be regular functions or generators. In the case of standard functions, the function is simply applied to the
|
||||
input data and the result is carried to the remainder of the pipeline, akin to F#'s functional piping. If, on the other hand, a
|
||||
stage is a generator, the values yielded by the generator are passed lazily to the remainder of the pipeline, which in many ways
|
||||
mirrors how piping is implemented in Bash. Note that Codon ensures that generator pipelines do not collect any data unless explicitly
|
||||
requested, thus allowing the processing of terabytes of data in a streaming fashion with no memory and minimal CPU overhead.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def add1(x):
|
||||
return x + 1
|
||||
|
||||
2 |> add1 # 3; equivalent to add1(2)
|
||||
|
||||
def calc(x, y):
|
||||
return x + y**2
|
||||
2 |> calc(3) # 11; equivalent to calc(2, 3)
|
||||
2 |> calc(..., 3) # 11; equivalent to calc(2, 3)
|
||||
2 |> calc(3, ...) # 7; equivalent to calc(3, 2)
|
||||
|
||||
def gen(i):
|
||||
for i in range(i):
|
||||
yield i
|
||||
|
||||
5 |> gen |> print # prints 0 1 2 3 4 separated by newline
|
||||
range(1, 4) |> iter |> gen |> print(end=' ') # prints 0 0 1 0 1 2 without newline
|
||||
[1, 2, 3] |> print # prints [1, 2, 3]
|
||||
range(100000000) |> print # prints range(0, 100000000)
|
||||
range(100000000) |> iter |> print # not only prints all those numbers, but it uses almost no memory at all
|
||||
|
||||
Codon will chain anything that implements ``__iter__``, and the compiler will optimize out generators whenever possible. Combinations
|
||||
of pipes and generators can be used to implement efficient streaming pipelines.
|
||||
|
||||
.. caution::
|
||||
The Codon compiler may perform optimizations that change the order of elements passed through a pipeline.
|
||||
Therefore, it is best to not rely on order when using pipelines. If order needs to be maintained, consider
|
||||
using a regular loop or passing an index alongside each element sent through the pipeline.
|
||||
|
||||
Parallel pipelines
|
||||
------------------
|
||||
|
||||
CPython and many other implementations alike cannot take advantage of parallelism due to the infamous global interpreter lock, a mutex
|
||||
that prevents multiple threads from executing Python bytecode at once. Unlike CPython, Codon has no such restriction and supports full
|
||||
multithreading. To this end, Codon supports a *parallel* pipe operator ``||>``, which is semantically similar to the standard pipe
|
||||
operator except that it allows the elements sent through it to be processed in parallel by the remainder of the pipeline. Hence, turning
|
||||
a serial program into a parallel one often requires the addition of just a single character in Codon. Further, a single pipeline can
|
||||
contain multiple parallel pipes, resulting in nested parallelism.
|
||||
|
||||
.. code:: python
|
||||
|
||||
range(100000) |> iter ||> print # prints all these numbers, probably in random order
|
||||
range(100000) |> iter ||> process ||> clean # runs process in parallel, and then cleans data in parallel
|
||||
|
||||
Codon will automatically schedule the ``process`` and ``clean`` functions to execute as soon as possible. You can control the number of
|
||||
threads via the ``OMP_NUM_THREADS`` environment variable.
|
||||
|
||||
Internally, the Codon compiler uses an OpenMP task backend to generate code for parallel pipelines. Logically, parallel pipe operators are
|
||||
similar to parallel-for loops: the portion of the pipeline after the parallel pipe is outlined into a new function that is called by the
|
||||
OpenMP runtime task spawning routines (as in ``#pragma omp task`` in C++), and a synchronization point (``#pragma omp taskwait``) is added
|
||||
after the outlined segment.
|
||||
|
|
@ -1,893 +0,0 @@
|
|||
Language Primer
|
||||
===============
|
||||
|
||||
If you know Python, you already know 99% of Codon. The following primer
|
||||
assumes some familiarity with Python or at least one "modern"
|
||||
programming language (QBASIC doesn't count).
|
||||
|
||||
Printing
|
||||
--------
|
||||
|
||||
.. code:: python
|
||||
|
||||
print('hello world')
|
||||
|
||||
from sys import stderr
|
||||
print('hello world', end='', file=stderr)
|
||||
|
||||
Comments
|
||||
--------
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Codon comments start with "# 'and go until the end of the line
|
||||
|
||||
"""
|
||||
There are no multi-line comments. You can (ab)use the docstring operator (''')
|
||||
if you really need them.
|
||||
"""
|
||||
|
||||
Literals
|
||||
--------
|
||||
|
||||
Codon is a strongly typed language like C++, Java, or C#. That means each
|
||||
expression must have a type that can be inferred at the compile-time.
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Booleans
|
||||
True # type: bool
|
||||
False
|
||||
|
||||
# Numbers
|
||||
a = 1 # type: int. It's 64-bit signed integer.
|
||||
b = 1.12 # type: float. Codon's float is identical to C's double.
|
||||
c = 5u # type: int, but unsigned
|
||||
d = Int[8](12) # 8-bit signed integer; you can go all the way to Int[2048]
|
||||
e = UInt[8](200) # 8-bit unsigned integer
|
||||
f = byte(3) # a byte is C's char; equivalent to Int[8]
|
||||
|
||||
h = 0x12AF # hexadecimal integers are also welcome
|
||||
h = 0XAF12
|
||||
g = 3.11e+9 # scientific notation is also supported
|
||||
g = .223 # and this is also float
|
||||
g = .11E-1 # and this as well
|
||||
|
||||
# Strings
|
||||
s = 'hello! "^_^" ' # type: str.
|
||||
t = "hello there! \t \\ '^_^' " # \t is a tab character; \\ stands for \
|
||||
raw = r"hello\n" # raw strings do not escape slashes; this would print "hello\n"
|
||||
fstr = f"a is {a + 1}" # an F-string; prints "a is 2"
|
||||
fstr = f"hi! {a+1=}" # an F-string; prints "hi! a+1=2"
|
||||
t = """
|
||||
hello!
|
||||
multiline string
|
||||
"""
|
||||
|
||||
# The following escape sequences are supported:
|
||||
# \\, \', \", \a, \b, \f, \n, \r, \t, \v,
|
||||
# \xHHH (HHH is hex code), \OOO (OOO is octal code)
|
||||
|
||||
Tuples
|
||||
~~~~~~
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Tuples
|
||||
t = (1, 2.3, 'hi') # type: Tuple[int, float, str].
|
||||
t[1] # type: float
|
||||
u = (1, ) # type: Tuple[int]
|
||||
|
||||
As all types must be known at compile time, tuple indexing works
|
||||
only if a tuple is homogenous (all types are the same) or if the value
|
||||
of the index is known at compile-time.
|
||||
|
||||
You can, however, iterate over heterogenous tuples in Codon. This is achieved
|
||||
by unrolling the loop to accommodate the different types.
|
||||
|
||||
.. code:: python
|
||||
|
||||
t = (1, 2.3, 'hi')
|
||||
t[1] # works because 1 is a constant int
|
||||
|
||||
x = int(argv[1])
|
||||
t[x] # compile error: x is not known at compile time
|
||||
|
||||
# This is a homogenous tuple (all member types are the same)
|
||||
u = (1, 2, 3) # type: Tuple[int, int, int].
|
||||
u[x] # works because tuple members share the same type regardless of x
|
||||
for i in u: # works
|
||||
print(i)
|
||||
|
||||
# Also works
|
||||
v = (42, 'x', 3.14)
|
||||
for i in v:
|
||||
print(i)
|
||||
|
||||
.. note::
|
||||
Tuples are **immutable**. ``a = (1, 2); a[1] = 1`` will not
|
||||
compile.
|
||||
|
||||
Containers
|
||||
~~~~~~~~~~
|
||||
|
||||
.. code:: python
|
||||
|
||||
l = [1, 2, 3] # type: List[int]; a list of integers
|
||||
s = {1.1, 3.3, 2.2, 3.3} # type: Set[float]; a set of floats
|
||||
d = {1: 'hi', 2: 'ola', 3: 'zdravo'} # type: Dict[int, str]; a simple dictionary
|
||||
|
||||
ln = [] # an empty list whose type is inferred based on usage
|
||||
ln = List[int]() # an empty list with explicit element types
|
||||
dn = {} # an empty dict whose type is inferred based on usage
|
||||
dn = Dict[int, float]() # an empty dictionary with explicit element types
|
||||
|
||||
Because Codon is strongly typed, these won't compile:
|
||||
|
||||
.. code:: python
|
||||
|
||||
l = [1, 's'] # is it a List[int] or List[str]? you cannot mix-and-match types
|
||||
d = {1: 'hi'}
|
||||
d[2] = 3 # d is a Dict[int, str]; the assigned value must be a str
|
||||
|
||||
t = (1, 2.2) # Tuple[int, float]
|
||||
lt = list(t) # compile error: t is not homogenous
|
||||
|
||||
lp = [1, 2.1, 3, 5] # compile error: Codon will not automatically cast a float to an int
|
||||
|
||||
This will work, though:
|
||||
|
||||
.. code:: python
|
||||
|
||||
u = (1, 2, 3)
|
||||
lu = list(u) # works: u is homogenous
|
||||
|
||||
.. note::
|
||||
Dictionaries and sets are unordered and are based on
|
||||
`klib <https://github.com/attractivechaos/klib>`__.
|
||||
|
||||
.. _operators:
|
||||
|
||||
Assignments and operators
|
||||
-------------------------
|
||||
|
||||
.. code:: python
|
||||
|
||||
a = 1 + 2 # this is 3
|
||||
a = (1).__add__(2) # you can use a function call instead of an operator; this is also 3
|
||||
a = int.__add__(1, 2) # this is equivalent to the previous line
|
||||
b = 5 / 2.0 # this is 2.5
|
||||
c = 5 // 2 # this is 2; // is an integer division
|
||||
a *= 2 # a is now 6
|
||||
|
||||
This is the list of binary operators and their magic methods:
|
||||
|
||||
======== ================ ==================================================
|
||||
Operator Magic method Description
|
||||
======== ================ ==================================================
|
||||
``+`` ``__add__`` addition
|
||||
``-`` ``__sub__`` subtraction
|
||||
``*`` ``__mul__`` multiplication
|
||||
``/`` ``__truediv__`` float (true) division
|
||||
``//`` ``__floordiv__`` integer (floor) division
|
||||
``**`` ``__pow__`` exponentiation
|
||||
``%`` ``__mod__`` modulo
|
||||
``@`` ``__matmul__`` matrix multiplication;
|
||||
``&`` ``__and__`` bitwise and
|
||||
``|`` ``__or__`` bitwise or
|
||||
``^`` ``__xor__`` bitwise xor
|
||||
``<<`` ``__lshift__`` left bit shift
|
||||
``>>`` ``__rshift__`` right bit shift
|
||||
``<`` ``__lt__`` less than
|
||||
``<=`` ``__le__`` less or equal than
|
||||
``>`` ``__gt__`` greater than
|
||||
``>=`` ``__ge__`` greater or equal than
|
||||
``==`` ``__eq__`` equal to
|
||||
``!=`` ``__ne__`` not equal to
|
||||
``in`` ``__contains__`` belongs to
|
||||
``and`` none boolean and (short-circuits)
|
||||
``or`` none boolean or (short-circuits)
|
||||
======== ================ ==================================================
|
||||
|
||||
Codon also has the following unary operators:
|
||||
|
||||
======== ================ =============================
|
||||
Operator Magic method Description
|
||||
======== ================ =============================
|
||||
``~`` ``__invert__`` bitwise inversion;
|
||||
reverse complement;
|
||||
``Optional[T]`` unpacking
|
||||
``+`` ``__pos__`` unary positive
|
||||
``-`` ``__neg__`` unary negation
|
||||
``not`` none boolean negation
|
||||
======== ================ =============================
|
||||
|
||||
Tuple unpacking
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
Codon supports most of Python's tuple unpacking syntax:
|
||||
|
||||
.. code:: python
|
||||
|
||||
x, y = 1, 2 # x is 1, y is 2
|
||||
(x, (y, z)) = 1, (2, 3) # x is 1, y is 2, z is 3
|
||||
[x, (y, z)] = (1, [2, 3]) # x is 1, y is 2, z is 3
|
||||
|
||||
l = range(1, 8) # l is [1, 2, 3, 4, 5, 6, 7]
|
||||
a, b, *mid, c = l # a is 1, b is 2, mid is [3, 4, 5, 6], c is 7
|
||||
a, *end = l # a is 1, end is [2, 3, 4, 5, 6, 7]
|
||||
*beg, c = l # c is 7, beg is [1, 2, 3, 4, 5, 6]
|
||||
(*x, ) = range(3) # x is [0, 1, 2]
|
||||
*x = range(3) # error: this does not work
|
||||
|
||||
*sth, a, b = (1, 2, 3, 4) # sth is (1, 2), a is 3, b is 4
|
||||
*sth, a, b = (1.1, 2, 3.3, 4) # error: this only works on homogenous tuples for now
|
||||
|
||||
(x, y), *pff, z = [1, 2], 'this'
|
||||
print(x, y, pff, z) # x is 1, y is 2, pff is an empty tuple --- () ---, and z is "this"
|
||||
|
||||
s, *q = 'XYZ' # works on strings as well; s is "X" and q is "YZ"
|
||||
|
||||
Control flow
|
||||
------------
|
||||
|
||||
Conditionals
|
||||
~~~~~~~~~~~~
|
||||
|
||||
Codon supports the standard Python conditional syntax:
|
||||
|
||||
.. code:: python
|
||||
|
||||
if a or b or some_cond():
|
||||
print(1)
|
||||
elif whatever() or 1 < a <= b < c < 4: # chained comparisons are supported
|
||||
print('meh...')
|
||||
else:
|
||||
print('lo and behold!')
|
||||
|
||||
if x: y()
|
||||
|
||||
a = b if sth() else c # ternary conditional operator
|
||||
|
||||
Codon extends the Python conditional syntax with a ``match`` statement, which is
|
||||
inspired by Rust's:
|
||||
|
||||
.. code:: python
|
||||
|
||||
match a + some_heavy_expr(): # assuming that the type of this expression is int
|
||||
case 1: # is it 1?
|
||||
print('hi')
|
||||
case 2 ... 10: # is it 2, 3, 4, 5, 6, 7, 8, 9 or 10?
|
||||
print('wow!')
|
||||
case _: # "default" case
|
||||
print('meh...')
|
||||
|
||||
match bool_expr(): # now it's a bool expression
|
||||
case True: ...
|
||||
case False: ...
|
||||
|
||||
match str_expr(): # now it's a str expression
|
||||
case 'abc': print("it's ABC time!")
|
||||
case 'def' | 'ghi': # you can chain multiple rules with the "|" operator
|
||||
print("it's not ABC time!")
|
||||
case s if len(s) > 10: print("so looong!") # conditional match expression
|
||||
case _: assert False
|
||||
|
||||
match some_tuple: # assuming type of some_tuple is Tuple[int, int]
|
||||
case (1, 2): ...
|
||||
case (a, _) if a == 42: # you can do away with useless terms with an underscore
|
||||
print('hitchhiker!')
|
||||
case (a, 50 ... 100) | (10 ... 20, b): # you can nest match expressions
|
||||
print('complex!')
|
||||
|
||||
match list_foo():
|
||||
case []: # [] matches an empty list
|
||||
...
|
||||
case [1, 2, 3]: # make sure that list_foo() returns List[int] though!
|
||||
...
|
||||
case [1, 2, ..., 5]: # matches any list that starts with 1 and 2 and ends with 5
|
||||
...
|
||||
case [..., 6] | [6, ...]: # matches a list that starts or ends with 6
|
||||
...
|
||||
case [..., w] if w < 0: # matches a list that ends with a negative integer
|
||||
...
|
||||
case [...]: # any other list
|
||||
...
|
||||
|
||||
You can mix, match and chain match rules as long as the match type
|
||||
matches the expression type.
|
||||
|
||||
Loops
|
||||
~~~~~
|
||||
|
||||
Standard fare:
|
||||
|
||||
.. code:: python
|
||||
|
||||
a = 10
|
||||
while a > 0: # prints even numbers from 9 to 1
|
||||
a -= 1
|
||||
if a % 2 == 1:
|
||||
continue
|
||||
print(a)
|
||||
|
||||
for i in range(10): # prints numbers from 0 to 7, inclusive
|
||||
print(i)
|
||||
if i > 6: break
|
||||
|
||||
``for`` construct can iterate over any generator, which means any object
|
||||
that implements the ``__iter__`` magic method. In practice, generators,
|
||||
lists, sets, dictionaries, homogenous tuples, ranges, and many more types
|
||||
implement this method, so you don't need to worry. If you need to
|
||||
implement one yourself, just keep in mind that ``__iter__`` is a
|
||||
generator and not a function.
|
||||
|
||||
Comprehensions
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
Technically, comprehensions are not statements (they are expressions).
|
||||
Comprehensions are a nifty, Pythonic way to create collections:
|
||||
|
||||
.. code:: python
|
||||
|
||||
l = [i for i in range(5)] # type: List[int]; l is [0, 1, 2, 3, 4]
|
||||
l = [i for i in range(15) if i % 2 == 1 if i > 10] # type: List[int]; l is [11, 13]
|
||||
l = [i * j for i in range(5) for j in range(5) if i == j] # l is [0, 1, 4, 9, 16]
|
||||
|
||||
s = {abs(i - j) for i in range(5) for j in range(5)} # s is {0, 1, 2, 3, 4}
|
||||
d = {i: f'item {i+1}' for i in range(3)} # d is {0: "item 1", 1: "item 2", 2: "item 3"}
|
||||
|
||||
You can also use collections to create generators (more about them later
|
||||
on):
|
||||
|
||||
.. code:: python
|
||||
|
||||
g = (i for i in range(10))
|
||||
print(list(g)) # prints number from 0 to 9, inclusive
|
||||
|
||||
Exception handling
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Again, if you know how to do this in Python, you know how to do it in
|
||||
Codon:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def throwable():
|
||||
raise ValueError("doom and gloom")
|
||||
|
||||
try:
|
||||
throwable()
|
||||
except ValueError as e:
|
||||
print("we caught the exception")
|
||||
except:
|
||||
print("ouch, we're in deep trouble")
|
||||
finally:
|
||||
print("whatever, it's done")
|
||||
|
||||
.. note::
|
||||
Right now, Codon cannot catch multiple exceptions in one
|
||||
statement. Thus ``catch (Exc1, Exc2, Exc3) as var`` will not compile.
|
||||
|
||||
If you have an object that implements ``__enter__`` and ``__exit__``
|
||||
methods to manage its lifetime (say, a ``File``), you can use a ``with``
|
||||
statement to make your life easy:
|
||||
|
||||
.. code:: python
|
||||
|
||||
with open('foo.txt') as f, open('foo_copy.txt', 'w') as fo:
|
||||
for l in f:
|
||||
fo.write(l)
|
||||
|
||||
Variables and scoping
|
||||
---------------------
|
||||
|
||||
You have probably noticed by now that blocks in Codon are defined by their
|
||||
indentation level (as in Python). We recommend using 2 or 4 spaces to
|
||||
indent blocks. Do not mix indentation levels, and do not mix tabs and spaces;
|
||||
stick to any *consistent* way of indenting your code.
|
||||
|
||||
One of the major differences between Codon and Python lies in variable
|
||||
scoping rules. Codon variables cannot *leak* to outer blocks. Every
|
||||
variable is accessible only within its own block (after the variable is
|
||||
defined, of course), and within any block that is nested within the
|
||||
variable's own block.
|
||||
|
||||
That means that the following Pythonic pattern won't compile:
|
||||
|
||||
.. code:: python
|
||||
|
||||
if cond():
|
||||
x = 1
|
||||
else:
|
||||
x = 2
|
||||
print(x) # x is defined separately in if/else blocks; it is not accessible here!
|
||||
|
||||
for i in range(10):
|
||||
...
|
||||
print(i) # error: i is only accessible within the for loop block
|
||||
|
||||
In Codon, you can rewrite that as:
|
||||
|
||||
.. code:: python
|
||||
|
||||
x = 2
|
||||
if cond():
|
||||
x = 1
|
||||
|
||||
# or even better
|
||||
x = 1 if cond() else 2
|
||||
|
||||
print(x)
|
||||
|
||||
Another important difference between Codon and Python is that, in Codon, variables
|
||||
cannot change types. So this won't compile:
|
||||
|
||||
.. code:: python
|
||||
|
||||
a = 's'
|
||||
a = 1 # error: expected string, but got int
|
||||
|
||||
A note about function scoping: functions cannot modify variables that
|
||||
are not defined within the function block. You need to use ``global`` to
|
||||
modify such variables:
|
||||
|
||||
.. code:: python
|
||||
|
||||
g = 5
|
||||
def foo():
|
||||
print(g)
|
||||
foo() # works, prints 5
|
||||
|
||||
def foo2():
|
||||
g += 2 # error: cannot access g
|
||||
print(g)
|
||||
|
||||
def foo3():
|
||||
global g
|
||||
g += 2
|
||||
print(g)
|
||||
foo3() # works, prints 7
|
||||
foo3() # works, prints 9
|
||||
|
||||
As a rule, use ``global`` whenever you need to access variables that
|
||||
are not defined within the function.
|
||||
|
||||
Imports
|
||||
-------
|
||||
|
||||
You can import functions and classes from another Codon module by doing:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# Create foo.codon with a bunch of useful methods
|
||||
import foo
|
||||
|
||||
foo.useful1()
|
||||
p = foo.FooType()
|
||||
|
||||
# Create bar.codon with a bunch of useful methods
|
||||
from bar import x, y
|
||||
x(y)
|
||||
|
||||
from bar import z as bar_z
|
||||
bar_z()
|
||||
|
||||
``import foo`` looks for ``foo.codon`` or ``foo/__init__.codon`` in the
|
||||
current directory.
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
Functions are defined as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def foo(a, b, c):
|
||||
return a + b + c
|
||||
print(foo(1, 2, 3)) # prints 6
|
||||
|
||||
How about procedures? Well, don't return anything meaningful:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def proc(a, b):
|
||||
print(a, 'followed by', b)
|
||||
proc(1, 's')
|
||||
|
||||
def proc2(a, b):
|
||||
if a == 5:
|
||||
return
|
||||
print(a, 'followed by', b)
|
||||
proc2(1, 's')
|
||||
proc2(5, 's') # this prints nothing
|
||||
|
||||
Codon is a strongly-typed language, so you can restrict argument and
|
||||
return types:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def fn(a: int, b: float):
|
||||
return a + b # this works because int implements __add__(float)
|
||||
fn(1, 2.2) # 3.2
|
||||
fn(1.1, 2) # error: 1.1. is not an int
|
||||
|
||||
def fn2(a: int, b):
|
||||
return a - b
|
||||
fn2(1, 2) # -1
|
||||
fn2(1, 1.1) # -0.1; works because int implements __sub__(float)
|
||||
fn2(1, 's') # error: there is no int.__sub__(str)!
|
||||
|
||||
def fn3(a, b) -> int:
|
||||
return a + b
|
||||
fn3(1, 2) # works, as 1 + 2 is integer
|
||||
fn3('s', 'u') # error: 's'+'u' returns 'su' which is str,
|
||||
# but the signature indicates that it must return int
|
||||
|
||||
Default arguments? Named arguments? You bet:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def foo(a, b: int, c: float = 1.0, d: str = 'hi'):
|
||||
print(a, b, c, d)
|
||||
foo(1, 2) # prints "1 2 1 hi"
|
||||
foo(1, d='foo', b=1) # prints "1 1 1 foo"
|
||||
|
||||
How about optional arguments? We support that too:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# type of b promoted to Optional[int]
|
||||
def foo(a, b: int = None):
|
||||
print(a, b + 1)
|
||||
|
||||
foo(1, 2) # prints "1 3"
|
||||
foo(1) # raises ValueError, since b is None
|
||||
|
||||
Generics
|
||||
~~~~~~~~
|
||||
|
||||
As we've said several times: Codon is a strongly typed language. As
|
||||
such, it is not as flexible as Python when it comes to types (e.g. you
|
||||
can't have lists with elements of different types). However,
|
||||
Codon tries to mimic Python's *"I don't care about types until I do"*
|
||||
attitude as much as possible by utilizing a technique known as
|
||||
*monomorphization*. If there is a function that has an argument
|
||||
without a type definition, Codon will consider it a *generic* function,
|
||||
and will generate different functions for each invocation of
|
||||
that generic function:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def foo(x):
|
||||
print(x) # print relies on typeof(x).__str__(x) method to print the representation of x
|
||||
foo(1) # Codon automatically generates foo(x: int) and calls int.__str__ when needed
|
||||
foo('s') # Codon automatically generates foo(x: str) and calls str.__str__ when needed
|
||||
foo([1, 2]) # Codon automatically generates foo(x: List[int]) and calls List[int].__str__ when needed
|
||||
|
||||
But what if you need to mix type definitions and generic types? Say, your
|
||||
function can take a list of *anything*? Well, you can use generic
|
||||
specifiers:
|
||||
|
||||
.. code:: python
|
||||
|
||||
def foo(x: List[T], T: type):
|
||||
print(x)
|
||||
foo([1, 2]) # prints [1, 2]
|
||||
foo(['s', 'u']) # prints [s, u]
|
||||
foo(5) # error: 5 is not a list!
|
||||
foo(['s', 'u'], int) # fails: T is int, so foo expects List[int] but it got List[str]
|
||||
|
||||
def foo(x, R: type) -> R:
|
||||
print(x)
|
||||
return 1
|
||||
foo(4, int) # prints 4, returns 1
|
||||
foo(4, str) # error: return type is str, but foo returns int!
|
||||
|
||||
|
||||
.. note::
|
||||
Coming from C++? ``foo(x: List[T], T: type): ...`` is roughly the same as
|
||||
``template<typename T, typename U> U foo(T x) { ... }``.
|
||||
|
||||
Generators
|
||||
~~~~~~~~~~
|
||||
|
||||
Codon supports generators, and they are fast! Really, really fast!
|
||||
|
||||
.. code:: python
|
||||
|
||||
def gen(i):
|
||||
while i < 10:
|
||||
yield i
|
||||
i += 1
|
||||
print(list(gen(0))) # prints [0, 1, ..., 9]
|
||||
print(list(gen(10))) # prints []
|
||||
|
||||
You can also use ``yield`` to implement coroutines: ``yield``
|
||||
suspends the function, while ``(yield)`` (yes, parentheses are required)
|
||||
receives a value, as in Python.
|
||||
|
||||
.. code:: python
|
||||
|
||||
def mysum[T](start: T):
|
||||
m = start
|
||||
while True:
|
||||
a = (yield) # receives the input of coroutine.send() call
|
||||
if a == -1:
|
||||
break # exits the coroutine
|
||||
m += a
|
||||
yield m
|
||||
iadder = mysum(0) # assign a coroutine
|
||||
next(iadder) # activate it
|
||||
for i in range(10):
|
||||
iadder.send(i) # send a value to coroutine
|
||||
print(iadder.send(-1)) # prints 45
|
||||
|
||||
.. _interop:
|
||||
|
||||
Foreign function interface (FFI)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Codon can easily call functions from C and Python.
|
||||
|
||||
Let's import some C functions:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from C import pow(float, float) -> float
|
||||
pow(2.0, 2.0) # 4.0
|
||||
|
||||
# Import and rename function
|
||||
from C import puts(cobj) -> void as print_line # type cobj is C's pointer (void*, char*, etc.)
|
||||
print_line("hi!".c_str()) # prints "hi!".
|
||||
# Note .c_str() at the end of string--- needed to cast Codon's string to char*.
|
||||
|
||||
``from C import`` only works if the symbol is available to the program. If you
|
||||
are running your programs via ``codon``, you can link dynamic libraries
|
||||
by running ``codon run -l path/to/dynamic/library.so ...``.
|
||||
|
||||
Hate linking? You can also use dyld library loading as follows:
|
||||
|
||||
.. code:: python
|
||||
|
||||
|
||||
LIBRARY = "mycoollib.so"
|
||||
from C import LIBRARY.mymethod(int, float) -> cobj
|
||||
from C import LIBRARY.myothermethod(int, float) -> cobj as my2
|
||||
foo = mymethod(1, 2.2)
|
||||
foo2 = my2(4, 3.2)
|
||||
|
||||
.. note::
|
||||
When importing external non-Codon functions, you must explicitly specify
|
||||
argument and return types.
|
||||
|
||||
How about Python? If you have set the ``CODON_PYTHON`` environment variable as
|
||||
described in the first section, you can do:
|
||||
|
||||
.. code:: python
|
||||
|
||||
from python import mymodule.myfunction(str) -> int as foo
|
||||
print(foo("bar"))
|
||||
|
||||
Often you want to execute more complex Python code within Codon. To that
|
||||
end, you can use Codon's ``@python`` annotation:
|
||||
|
||||
.. code:: python
|
||||
|
||||
@python
|
||||
def scipy_here_i_come(i: List[List[float]]) -> List[float]:
|
||||
# Code within this block is executed by the Python interpreter,
|
||||
# and as such it must be valid Python code
|
||||
import scipy.linalg
|
||||
import numpy as np
|
||||
data = np.array(i)
|
||||
eigenvalues, _ = scipy.linalg.eig(data)
|
||||
return list(eigenvalues)
|
||||
print(scipy_here_i_come([[1.0, 2.0], [3.0, 4.0]])) # [-0.372281, 5.37228] with some warnings...
|
||||
|
||||
Codon will automatically bridge any object that implements the ``__to_py__``
|
||||
and ``__from_py__`` magic methods. All standard Codon types already
|
||||
implement these methods.
|
||||
|
||||
Classes and types
|
||||
-----------------
|
||||
|
||||
Of course, Codon supports classes! However, you must declare class members
|
||||
and their types in the preamble of each class (like you would do with
|
||||
Python's dataclasses).
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Foo:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __init__(self, x: int, y: int): # constructor
|
||||
self.x, self.y = x, y
|
||||
|
||||
def method(self):
|
||||
print(self.x, self.y)
|
||||
|
||||
f = Foo(1, 2)
|
||||
f.method() # prints "1 2"
|
||||
|
||||
.. note::
|
||||
Codon does not (yet!) support inheritance and polymorphism.
|
||||
|
||||
Unlike Python, Codon supports method overloading:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Foo:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __init__(self, x: int, y: int): # constructor
|
||||
self.x, self.y = 0, 0
|
||||
def __init__(self, x: int, y: int): # another constructor
|
||||
self.x, self.y = x, y
|
||||
def __init__(self, x: int, y: float): # another constructor
|
||||
self.x, self.y = x, int(y)
|
||||
def __init__(self):
|
||||
self.x, self.y = 0, 0
|
||||
|
||||
def method(self: Foo):
|
||||
print(self.x, self.y)
|
||||
|
||||
Foo().method() # prints "0 0"
|
||||
Foo(1, 2).method() # prints "1 2"
|
||||
Foo(1, 2.3).method() # prints "1 2"
|
||||
Foo(1.1, 2.3).method() # error: there is no Foo.__init__(float, float)
|
||||
|
||||
Classes can also be generic:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Container[T]:
|
||||
l: List[T]
|
||||
def __init__(self, l: List[T]):
|
||||
self.l = l
|
||||
...
|
||||
|
||||
Classes create objects that are passed by reference:
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Point:
|
||||
x: int
|
||||
y: int
|
||||
...
|
||||
|
||||
p = Point(1, 2)
|
||||
q = p # this is a reference!
|
||||
p.x = 2
|
||||
print((p.x, p.y), (q.x, q.y)) # (2, 2), (2, 2)
|
||||
|
||||
If you need to copy an object's contents, implement the ``__copy__`` magic
|
||||
method and use ``q = copy(p)`` instead.
|
||||
|
||||
Codon also supports pass-by-value types via the ``@tuple`` annotation:
|
||||
|
||||
.. code:: python
|
||||
|
||||
@tuple
|
||||
class Point:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
p = Point(1, 2)
|
||||
q = p # this is a copy!
|
||||
print((p.x, p.y), (q.x, q.y)) # (1, 2), (1, 2)
|
||||
|
||||
However, **by-value objects are immutable!**. The following code will
|
||||
not compile:
|
||||
|
||||
.. code:: python
|
||||
|
||||
p = Point(1, 2)
|
||||
p.x = 2 # error! immutable type
|
||||
|
||||
Under the hood, types are basically named tuples (equivalent to Python's
|
||||
``collections.namedtuple``).
|
||||
|
||||
You can also add methods to types:
|
||||
|
||||
.. code:: python
|
||||
|
||||
@tuple
|
||||
class Point:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
def __new__(): # types are constructed via __new__, not __init__
|
||||
return Point(0, 1) # and __new__ returns a tuple representation of type's members
|
||||
|
||||
def some_method(self):
|
||||
return self.x + self.y
|
||||
|
||||
p = Point() # p is (0, 1)
|
||||
print(p.some_method()) # 1
|
||||
|
||||
Type extensions
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
Suppose you have a class that lacks a method or an operator that might be really useful.
|
||||
|
||||
Codon provides an ``@extend`` annotation that allows programmers to add and modify
|
||||
methods of various types at compile time, including built-in types like ``int`` or ``str``.
|
||||
This actually allows much of the functionality of built-in types to be implemented in
|
||||
Codon as type extensions in the standard library.
|
||||
|
||||
.. code:: python
|
||||
|
||||
class Foo:
|
||||
...
|
||||
|
||||
f = Foo(...)
|
||||
|
||||
# We need foo.cool() but it does not exist... not a problem for Codon
|
||||
@extend
|
||||
class Foo:
|
||||
def cool(self: Foo):
|
||||
...
|
||||
|
||||
f.cool() # works!
|
||||
|
||||
# How about we add support for adding integers and strings:
|
||||
@extend
|
||||
class int:
|
||||
def __add__(self: int, other: str):
|
||||
return self + int(other)
|
||||
|
||||
print(5 + '4') # 9
|
||||
|
||||
Note that all type extensions are performed strictly at compile time and incur no runtime overhead.
|
||||
|
||||
Magic methods
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Here is a list of useful magic methods that you might want to add and
|
||||
overload:
|
||||
|
||||
================ =============================================
|
||||
Magic method Description
|
||||
================ =============================================
|
||||
operators overload unary and binary operators (see :ref:`operators`)
|
||||
``__copy__`` copy-constructor for ``copy`` method
|
||||
``__len__`` for ``len`` method
|
||||
``__bool__`` for ``bool`` method and condition checking
|
||||
``__getitem__`` overload ``obj[key]``
|
||||
``__setitem__`` overload ``obj[key] = value``
|
||||
``__delitem__`` overload ``del obj[key]``
|
||||
``__iter__`` support iterating over the object
|
||||
``__str__`` support printing and ``str`` method
|
||||
================ =============================================
|
||||
|
||||
Other types
|
||||
~~~~~~~~~~~
|
||||
|
||||
Codon provides arbitrary-width signed and unsigned integers, e.g. ``Int[32]`` is a signed 32-bit integer while ``UInt[128]`` is an unsigned 128-bit integer, respectively (note that ``int`` is an ``Int[64]``). Typedefs for common bit widths are provided in the standard library, such as ``i8``, ``i16``, ``u32``, ``u64`` etc.
|
||||
|
||||
The ``Ptr[T]`` type in Codon also corresponds to a raw C pointer (e.g. ``Ptr[byte]`` is equivalent to ``char*`` in C). The ``Array[T]`` type represents a fixed-length array (essentially a pointer with a length).
|
||||
|
||||
Codon also provides ``__ptr__`` for obtaining a pointer to a variable (as in ``__ptr__(myvar)``) and ``__array__`` for declaring stack-allocated arrays (as in ``__array__[int](10)``).
|
||||
|
||||
LLVM functions
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
In certain cases, you might want to use LLVM features that are not directly
|
||||
accessible with Codon. This can be done with the ``@llvm`` attribute:
|
||||
|
||||
.. code:: python
|
||||
|
||||
@llvm
|
||||
def llvm_add[T](a: T, b: T) -> T:
|
||||
%res = add {=T} %a, %b
|
||||
ret {=T} %res
|
||||
|
||||
print(llvm_add(3, 4)) # 7
|
||||
print(llvm_add(i8(5), i8(6))) # 11
|
||||
|
||||
--------------
|
||||
|
||||
Issues, feedback, or comments regarding this tutorial? Let us know `on GitHub <https://github.com/exaloop/codon>`__.
|
||||
|
|
@ -1,66 +0,0 @@
|
|||
Calling Python from Codon
|
||||
=========================
|
||||
|
||||
Calling Python from Codon is possible in two ways:
|
||||
|
||||
- ``from python import`` allows importing and calling Python functions from existing Python modules.
|
||||
- ``@python`` allows writing Python code directly in Codon.
|
||||
|
||||
In order to use these features, the ``CODON_PYTHON`` environment variable must be set to the appropriate
|
||||
Python shared library:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export CODON_PYTHON=/path/to/libpython.X.Y.so
|
||||
|
||||
For example, with a ``brew``-installed Python 3.9 on macOS, this might be
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib
|
||||
|
||||
Note that only Python versions 3.6 and later are supported.
|
||||
|
||||
``from python import``
|
||||
----------------------
|
||||
|
||||
Let's say we have a Python function defined in *mymodule.py*:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def multiply(a, b):
|
||||
return a * b
|
||||
|
||||
We can call this function in Codon using ``from python import`` and indicating the appropriate
|
||||
call and return types:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from python import mymodule.multiply(int, int) -> int
|
||||
print(multiply(3, 4)) # 12
|
||||
|
||||
(Be sure the ``PYTHONPATH`` environment variable includes the path of *mymodule.py*!)
|
||||
|
||||
``@python``
|
||||
-----------
|
||||
|
||||
Codon programs can contain functions that will be executed by Python via ``pydef``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@python
|
||||
def multiply(a: int, b: int) -> int:
|
||||
return a * b
|
||||
|
||||
print(multiply(3, 4)) # 12
|
||||
|
||||
This makes calling Python modules like NumPy very easy:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@python
|
||||
def myrange(n: int) -> List[int]:
|
||||
from numpy import arange
|
||||
return list(arange(n))
|
||||
|
||||
print(myrange(5)) # [0, 1, 2, 3, 4]
|
||||
Loading…
Reference in New Issue