Packages come from git (a registry can be added later; a bare version in rex.toml is reserved for it). rex.toml [deps] names them, rex.lock pins exact commits, and run/build/check/test download what is missing. `import json` reads the package's json.rex; package files get `@json...` module keys so they never mix with the project's own files. Newest tag of the same major version wins when packages disagree. The language server reads downloaded packages. Docs: docs/14-packages.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
244 lines
20 KiB
Markdown
244 lines
20 KiB
Markdown
# Rex language spec (draft)
|
|
|
|
Goal: Go-like simplicity, C++/Rust speed, no GC, no low-level syntax.
|
|
Targets: game dev, CLI tools, desktop apps.
|
|
|
|
## Decisions
|
|
- **Methods**: declared inside the struct body. Fields are accessed directly
|
|
(`self` available for the whole value). No receiver, no `mut` keyword:
|
|
the compiler infers whether a method mutates.
|
|
- **Variables**: `var x = 1` creates a changeable variable (type inferred;
|
|
`var x: int = 1` with a type). `x = 2` changes it. `const X = 1` is fixed.
|
|
- **Types are written after a colon**, everywhere: `var x: int = 1`,
|
|
`func area(w: float, h: float): float`, `struct User { name: string, age: int }`,
|
|
enum data `Circle(r: float)`, generic bounds `[T: Ordered]`, several results `func f(): (int, error)`.
|
|
Names in a row share a type: `(a, b: int)`, `x, y: float`. Without a colon
|
|
the compiler explains what to add.
|
|
- **Sized numbers**: `i8 i16 i32 u8 u16 u32 u64 f32`; `i64` is `int` and `f64` is
|
|
`float` (same type). A number literal takes the type asked for (`var x: u8 = 200`,
|
|
`b + 1`, `[1, 2]` as `[]u8`) and a literal that doesn't fit is an error. Different
|
|
number types never mix: convert with `u8(x)`, `f32(x)`, `int(x)`, ... Integers wrap
|
|
around on overflow; float to integer stops at the nearest end (NaN is 0). Dividing by
|
|
zero panics. Unsigned values can't be negated. `f32` prints the shortest text that
|
|
reads back the same.
|
|
- **Sharing**: `shared T` (reference counted), `weak T` (non-owning).
|
|
Everything else is plain ownership, inferred by the compiler.
|
|
- **Errors**: Go-style `(value, error)` returns, with `?` to propagate.
|
|
- **Missing values**: `T?` with `nil`. Compiler requires a nil check before use.
|
|
- **Strings**: interpolation with `"Hello {name}"`; literal brace is `\{`.
|
|
- **Visibility**: private by default, `pub` makes it public.
|
|
- **Loops**: `for` only: `for i in 0..10`, `for x in items`, `for cond { }`,
|
|
`for { }` (infinite), with `step` for strides. No C-style three-part loop.
|
|
- **Numbers**: `int` (64-bit) and `float` (64-bit) for everyday code; sized
|
|
types `i8 i16 i32 i64 u8 u16 u32 u64 f32 f64` when exact control is needed.
|
|
`var a = 42` is `int`, `var b = 3.14` is `float`.
|
|
- **print**: `print(a, b, c)` writes values separated by spaces, then a newline.
|
|
- **Conversions**: no implicit int/float mixing, except that an integer
|
|
literal fits a float (`var x: float = 1`, `x * 2`). Use `int(x)` / `float(x)`.
|
|
- **Integer overflow** wraps around; `/` and `%` by zero panic (exit code 2).
|
|
- **Evaluation order** is left to right, always (arguments, `&&`/`||` short-circuit).
|
|
- **Top-level `const`** may not call functions; uses are replaced by the value.
|
|
- **Structs are values**: `var b = a` copies; passing to a function copies.
|
|
Methods go inside the struct and see fields directly; `self` is the whole value.
|
|
Inside a method, a name resolves as: local/parameter, then field, then method
|
|
of the same struct (call it bare: `update(dt)`), then global function/const.
|
|
- **Mutation is inferred**: a method that assigns to a field (or calls a mutating
|
|
method on `self`) is mutating. Mutating methods can't be called on a `const`
|
|
or on a temporary like `make().update()`.
|
|
- **Struct literals**: positional `Vec2{1, 2}` needs every field; named
|
|
`Player{name: "A"}` may leave fields out (they start at zero: 0, 0.0, false, "").
|
|
Initializers run in the order written.
|
|
- **Printing a struct** shows `Vec2{x: 1.0, y: 2.0}`. `==` on structs is not
|
|
allowed yet; compare fields.
|
|
- **Lists** `[]T`: `[1, 2, 3]`, `var xs: []int` (empty), `xs[i]`, `xs[i] = v`,
|
|
`xs.len()`, `xs.push(v)`, `xs.pop()`, `xs[1..3]` (a copy), `for x in xs`,
|
|
`for i, x in xs`. Lists are values: `var b = a` and passing to a function
|
|
copy. A function that never changes a list parameter gets it without a copy
|
|
(the compiler works this out); returning a local list moves it.
|
|
Looping walks the list as it was when the loop started. `print` shows `[1, 2]`.
|
|
Out-of-range index/slice and `pop()` on an empty list panic (exit code 2).
|
|
`s.len()` on a string is its length in bytes.
|
|
- **Enums** carry data per case: `enum Shape { Circle(r float) Rect(w, h float) Point }`,
|
|
with methods inside, like structs. Build a value with the qualified name:
|
|
`Shape.Circle(2.0)`, `Shape.Point`. Enums are values (copied like structs); a
|
|
case may hold lists. The zero value is the first case. Printing shows
|
|
`Shape.Circle(2.0)`. `==` works on enums whose cases carry no data.
|
|
An enum can't contain itself directly, but may hold a list of itself.
|
|
- **match** is exhaustive: missing cases are a compile error (the message lists
|
|
them); `_` covers the rest. Works on enums, int, bool, string.
|
|
Patterns: `Circle(r)`, `Rect(w, _)`, `Point`, `Shape.Point`, literals, `_`, or a
|
|
lowercase name that takes the whole value. A capitalized name that isn't a case
|
|
is an error (never a silent catch-all). Nested patterns come later.
|
|
As a statement, arms may be blocks (and may `break`/`continue`/`return`).
|
|
As a value (`var x = match ...`, `return match ...`), every arm is a single
|
|
expression, or a lone `return`/`break`/`continue`. Arms of a value `match` must
|
|
have the same type.
|
|
- **Errors**: `error` is a built-in type: nil (no error) or a message. Make one with
|
|
`error("msg {x}")`; print it or use it in `"{err}"` (nil prints `nil`).
|
|
A function may return several values: `func f(): (int, error)`; take them with
|
|
`var n, err = f()` (`_` throws one away). `return v, nil` or the shorthand
|
|
`return error("x")` (zero values are filled in). The error ignored by a bare
|
|
`f()` statement is a compile error.
|
|
`call()?` passes the error up: on error the enclosing function returns it (zero
|
|
values for the rest), otherwise `?` gives the value. Needs a function whose last
|
|
result is `error`; works on `(T, error)` and `error` calls.
|
|
`func main(): error` is allowed: an error prints `error: msg` to stderr, exit code 1.
|
|
- **Optionals**: `T?` holds a `T` or `nil`. A `T` converts to `T?` implicitly.
|
|
Compare with `x == nil` / `x != nil`. To use the value, check first: the compiler
|
|
knows `x` is set inside `if x != nil { }`, after `if x == nil { return }` (or
|
|
break/continue), and on the right of `x != nil &&` / `x == nil ||`. Assigning to
|
|
`x` forgets that. (Reads are also checked at runtime, so it can never go wrong.)
|
|
Not yet: `T?` as a struct field / enum data / list element, `??`, `x!`, `match` on nil,
|
|
`?` on optionals.
|
|
- **Last use moves**: `var b = a` copies `a`, unless `a` is never used again (and no loop
|
|
is in the way): then `b` takes the value and nothing is copied (`src/moves.rs`).
|
|
- **Generic functions**: `func max[T: Ordered](a, b: T): T`. The types are worked out from the
|
|
arguments (`max(3, 9)`, `first(["a"])`; bare number literals become `int`, or `float` if any is
|
|
a float); there is no `max[int](...)` yet. Each set of types gets its own compiled copy, so
|
|
generics cost nothing at runtime. Rules after the colon: `Ordered` (numbers and strings) and
|
|
`Number`; no rule means anything goes, and a mistake then shows up when the copy is checked
|
|
(the message names the types). A generic function that is never used is not checked.
|
|
A generic function needs a parameter that mentions its type.
|
|
- **Interfaces** are rules for generics, Go-style: no `implements`. A type follows an
|
|
interface just by having its methods with the same parameters and result:
|
|
`interface Shape { area(): float name(): string }`, then `func show[T: Shape](s: T)`
|
|
(also `struct Gallery[T: Shape]`, `[T: m.Shape]` from another file). A type that doesn't
|
|
gets a message naming the missing or different method. Generic code may call the methods
|
|
the interface lists even if they are private in another file.
|
|
- **Interface values**: an interface is also a type: `var s: Shape = Circle{2.0}`, `[]Shape`,
|
|
`func f(s: Shape)`, struct fields, results. A struct or enum value with the right methods
|
|
converts automatically (the value is copied into the interface value), and `s.area()` calls
|
|
the method of whatever is inside. Interface values are values like any other: `var t = s`
|
|
copies the inside, and a method that changes the value (`s.scale(2.0)`) changes only that
|
|
variable's copy. `nil` is the empty interface value (`s == nil`); calling a method on it
|
|
panics (exit code 2). A list can mix types: `var xs: []Shape = [Circle{1.0}, Rect{2.0, 3.0}]`.
|
|
Printing shows the value inside. Not yet: turning an interface value back into its type,
|
|
one interface value used as another interface, interfaces with type parameters.
|
|
Interfaces can't take type parameters yet.
|
|
- **Generic structs and enums**: `struct Stack[T] { items: []T func push_item(x: T) { items.push(x) } }`,
|
|
`enum Maybe[T] { Some(value: T) None }`. Use them with the types written out:
|
|
`Stack[int]{}`, `var s: Stack[string]`, `Pair[string, float]{"pi", 3.14}`,
|
|
`Maybe[int].Some(3)`, `func f(): Maybe[int]`, and from another file `m.Stack[int]{}`.
|
|
The types are never guessed (`Stack{}` is an error). Like functions, each list of types
|
|
becomes its own type (`Stack[int]` prints with its types), rules `[T: Number]` work, and
|
|
they may contain themselves through lists (`kids: []Tree[T]`). A capitalized name followed
|
|
by `[types]` and then `{` or `.` is read as a generic type, anything else as indexing.
|
|
A generic type that is never used is not checked.
|
|
- **Shared values**: `var tex = shared Texture{...}` makes one value with a counter. `var b = a`,
|
|
passing it, putting it in a list or a struct field all give another owner (no copy of the value);
|
|
the value is freed when the last owner goes. Use it like the thing inside: `tex.name`,
|
|
`tex.touch()`, `tex.uses = 1`, `match light { ... }`. A change through any owner is seen
|
|
by all. Only structs and enums can be shared for now. Written as a type: `shared Texture`.
|
|
A shared value is not copied by `var b = a`, so two owners really are one value.
|
|
- **Weak pointers**: `weak win` gives a non-owning pointer to a shared value (type `weak Window`;
|
|
`weak Window?` means the same). It doesn't keep the value alive, so a window and its buttons
|
|
can point at each other without leaking (the button's side is `weak`). `w == nil` /
|
|
`w != nil` tell whether the value is still there; using a weak pointer whose value is gone
|
|
(or that was never set) panics with a clear message (exit code 2). A weak pointer starts
|
|
as `nil` (`Button{label: "x"}`) and can be assigned `nil`. To use the value as an owner, `w.upgrade()` gives a `shared T?`: the shared
|
|
value (now with one more owner, so it stays alive while you hold it) or `nil` if it is gone.
|
|
Not yet: `==` between shared values (identity), shared lists/strings/numbers.
|
|
- **Function values and closures**: the type is `func(int, string): bool` (no `: result` means
|
|
nothing is returned). Write one as `func(x: int): int { return x + 1 }`, or short as
|
|
`x => x + 1` / `(a, b) => a + b` where the types are known from the place it is used
|
|
(a parameter, a typed variable, a return, a struct field) or, for a generic function,
|
|
from its other arguments (`map(nums, x => x * 2)`). A top-level function is a value too:
|
|
`apply(double, 4)`. Call a function value like a function: `f(3)`, `b.on_click(name)`,
|
|
`make()(3)`. A closure **copies** the variables it uses when it is made and keeps its own copies
|
|
(so it can count: `n += 1` inside changes only its copy, which persists between calls).
|
|
Copying a closure (`var g = f`, passing it) copies its state too. Calling a function value
|
|
counts as changing it, like a method that changes `self`. A function value that was never
|
|
set panics when called (exit code 2). Not yet: closures with block bodies in short form,
|
|
methods as values, several results from a function value, `==` on functions.
|
|
- **Memory**: no GC, nothing to write. Every value has exactly one owner and is
|
|
freed when its owner's block ends (also on early `return`, `break`, `continue`
|
|
and `?`). Lists, structs, enums, optionals, errors and strings all follow this.
|
|
Assigning or passing a variable copies it (`var b = a`); a fresh value (a call
|
|
result, a literal) or a returned local moves without copying. Strings are
|
|
immutable and counted, so copying one only bumps a counter; literals are never
|
|
freed. A parameter that the function doesn't change is passed by address, so
|
|
passing lists around costs nothing. Every test program runs under the address
|
|
sanitizer, so a leak, double free or use after free fails the test.
|
|
|
|
- **Imports**: `import os` (built-in) or `import utils` (the file `utils.rex` next to the
|
|
main file; `import tools.fs` is `tools/fs.rex`, used as `fs.name`). Use things as
|
|
`utils.double(2)`, `utils.Vec2{1.0, 2.0}`, `utils.Shape.Circle(1.0)`, `utils.SCALE`,
|
|
and `utils.Vec2` in types. Only `pub` functions, consts, types, methods and fields can be used
|
|
from another file (fields are private too unless written `pub x: int`). Each file has its own imports and its
|
|
own names, so two files may both define `f`. Import cycles and missing files are errors;
|
|
errors point at the file they are in. Names of module types print with the module
|
|
(`utils.Vec2{...}`). C headers: see **C libraries** below.
|
|
- **Built-in modules**: `os` (`read_file write_file append_file exists exit env
|
|
read_line`, and the value `args`), `math` (`sqrt pow floor ceil round trunc sin cos tan asin acos atan atan2 exp log log10 hypot min max abs PI E`),
|
|
`time` (`now sleep`). Text has methods: `upper lower trim split lines fields replace
|
|
repeat contains starts_with ends_with index_of to_int to_float`; `[]string` has `join`.
|
|
|
|
- **Maps** `map[K, V]`: `{"a": 1}` (types guessed), `map[string, int]{}` or `var m: map[string, int] = {}` (empty).
|
|
Keys: int/whole-number types, string, bool. Values: anything except `T?`. `m[k] = v` adds or replaces,
|
|
`m[k]` panics if the key is missing, `m[k] += 1` treats a missing key as zero, methods `has get get_or
|
|
remove keys values len`, `for k, v in m` / `for k in m`. Entries keep insertion order (hash index +
|
|
ordered arrays; `remove` is O(n)). Maps are values (`var b = a` copies). Not yet: `==`, generic `map[K,V]` params.
|
|
- **Docs**: `docs/` holds learner notes in markdown; a test compiles and runs every example in them.
|
|
- **Characters** `char`: one Unicode character, `'a'` `'\n'` `'é'` (escapes `\n \t \r \0 \\ \' \"`). Compare with
|
|
`== != < > <= >=`, `match` on them (`'+' => ...`), use as map keys; no arithmetic (`int(c)`, `char(n)`).
|
|
`string(x)` (or `"{c}"`) makes text. `for c in s` / `for i, c in s` walk characters (`i` = byte position,
|
|
invalid UTF-8 comes out as U+FFFD). `s[i]` is the character starting at byte `i` (panics out of range or
|
|
in the middle of a character); strings can't be assigned to. Methods `is_digit is_letter is_alnum is_space
|
|
is_upper is_lower upper lower` (ASCII plus Latin-1 letters). `s.chars()` gives a `[]char`. Not yet: `[]char`→string.
|
|
- **Formatter** `rex fmt [--check] <file|folder>..` (`src/fmt.rs`): text based, so comments and blank lines survive.
|
|
4-space indent, spaces around binary operators and after `,` `:`, tight `Point{...}` / map literals and `[]T`,
|
|
at most one blank line, trailing `//` comments of neighbouring lines aligned. It never moves line breaks (a one-line
|
|
block stays one line), indents continuation lines (after an operator, or inside open brackets) one level, and
|
|
verifies that the token stream is unchanged before writing. `examples/` must stay formatted (a test checks).
|
|
- **Editor support** (`editors/`): `rex lsp` is a language server (`src/lsp.rs`, own JSON in `src/json.rs`, no
|
|
dependencies): full-text sync, error diagnostics from the real checker (imports read from disk or from open
|
|
buffers; errors in other files shown at the top of the importing file; UTF-16 columns), whole-file formatting.
|
|
`editors/vscode` (extension + TextMate grammar), `editors/vim` (syntax), `editors/README.md` (Neovim, Helix, others).
|
|
Also outline, go to definition, hover, completion and signature help from a token index (`src/index.rs`),
|
|
including names from C headers. Not yet: references, rename; the compiler reports only the first error.
|
|
- **C libraries** (`src/cheader.rs`): `import "zlib.h"` reads the header (no hand-written declarations): it runs
|
|
`cc -E -dD` and parses the declarations, enum values and object-like `#define`s from the header and the headers it
|
|
includes itself (in a shared folder like /usr/include: the header and what it includes, minus the C library).
|
|
Names are used as `zlib.name` (prefix = file name without `.h`; `import z "zlib.h"` picks one);
|
|
`import "zlib.h" { a, b }` makes only those names visible, without a prefix. Linking: `pkg-config` with names
|
|
guessed from the header path, else `import "x.h" link "x"` (`-lx`). A header next to the program adds `-I` and
|
|
builds the `.c`/`.cpp`/`.cc` file with the same name into the program (C++ with `c++`, linked with `-lstdc++`).
|
|
Types: C ints -> sized numbers (`int` i32, `long` int, `size_t` u64), `double` float, `char` i8 (takes `'a'`),
|
|
`char *` -> string (copied both ways, NULL -> ""), enums -> `i32` constants, plain structs -> Rex structs with the
|
|
C field names (`raylib.Color`; converted field by field at calls), other pointers -> pointer types
|
|
(`Ty::CPtr`, `stdio.FILE`, `nil` = NULL, `==` compares); `void *` (`pointer`) mixes with every pointer type.
|
|
`T *` parameters take a variable (the address is passed; counts as a change for inferred mutation) or a list
|
|
(its items); `const T *` also takes any value. Constants: numbers, strings, and struct literals like
|
|
`CLITERAL(Color){...}` (emitted as the macro). Programs that import headers are built with `-std=gnu11`.
|
|
Not supported (a clear error when called): variadic functions, callbacks, unions/arrays by value, lists of
|
|
structs, `long double`, function-like macros. Rex never frees C memory. `rex header x.h [word]` lists what Rex sees.
|
|
- **Packages** (`src/pkg.rs`): git repositories (a registry may come later; `json = "1.2.0"` is reserved for
|
|
it and is an error now). `rex.toml` `[deps]` maps a name to `"host/owner/repo@rev"` (https is added; `://`
|
|
and `git@host:` addresses kept as written; rev = tag, branch or commit; none = main branch) or a folder
|
|
(`"../lib"`). `rex.lock` (lines `name url rev commit`) pins commits. Cache: `$REX_HOME` or `~/.cache/rex`
|
|
(`git/` bare mirrors, `pkg/<url>/<commit>/` unpacked with `git archive`). `run/build/check/test` download what
|
|
is missing and rewrite the lock when rex.toml changed; the LSP only reads what is downloaded. Versions: one per
|
|
package name in a build; same repo + same major -> the newest tag wins (re-walk), else an error naming both.
|
|
The project's folder deps override. `import json` -> the package's `json.rex`, `import json.p` -> its `p.rex`;
|
|
inside the package, imports count from its top folder. Module keys of package files start with `@`
|
|
(`@json.p`, see `loader::Scope`), and the loader sets `Import.alias` to the written last word. A project file
|
|
`json.rex` next to a package `json` is an error. Commands: `rex init [name]`, `rex add <addr>[@rev] [--name n]`
|
|
(newest tag when none given; name from the package's `[package] name`, else the last address part),
|
|
`rex remove`, `rex get`, `rex update [names]` (newest tag of the same major; branch deps move), `rex deps`.
|
|
- **String slicing**: `s[a..b]`, `s[a..]`, `s[..b]` (byte positions, bounds checked; panics with "the length is N").
|
|
- **Running**: `rex run file.rex a b c` passes `a b c` to the program (`os.args`).
|
|
- **Example**: `examples/calc/` is a multi-file calculator (tokenizer, parser, evaluator, shared scope); `session.txt` / `session.out` are checked by a test.
|
|
|
|
## Open questions
|
|
- Braces vs indentation (currently braces, no semicolons)
|
|
- Concurrency details
|
|
|
|
## Status
|
|
Working: functions, int/float/bool/string, var/const, if/else, for (infinite,
|
|
while, range with step, over lists), break/continue, string interpolation, print,
|
|
structs with methods (value semantics, inferred mutation), lists (value
|
|
semantics, bounds checked), enums with data + exhaustive match, errors (`error`,
|
|
multi-value returns, `?`), optionals (`T?`, `nil`, narrowing), `rex run`.
|
|
Not yet: nested match patterns, `==` on lists/structs, top-level const lists. These give a clear "not supported yet" error.
|
|
Known gaps: memory is only freed on normal exits (a panic just ends the program).
|