zed-p8/README.md

# zed-p8

A Zed extension for the [PICO-8](https://www.lexaloffle.com/pico-8.php) fantasy
console. Reasonable editor support for the entire `.p8` cartridge file format
and for PICO-8's Lua dialect — including the parts where PICO-8 deviates from
standard Lua 5.2 (compound assignments, `?` print shorthand, single-line
`if (cond) ...`, `!=`, `0b...` binary literals, integer divide `\`, peek
operators `@`/`%`/`$`, the rotate/logical-shift family `<<>` / `>><` / `>>>`,
and `#include`).

## Status — v0.3 (unreleased)

Working today:

- **`tree-sitter-p8-cart`** ( `grammars/p8-cart/` ) — parses the `.p8`
  cartridge container: the magic header line, `version` line, and the named
  sections `__lua__`, `__gfx__`, `__gff__`, `__label__`, `__map__`, `__sfx__`,
  `__music__`. Unknown `__name__` markers fall through to `unknown_section`.
- **`tree-sitter-pico8-lua`** ( `grammars/pico-8-lua/` ) — fork of
  [`tree-sitter-grammars/tree-sitter-lua`](https://github.com/tree-sitter-grammars/tree-sitter-lua)
  with the PICO-8 dialect added. Handles every dialect form the manual
  documents: see *Dialect coverage* below. Upstream attribution is preserved
  in `grammars/pico-8-lua/UPSTREAM-LICENSE.md`.
- **`Pico-8 Cartridge`** language ( `languages/pico-8-cart/`, suffix `.p8` ) —
  config, marker highlights, outline view, and an injection that hands the
  `__lua__` body to the Pico-8 Lua grammar.
- **`Pico-8 Lua`** language ( `languages/pico-8-lua/`, suffix `.p8lua` ) —
  config, dialect-aware highlights with PICO-8 builtins recognized, brackets,
  indents, outline.

The `.p8lua` suffix is the convention for bare Pico-8 Lua source files — the
ones pulled into a cart via `#include`. Plain `.lua` files are intentionally
*not* claimed by this extension, so users who keep stock Lua files alongside
their PICO-8 work continue to get standard Lua treatment.

### Dialect coverage

| Feature | Status |
|---|---|
| `!=` ( alias for `~=` ) | ✓ |
| Compound assignment: `+= -= *= /= %= \= ^= ..= &= \|= ^^= <<= >>= >>>= <<>= >><=` | ✓ |
| Integer divide `\` and modulo `%` ( binary ) | ✓ |
| Bitwise XOR `^^` ( binary, in addition to upstream's `~` ) | ✓ |
| Logical shift right `>>>`, rotate left `<<>`, rotate right `>><` | ✓ |
| Hex literals with fractional part: `0x11.4000` | ✓ |
| Binary literals: `0b1010` | ✓ |
| Memory peek prefix unary: `@addr`, `%addr`, `$addr` | ✓ |
| Single-line `if (cond) stmt [else stmt]` ( no `then`/`end` ) | ✓ |
| Single-line `while (cond) stmt` ( no `do`/`end` ) | ✓ |
| `?` print shorthand statement | ✓ |
| `#include path` directive | ✓ |
| `_init` / `_update` / `_update60` / `_draw` highlighted as builtins | ✓ |

### Line-significance (resolved in v0.3)

PICO-8's shorthand `if (cond) ...` and `while (cond) ...` are
line-bounded: a later-line `else` belongs to an enclosing standard
`if`, not the shorthand, and a multi-statement single-line shorthand
body collects every statement on the line. The external scanner emits
a zero-width `LINE_END` token at `\n` / `\r` / EOF when (and only
when) the parser is at the body-or-terminator decision point of a
shorthand statement, so the AST now matches PICO-8 semantics — see
[`grammars/pico-8-lua/KNOWN_LIMITATIONS.md`](grammars/pico-8-lua/KNOWN_LIMITATIONS.md)
for the wiring detail and
[`grammars/pico-8-lua/test/corpus/shorthand_line_end.txt`](grammars/pico-8-lua/test/corpus/shorthand_line_end.txt)
for the test corpus.

### Known limitations

- **No language server.** No completion, hover docs, or diagnostics for
  PICO-8 builtins yet — only a static `function.builtin` highlight on
  recognized names. See Roadmap.
- **No `.p8.png` support.** Only the plain-text `.p8` format is handled —
  the PNG-steganography variant is out of scope for a text-focused IDE
  extension.
- **Hex sections are unhighlighted blobs.** `__gfx__`, `__map__`, `__sfx__`,
  `__gff__`, `__music__`, `__label__` parse as opaque line bodies. Roadmap
  v0.4 covers per-section highlighters.

## Repository layout

```
zed-p8/
  extension.toml                  ← Zed extension manifest
  package.json                    ← workspace root; hosts tree-sitter-cli
  grammars/
    p8-cart/                      ← cart-format tree-sitter grammar
      grammar.js
      tree-sitter.json
      src/                        ← generated parser ( committed )
    pico-8-lua/                   ← Pico-8 Lua dialect tree-sitter grammar
      grammar.js
      tree-sitter.json
      package.json                ← marks this dir as ESM for node
      src/                        ← generated parser + scanner.c ( committed )
      UPSTREAM-LICENSE.md         ← MIT, tree-sitter-lua by Munif Tanjim
  languages/
    pico-8-cart/                  ← Pico-8 Cartridge language files
      config.toml
      highlights.scm
      injections.scm              ← injects pico-8-lua into __lua__ body
      outline.scm
    pico-8-lua/                   ← Pico-8 Lua language files
      config.toml
      highlights.scm
      brackets.scm
      indents.scm
      injections.scm
      outline.scm
  examples/
    hello.p8                      ← minimal test cart
  references/                     ← upstream PICO-8 manual + Zed doc links
```

Both grammars live in subdirectories of this same repository. Zed's
`[grammars.*]` block supports a `path` field, so the extension manifest
points each grammar at this repo's git URL plus the relevant subdir.

## Local development

Prerequisites: Node.js ( for `tree-sitter-cli` ) and Zed. Rust is *not*
required unless / until we add a language-server harness ( v0.3 ).

```sh
npm install                        # one-time, installs tree-sitter-cli
```

### Edit a grammar and reload

1. Edit `grammars/<name>/grammar.js`.
2. Regenerate from the grammar's directory:

   ```sh
   ( cd grammars/p8-cart   && npx tree-sitter generate )
   # or
   ( cd grammars/pico-8-lua && npx tree-sitter generate )
   ```

3. Sanity-check by parsing a sample file:

   ```sh
   ( cd grammars/p8-cart   && npx tree-sitter parse ../../examples/hello.p8 )
   ( cd grammars/pico-8-lua && npx tree-sitter parse path/to/file.p8lua )
   ```

4. Commit the regenerated `src/parser.c` along with the grammar change.
   Zed clones the grammar repo at the SHA in `extension.toml`, so changes
   only take effect after a commit.

5. Update the `rev` field of the affected `[grammars.*]` block(s) in
   `extension.toml` to the new SHA, then in Zed run `zed: install dev
   extension` ( or *Install Dev Extension* on the Extensions page ) and
   select this directory. Reinstall after every commit that should be
   picked up.

   Logs: `zed: open log`. Run `zed --foreground` for live stdout.

### Edit only language queries

Files under `languages/*/` ( `highlights.scm`, `injections.scm`, etc. )
are loaded directly by Zed — no regeneration step needed. Reinstall the
dev extension to pick up changes.

### Tests

Sample carts live under `examples/`; parse them directly with
`tree-sitter parse <file>` for ad-hoc checks.

The cart grammar has a corpus under `grammars/p8-cart/test/corpus/` —
run `( cd grammars/p8-cart && npx tree-sitter test )`. The corpus
covers the empty-section skeleton, normal Lua content, the case where
a Lua identifier resembles a section marker ( e.g. `local __foo__ = 1`
must remain a `line`, not be re-tokenized as a marker ), and the
fallback `unknown_section` rule.

The Lua grammar has a corpus under `grammars/pico-8-lua/test/corpus/` —
run `( cd grammars/pico-8-lua && npx tree-sitter test )`. The corpus
exercises shorthand `if`/`while` line-end behavior: dangling-else,
multi-statement bodies, EOF termination, nested same-line shorthands,
and coexistence with standard `if (parenthesized) then ... end`.

## Roadmap

### v0.3 — Language server integration

The line-significance prerequisite is now satisfied (see *Line-significance*
above), so LSP features that walk the AST — unreachable-code lint,
goto-definition through a conditional branch — have a correct structure
to work against.

Wire up [`japhib/pico8-ls`](https://github.com/japhib/pico8-ls) ( or whichever
PICO-8 LSP is most maintained at the time ) for:

- Completion of PICO-8 builtins ( `spr`, `circfill`, `btn`, `flr`, … ).
- Signature help and hover docs sourced from the manual.
- Cart-aware analysis — the LSP already understands `.p8` section markers
  and only analyzes the `__lua__` body.
- Per-cart diagnostics.

This requires a Rust component ( the `zed_extension_api` crate ) that
downloads the LSP binary and defines `language_server_command`.
See [Zed's developing-extensions docs](https://zed.dev/docs/extensions/developing-extensions).

### v0.4 — Polish

- LuaCATS / EmmyLua stub file enumerating PICO-8's ~110 globals, for users
  who'd rather wire up `lua-language-server` against their `#include`-d
  `.p8lua` files.
- Per-section highlighters for the hex blocks: `__gfx__` colored by palette
  index, `__sfx__` / `__music__` parsed as note streams, `__map__` as tile
  indices, `__gff__` as flag bytes.
- Snippets for common idioms ( the `_init` / `_update` / `_draw` triad,
  `for x=0,127 do … end`, palette swap setup, etc. ).

## References

- PICO-8 manual: `references/pico-8_manual.txt`
- Zed extension docs: see links in `references/zed-doc-links.md`
- Cart file-format spec ( community wiki, not in the official manual ):
  https://pico-8.fandom.com/wiki/P8FileFormat
- Upstream Lua grammar: https://github.com/tree-sitter-grammars/tree-sitter-lua
  ( MIT, by Munif Tanjim — preserved in `grammars/pico-8-lua/UPSTREAM-LICENSE.md` )

## License

The cart grammar and the Zed extension files are 0BSD ( see `LICENSE` ).
The PICO-8 Lua grammar is a fork of MIT-licensed `tree-sitter-lua`; the
upstream license is preserved at `grammars/pico-8-lua/UPSTREAM-LICENSE.md`
and applies to the derived files in that directory.