Plan
This document explains planned work (what's on the roadmap, etc).
The Prioritized TODOs section lists all TODOs that have currently been prioritized. This is essentially the short-term roadmap.
The Backlog section lists medium-term TODO items, or ideas that seem interesting.
The Icebox section lists TODO items that are much more difficult to do, but are still interesting. It also has items which have a low priority.
New TODOs start out in the Backlog and over time they are then moved around to other sections as needed.
1.1 TODO build: change clone path to use HTTPS
Change .build.yml to use https://git.sr.ht/~listx/lilac instead of SSH (because it won't be a private repo).
1.2 TODO CI: add site publishing step
Publish main branch to https://lilac.funloop.org/
2 Backlog
Ideally the backlog would use something like a Story Map to encode both time and priority as two separate dimensions. However we just have a flat list of items for now because we don't have a good way of representing story maps.
2.1 TODO add to Nixpkgs
This way, Nix users can just install it with Nix without having to clone the source repo.
2.2 TODO write manpage
The manpage should have some minimal examples of how to use Lilac. It should be a condensed version of the larger (complete) user guide.
2.4 TODO code blocks: support named tabs
This comes up in technical writing a lot when writing guides for different platforms. We need some way of grouping blocks together. The simplest is probably just grouping consecutive blocks with the same "tag" metadata as tabs.
2.7 TODO allow users to supply their own CSS
Add a config option which specifies the CSS filename path to load, in addition to the default CSS.
3.1 TODO GTOC: optionally treat missing index.org files as an error
We can just add GTOC generation preferences into the data:ProjectConf, which can let the user choose different "flavors" of GTOC:
full GTOC (expect index.org at every folder, not finding one in a folder is an error). That is, there should be an index.org at every folder where Org files (objects) reside.
root GTOC only (only expect root index.org, any others are optional)
no GTOC (skip all GTOC generation --- this is the default)
By default we expect no GTOC at all (this makes sense because this whole concept is not a native Orgmode thing). This is good, but needs to be done after implementing GTOC, because we have to also move the data:ProjectConf to the compilation step (a bit more "upstream" than weaving, which is arguably at the very end).
We need to read the project configuration during compilation, so that we can affect archive generation.
3.4.1 TODO color themes
Let colors be defined in data:ProjectConf, and then reuse them during weaving (instead of having them hardcoded).
3.6 TODO "raw" or "copy-editor" view of code cells
This could be a widget where we show the exact view of the raw UTF-8 used to write the cell. We could add subtle bits for representing spaces, newlines, and others (maybe even a Unicode code point view, similar to how you can describe the character underneath point in Emacs).
We could also show cell metadata, if it has any (such as blocks, headings, etc).
4.1 TODO track time-sensitive things with invisible timestamps only the compiler can see?
Sort of like citations, but, imagine if we link to something (e.g., a version of software in 2025). And then we want to check if that's still accurate. We need some way of knowing what info (from an external source) may be out of date.
We need some hidden way of saying "please remember a special TIMESTAMP" and then have the compiler spew warnings if the "freshness" has not been renewed. Basically the idea of document "freshness", but more granular. Instead of doc freshness, we could check for freshness of specific parts. So any phrase like "as of YYYY" could be earmarked for checking.
We could use a special link with a timestamp, like this:
[lilac_freshness_expiry:<TIMESTAMP>]Aside: we probably don't need document freshness because it's a bit moot in the world of literate programming where the doc itself requires updates to it (you can't just update code alone without updating docs with literate programming).
4.2 TODO Orgmode formatter
If we can "render" an OrgDoc back out to Text, we can format it. What we need is a way to also track comments though, so that comments are preserved in the formatting process. So we can't use lexeme any more and we need to manually parse comments and also skip whitespace between every cell.
This doesn't seem too difficult to do... One ugly thing though is how to do line wrapping...
4.3 TODO allow some kind of inline "commentary" system
Allow writing comments that refer to the previous cell. And then display the comment for the cell on hover, for example.
4.4 TODO tangle/detangle interop
There are several ways to understand the relationship between Org and tangled files:
add special comments in the tangled text (this is what Org can do and presumably Orgmode can already understand how to detangle (jump to the right Org block) using such special comments)
add LINE and COLUMN pragmas (or #line directives) in the target language, if it supports such pragmas
do a fresh parse of Org, then ask "where does this tangled line come from?"
The last option is the most compute-intensive, but also the least messy. This is because the first two options result in peppering the tangled output with lots of custom line numbers, which can all change completely if we modify a single line near the beginning of the Org file.
How about a special commenting style that is removed from line numbers? For example we could add comments saying "BEGIN f:foo" and "END f:foo" to say that the text belongs to a particular named block. Then we can easily just search for that block name. We don't have to worry about line numbers either.
The problem with the above is we have to litter our source code output with such generated, mechanical noise. Instead, we should have a separate .loc file ("Location file") or similar which encodes this information for us when we tangle. Then the tool we use can just read this info when interpreting line number information from compiler errors.
The .loc file is what the JS world does with their "source maps". It's an external file that knows how to map a JS file into another file (the original file).
Our tangle-loop.sh can use sed to just replace the line number info with what we find in .loc. And then we can keep this stuff removed (outside of) source control.
4.4.1 Chicken-egg issues
When developing Lilac, we should be able to tangle ourselves (build shouldn't be broken). Then if we introduce some error so the build breaks, we should still be able to generate the correct .loc file with the first tangle() invocation. Then on the second invocation of tangle() we will be unable to build ourselves. In this broken state, it's important to be able to interpret the .loc file with a tool that is not native to Lilac (the understanding is that Lilac is broken). So we have to design the .loc file in such a way that any rudimentary (such as Bash) tool can understand it.
4.4.2 Reference
Noweb just uses a flag to set this for the source language. See https://man.cx/notangle(1).
See https://github.com/ghc/ghc/blob/master/utils/unlit/unlit.c for GHC/Haskell which claims the CPP understands line pragmas.
Compare with https://zyedidia.github.io/literate/manual.html. There it appears they support either injecting line number directives or pragmas as above, but also support understanding the compiler errors when they are fed into the lit program which can perform substitutions of those line numbers on the fly(?).
In Haskell there is both a LINE and COLUMN pragma. See https://downloads.haskell.org/ghc/latest/docs/users_guide/exts/pragmas.html#line-pragma.
See https://gcc.gnu.org/onlinedocs/cpp/Line-Control.html for GCC and C.
For an example, see: https://learn.microsoft.com/en-us/cpp/preprocessor/hash-line-directive-c-cpp?view=msvc-170
4.5 TODO blocks: teach "pad-consecutive-noweb" (?) boolean metadata
Allow users to disable automatic padding per-block.
4.6 TODO doc: teach "pad-consecutive-noweb" (?) boolean metadata
Allow users to set document-wide default for disabling automatic padding (which can be overridden at the block level)
4.7 TODO (Git integration) add raw git links
If there is text like 68541ad (capture unhandled edge case for parsing styled text, 2025-01-26) (hash, parenthetical with a YYYY-MM-DD date at the end), convert it to a link. You'd need to have a "default Git repo" setting somewhere, probably in the doc metadata, and Lilac would have to learn how to read out this information locally using Git CLI or something else.
This may be avoided if we can just copy out the Org-formatted info directly out of tig where we get this info. Then we can just paste out of tig directly into here (with the right link that points to GitHub for example), and we're fine. The only downside with this is if we change Git hosting providers, in which case our existing links would point to the wrong (older) provider.