sirocyl

noted computer gremlinizer

working on a @styx-os.

 

laptop.
                                                                                                     

"accidentally-vengeful telco nerd"
—Tom Scott

platform sec researcher, OS dev, systems architect, composer; Other (please specify). vintage computer/electronics nut.

I am open to tag suggestions - if there is something you want me to tag on my posts, leave a comment. <3


take a look at
this cool bug I found 🪲
discord
@sirocyl
revolt.chat (occasionally active)
@sirocyl#5128
styx linux OS project
styx-os.org/

NireBryce
@NireBryce

give me an MFD with tappable aliases at the very least.

give me a list of envvars I can paste the contents of by tapping their buttons. there's so much here that can be done with existing tools even


sirocyl
@sirocyl

(this is a bit stream-of-consciousness writing here, and a lot of it is copied from things I noted earlier.)

honestly, I've done some brainstorming on what a shell should become, and I think I have some answers.

Ideally, you can operate this entire shell from a mouse or touchscreen, or a menu interface, without having to bring up a keyboard or open a manfile into another window, because all of the manipulable objects, shell commands, and all their possible arguments, are available from the library and browser panels, and their documentation is viewable inline.

First, some definition of what a "shell" is, opposed to a "console" or a "terminal". A console is an interface by which one works with the computer. For equivalents on Windows: The local desktop is a console; Microsoft Management Console is a console in the Windows Desktop; Hyper-V Manager is a console within Microsoft Management Console.

A terminal is an interface by which serial data, usually text, is received and generated. A VT-100 is a terminal, an IBM 3270 is a terminal, as is a terminal emulator, the Windows Terminal, KDE Konsole and Xterm.

A shell is an interface by which the programming of the machine (machine language monitor, BASIC ROM, C compiler, API) is abstracted from the user, by way of manipulating objects or commands.

The key here is that all three of them are part of a user interface. A shell can run on/to a terminal to present a console to work at.

In our case, the shell is what we’re working on defining here, though elements of the terminal (especially UI) play a part here.

This is a new shell, and it should act like it.

Anything short of outright revolutionary, in terms of shells, will land us with yet another “incompatible Bash-like with Features”, like zsh, fish, dash, and others before it.

While terminal/vt100/tty serial buffer compatibility is a nice-to-have, it shouldn’t be the defining feature of the shell, it shouldn’t be the mandatory interface. We have a GUI, we should use it. Therefore, this will be a GUI application, unless it is launched from within a vt100-like/ANSI terminal session.

Make it a visual, and block-oriented agglutinative language.

I’m not talking about a node-based scripting environment (like Quartz Composer, Blender’s node graph UI, or Unreal’s Blueprint system) - I’m focusing more on a “lego”, agglutinative series of code and function blocks, which can themselves contain code and function blocks.

I’ve always had this idea of doing a shell as “Scratch, for adults.”
The gist of it is, you can freely enter commands line-by-line typed out, and they’ll be parsed into blocks transparently as you go along; or, you can attach together blocks to specify the operation that you’re performing, with transparent on-the-fly documentation and a menu or “library” of available commands to pull from.

This format would save to plain text, and be parsed into command blocks; or be saved as a compact token/AST form, and “reconstituted” on the fly.

Either way, it’s intended to be accessible to newcomers or less-experienced computer users, while not getting in the way of people who intend to program scripts for the shell like the way they’re used to. A skilled command wizard could hide away the panels and simplify the “coding block” UX to being little more than on-the-fly syntax highlighting; the code is entered by keyboard, as usual.

Reach into the desktop, and allow rich scripting through both API and UI.

This shell should also be able to comprehend and automate UI interactions, namespacing and identifying specific UI elements in terms of code, in a way that’s less liable to breakage. I’m thinking along the terms of AppleScript and Automator, on that end.

One UI puzzle that could be solved for automation tasks, is a drag-and-drop “connector” line, which you pull from the shell window to a specific UI element, and then that element is referenced into the code transparently.

It should also be able to do object introspection on the desktop model and API, where present, to provide a cohesive scripting language to wrap around both command invocation and function calls to the API.


Expanding on the visual shell.

So, circling back to the "Scratch-like" command language interface. I am not going to front - what I'm suggesting is straight up, Scratch for Adults, and without the weird "video game/turtle graphics" paradigm/model it works in.

Think of how tags work on cohost. You begin typing the tag, and a dropdown completion-box with matching tags shows up. You select or finish a tag, and that tag becomes a "chip", or a block, by itself.

I'm thinking, a similar UX for that, for selecting and entering commands, and their arguments (options, files, objects).

The names of ls, cp, rm are fine, if you know what they mean; but they are cryptic to non-UNIX people.

So first, you start with a GUI, with a text entry box.
As you begin to enter a command, or keywords from a command's documentation, that command will light up in a drop-down box below the command line.

You can also select a command from the palette on the sidebar, or "library", with the most commonly used and most recent commands up on top, and then other commands sorted by category among other things. (This palette is searchable, with a searchbar above it. The search bar pulls from command documentation, such as manfiles, too.) You drag over the command from the palette, and it "clicks" into the command line, like a block; or you can type in the command, and it'll become a block.

So, if you type in delete, remove, or erase, a drop-down selection shows up, something like:
~ > erase
(rm - delete a file)
(clear - clear the screen)

and selecting one of these, say rm would replace the "erase" in the command with it.

Now the library, on the sidebar, becomes a library of the rm command. It now contains potential command options for the active command, and the browser below it changes to contain files for the command to act on, or objects (like clipboard, windows) to be manipulated.
The library shows a list like "recursive, force, verbose" - and allows you to click and drag those over, or to type in "recursive", hit TAB, and have it put -r on the command line.

Below that, is a file tree view, the "Browser", starting from the current working directory, which is highlighted or tagged with a visual flag or bullet.
The browser presents you with options to select files to act on, either recursively or individually. Changing the working directory is easy with this view - right click the directory and "Change Directory", for instance. You don't need to delete the command you're working on to do so; it'll reflect in the shell that this is now the current directory.
For commands which work on other object types (e.g., URIs, network addresses) the Browser view below the Library will change to match. If multiple object types are possible, it is tabbed.

Finally, for scripting, one can expand the entry line into a "workpad", with the same affordances as the shell; but now you have additional commands for control flow, conditionals, parsing a return value from the called program, and the like. You can also make blocks out of other commands, and use those blocks themselves as commands, much like function definitions in Bash, except visually.

This shell internally represents all commands as an AST, and the commands are quickly parsed into tokens by which "gadgets" dispatch either syscalls like fork/execve, or program state control, and the like, internal to the shell.

A script can be saved in a couple ways and formats.

Normally, when you save the script, and it has a name, that name is now in your "library" as any other command. It saves to disk in your user directory under the shell's folder or under ~/bin, along with your other personal library items and programs. It's in a binary AST representation in this place, and is "reconstituted" when edited.

Another option, is to save the script out to a file. You can save in the same compact AST representation, or as a plain-text document, not unlike a traditional shell scripting language. When that document is loaded into the shell, it is parsed and reconstituted into the visual paradigm of connected blocks of code and commands.

You can edit the plain text version independently of the shell, in "your favorite text editor"; and it can be easily diffed against on Git and other source-based VCS.

Don't generate native code - generate tightly-wound function call lists.

A tokenized format that calls to precompiled code fragments, or “gadgets”, is better than a true codegen/JIT proper, especially for security and traceability of the code. I’m thinking like TCG from Qemu, or the “pseudo-JIT” codegen used by iSH.

A big benefit here, is that generating calls into existing code will reduce the overall complexity and security boundaries necessary to support this shell language.
You would have very performant high-level code generation without the need to generate code and violate W^X enforcement.

Finally, the lowest-level representation of code can be a more arbitrary set of functions, e.g., for file manipulation and syscalls, rather than machine-code level assembly.

Work with programs, not in spite of them.

A lot of programs go through a lot of hell to make their complicated interfaces adapt to a rigid "shell command line" structure. Sometimes, this is complicated enough to warrant writing a whole GUI that spits out a valid command line for said program.

Something like ffmpeg would definitely benefit from a “visual shell composer” approach directly, over a “bespoke GUI”.

This is something that this shell could easily take advantage of.

And one could make meta-commands, blocks that specifically change how the shell’s Library behaves, for ffmpeg itself.

Don't use "sixels" or other VTxxx visual hacks.

If we’re going to have the shell output to a vt-tty compatible serial terminal, it should respect ASCII and basic control/escape sequences, and not depend on slow, inefficient, and potentially incompatible terminal extensions.

Rather than hack around to attempt to make graphics work (and invariably spew pages of mojibake to a printer or something), we should know the terminal, and expect that graphics may be unavailable in some cases - but display native graphics (visual bitmaps/UI elements, not "sixels") in the shell when a rich enough terminal is available, where hacks aren’t necessary to make graphics and text work side-by-side.

An image - especially color - drawn in sixel escapes is significantly larger in memory than a plain binary bitmap of the same size, colors and resolution.

Make the shell document itself.

Think back to qbasic - how did you learn it? It had a robust and in-depth help system. When you pressed F1 in a keyword, it would tell you what that keyword is, what it does, how that keyword is used, and importantly, show examples. man is honestly long in the tooth, and the help command only concerns itself with builtins.

Have its language be powerful, but clean.

I feel that a robust metalanguage would do wonders to making automation and scripting accessible to both skillful systems-administrators and users with bright ideas and a tenuous grasp on the efficacies of programming.

We shouldn't go full-ham on OOP techniques with this shell, by forcing even simple scripting to be complicated and objectful with classes and inheritance. I think an imperative, sequential task engine suffices for the large end of it. After all, we are calling into programs like rm and cp - we're not taking that over.
What we do want to be object-oriented, is access to data. Said data being files on the filesystem, and objects in the API at large.
And what we also want, is a good interface to program in said batch-style language.

We also don't want weird hacks and cryptic sigils like the IF and FOR command, %~dp0 and :~3,-3 in Windows/DOS BAT, or the hackish [ command in UNIX shells. We want things like that to be as much a part of the shell language as possible, and in a way that's sensible.

Pipes and stdio redirection can flow visually, too.

It should be functionally orthogonal to bash/zsh/...

There's been projects in the past that relitigate the concept of a shell, but in doing so, sacrifice a lot of the qualities of existing shells.

a lot of this is paraphrased or quoted out of my posts in this Haiku discussion thread.


You must log in to comment.

in reply to @sirocyl's post:

been thinking about discoverability/access a lot lately and found two things I really like along these lines:

  1. Gleam (gleam.run) has a notion of labelled arguments for functions that makes naming and documentation look really intuitive. It's a really early language but I think this type of syntax is one of its better ideas, and I can imagine a shell incorporating this into a block-based visual representation quite naturally.

  2. The idea of "what is this" mode where you tap a button and enter a mode where interacting with an object pulls up documentation, examples, and so on. And making the doc format for this thing such a first-class citizen that you can embed shell programs in the docs that the user can run in a sandboxed environment

Pinned Tags