Modularization

Following previous post it is a good moment to talk about ongoing modularization of Yi.

Rewind to a moment I started contributing to Yi: 2012. After two years of using vim I found a bug in it. It wasn’t anything too serious like a crash or corruption of user text, just some undocumented inconsistency in behavior. I thought, well, it’s open source why not try to fix it? This was the first time I looked at vim’s source code and was completely overwhelmed. Hundreds of thousands of lines of C. It’s not unprecedented, of course, but it’s A LOT. At that point I’ve only seen a codebase comparable in size one time at work, but that was, while larger, much more modular.

After some hours of trying to find a relevant place in vim, I was sufficiently lost to arrive at a question “Is there a vim-like editor that is written simpler?”. I remember looking at Kate, Yzis and Yi. Was I going through a list of vi emulations in reverse lexicographical order? Probably. Was I using KDE at the time? Definitely. Anyway, Yi seemed interesting because it had about 20 thousands lines of code and had multiple frontends (Terminal, Gtk and Cocoa) and multiple keymaps (vim and emacs, naturally).

This is how Yi was split into packages, or rather into a library part and an executable part within one cabal project at the time:

> cloc yi/src/library
     158 text files.
     158 unique files.
      29 files ignored.

github.com/AlDanial/cloc v 1.72  T=0.49 s (262.7 files/s, 57554.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Haskell                        129           4514           4647          19103
-------------------------------------------------------------------------------

> cloc yi/src/executable
       1 text file.
       1 unique file.
       0 files ignored.

github.com/AlDanial/cloc v 1.72  T=0.01 s (95.5 files/s, 1241.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Haskell                          1              5              3              5
-------------------------------------------------------------------------------

So basically one package. Interestingly, Yi had a custom prelude and that gathered lots of complaints over the following years until Mateusz finally removed it in 2014. And yet these days custom preludes seem to be all the rage? By contrast, top result in google for “custom prelude” from 2012 is this answer by Don Stewart saying not to do it.

Of course, I was not the first person to come up with the idea to split emacs emulation from vim one and terminal interface from GUI. In fact, here is the quote from 2012 README:

We also want to simplify the core Yi package to make it more accessible, splitting some parts into several packages.

Back then it was significantly harder to do just because of the tooling. Not only there was no stack at the time, cabal sandboxes were not a thing until late 2013. I fondly remember the character-building days of nuking your global ghc and cabal directories.

But thanks to cabal and later also stack folks things were steadily improving since.

The tooling situation was not the only difficulty, Yi had several circular references between modules. One by one, we untangled these and split some libraries potentially useful outside of yi.

yi-rope is actually used by our friendly competitor rasa and haskell-lsp.

oo-prototypes still blows my mind five years later. Yi was my intro to Haskell after a HelloWorld and I was basically greeted by a module saying “here we implement OOP inheritance in 7 lines out of thin air”.

yi-language I’m not too happy about, because we split it not because it’s a self-contained thing, but just to isolate alex-related stuff that was killing incremental compilation. I’m hoping to reshuffle this part, so that pieces of yi-language like Yi.Buffer.Basic and Yi.Region end up in yi-core and everything alex-related lives in yi-alex-utils-or-something and becomes entirely optional, that is you will be able assemble an editor without a dependency on alex.

Some time later stack came about and made it easy to work with multiproject repos and we finally split all the frontends and keymaps into separate projects.

So this is how project structure looks now:

Project                     Lines of haskell code
yi-core                     10335
yi-dynamic-configuration    81
yi-frontend-pango           1566
yi-frontend-vty             407
yi-fuzzy-open               214
yi-intero                   140
yi-ireader                  124
yi-keymap-cua               147
yi-keymap-emacs             643
yi-keymap-vim               4669
yi-language                 803
yi-misc-modes               449
yi-mode-haskell             1246
yi-mode-javascript          601
yi-snippet                  375

So in my mind the next thing in modularization of Yi is moving alex-powered highlighting into a plugin while making yi-core expose some general interface. It is already possible to make syntax highlighting without alex, e.g. I have rainbow parens mode in my config where actual parsing is done by regex-applicative, but it doesn’t feel like a first class citizen.

Finally, if this story was interesting to you, you’re very welcome to join the development!

Do you care about how pretty does editor look? Make a new shiny frontend!

Have an idea about a new crazy ergonomic control scheme? Try it out as a new keymap for Yi.

Maybe you’re interested in optimizing haskell code? Yi has plenty of that.

In any case, don’t hesitate to file an issue, make a PR or chat.