Hacker News new | threads | past | comments | ask | show | jobs | submit DanielBMarkham (43825) | logout
Why Emacs has buffers (masteringemacs.org)
149 points by susam 16 hours ago | flag | hide | past | favorite | 73 comments

Small correction:

> Emacs is not bifurcated, but a lapse of judgment 40 years ago could’ve made it so

Emacs uses a gap buffer because TECO used a gap buffer (before it became Emacs, it was just gene ciccarelli’s TECO init file) and that decision goes back 60 years

I don't think this is about using a gap buffer specifically, but rather about making buffers first-class objects. With respect to the line you quote, what matters is not the specific implementation of a buffer which is chosen, but the fact that code works in terms of buffers rather than strings.

Thanks. That (lack of) distinction of course was also in TECO from the beginning, but then again, though I didn't start using TECO myself until the late 70s, even then it wasn't common to think of strings the way they are thought of today.

The most common machines at the time had all sorts of strings and buffers of various character widths (on a single machine) and I remember back then I often encountered and used the term "string" in a more abstract way to refer to contiguous set of symbols rather than the more concrete way it is used today that was starting to evolve at that time.

This conflates the user concept of a buffer with the gap buffer implementation emacs happens to use.

This is a pity, because buffers as a user sees them are a strength of emacs, the article covers several of the reasons for this. Other editors have files which don't happen to be saved to disk or have a name, and that might sound like the same idea as a buffer but in practice it isn't, and emacs does a lot of neat stuff because it has buffers.

But this gets folded in with a sort of handwavy argument about the gap-buffer enabling a certain kind of low-level programming which I don't view as at all justified. It's a historical accident of emacs that it uses an adequate-but-by-no-means-optimal data structure.

We're talking about lisp here, it's high level. Providing a user abstraction of a string with a cursor into it, and some line information, that's basic.

The reason Emacs is built on gap buffers is because of TECO, as a sibling comment points out, and we shouldn't confuse the practical impossibility of moving emacs over to something like a rope, with the viability of having a programmable editor with good user ergonomics which does use a rope, or some other enriched data structure with more useful qualities than the gap buffer.

> This conflates the user concept of a buffer with the gap buffer implementation emacs happens to use.

I did not conflate these things at all. Gap buffers, etc. are irrelevant; a boring implementation detail. Unfortunately, _most_ editors leave it at that, or with a thin veneer on top of it, and call it a day. That's why I go to great pains to elucidate why "string-based" editing is a Bad Idea, and why Emacs _does not_ make it a first-class citizen in elisp. I dedicate the rest of the article explaining why Emacs does not operate on such a low level; that in Emacs, in fact, the buffer and elisp are twinned, and that almost all of Emacs operates on the level of a "buffer" as that concept exists in Emacs and Emacs lisp.

In light of what you've said, I reread the What Of Strings section a few times.

I've gone from misunderstanding it to not understanding it, which is an improvement I suppose. Because you conclude by saying that the buffer is the same as the string, so that makes the string-based versus buffer-based comparison feel like a matter of terminology. If that section isn't about some tight matching of the data structure to the user experience (including programming) then I don't understand it.

> Emacs is not bifurcated, but a lapse of judgment 40 years ago could’ve made it so: a complex, string-fiddly world for Serious Programmers; and a safe, boring, but disjointed, world that “users” are expected to use.

If the above isn't about the gap buffer (I don't dispute that, you wrote it) then I don't know what's being referred to, maybe that could be bolstered a bit for those of us who don't have a detailed knowledge of emacs history.

> But this gets folded in with a sort of handwavy argument about the gap-buffer enabling a certain kind of low-level programming

Did we read the same thing? The article explicitly disclaims the underlying data structure as irrelevant!

Fancier programmers may talk about ropes and gap buffers as a way of optimizing how you store and modify text, and yet I must insist that this is still not a good enough answer to explain what a buffer is in Emacs. It’d satisfy the technical answer, but not the philosophical one.

The words "gap", "data structure", etc. don't appear at all in the article after that.

For those (like me) being stuck on "skeuomorphic" in the second sentence:

> A skeuomorph is a derivative object that retains ornamental design cues (attributes) from structures that were necessary in the original. Skeuomorphs are typically used to make something new feel familiar in an effort to speed understanding and acclimation. They employ elements that, while essential to the original object, serve no pragmatic purpose in the new system. Examples include pottery embellished with imitation rivets reminiscent of similar pots made of metal and a software calendar that imitates the appearance of binding on a paper desk calendar.

(from https://en.wikipedia.org/wiki/Skeuomorph)

If I may, I'd like to mention a text editor that I am making as an example of relevant skeuomorphism. It's called Tentacle Typer.

It makes regular ol' .txt files in mostly the traditional way, but you write text documents as an eldritch tentacle monster with a magic mechanical typewriter inside a steampunk sandbox.


That some serious... hem... serious... well, I'm not sure of what I've seen but that's really serious ! :-)

I have been doing my best!

As a person who collects text editors:

Wow. Thanks for sharing.

My pleasure. I hope it earns a place in your collection when I'm ready to release. (Trying my best for a demo this month.)

I think one of the coolest features is that you learn different spells depending on what you write.

For example, if you write horror you'll learn spookier spells than if you write romance or technobabble.

https://twitter.com/LeapJosh/status/1519398797079625733 and then you create rituals using those spells powered by yet more writing.

That is so cool!

Thank you :D

> > Skeuomorphs are typically used to make something new feel familiar in an effort to speed understanding and acclimation.

Oh my goodness this word applies to a lot of mainstream programming language design.

For those (like me) who couldn't go a day without seeing a dozen articles about skeuomorphism just a few years ago. I'm here for you.

(I don't mean to detract from the parent comment, which is helpfully defining an unintuitive term, I'm just happy to see the conversation has moved on!)

Skeuomorphic was quite the buzzword back near the dawn of the iPhone

The classic example being the Banana shaped phone receiver icon on android and ios. There are People alive today only know that shape as the call icon and never use that type of land line.

At the moment of posting, I am alive, and not only know the shape as that of the handset I grew up using, but have such a phone in my kitchen.

Ambiguities of English leaving out determiner or be verb.

Salivatingly fascinating.

So, onomatopoeia for the UI?

Not quite? Skeuomorphism is the usage of design elements that in the past were necessary. Many people know the word now due to the UI interface trend in the early 2000s. Skeuomorphism has meaning outside of UIs though. You can see it in your daily life. Common examples are laminate doors that have fake joinery, rivet patterns on a ceramic flower pot, or the lattice of a French window.


I think keyboards on smartphones are an example: they tend to resemble regular, physical desktop keyboards, rather than just a grid of letters. It seems like that's the main reason for the 'A' to be offset from the 'Q'. (Whereas, say, a Blackberry's keyboard was a grid of keys; and entering the symbol mode of smartphone keyboards is also a grid).

Well, colour me sproused.

> Emacs is not bifurcated, but a lapse of judgment 40 years ago could’ve made it so: a complex, string-fiddly world for Serious Programmers; and a safe, boring, but disjointed, world that “users” are expected to use. That’s how most editors and IDEs work, and you have to pay the piper if you want to join the Serious Programmers... But not so in Emacs. You see, in Emacs, the buffer you see and interact with on your screen is the string.

Can someone expand on this? I don't understand it.

If Emacs had

    (defun kill-paragraph (string cursor)
        ...) ; resulting in a new string/cursor
its use would look nothing like the equivalent editing operation, and the user and Emacs Lisp worlds would be nearly incommensurate. You might as well have a plugin system!

To expand on some of the implicit knowledge here: it's helpful to know that Emacs user-level commands are just Lisp functions bound to keys (e.g. one could bind kill-paragraph to C-M-k). The command an end-user binds to use interactively is the exact same command that a programmer would use when writing a program.

If things were bifurcated, then as morelisp notes there would be one suite of functions for complex string manipulation, and there would be some binding of keys to commands, and there would be some glue code between them which would have to be written for anything exposed to the user. Which means someone would have to write that glue, someone would have to make choices about it, someone would have to maintain it. That would have required quite a lot of activation energy to move from being a user to being a programmer, as Mr. Petersen points out in his excellent entry.

Instead, in Emacs users can program and programmers can use. Famously, secretaries learned to program Emacs without any guidance or prompting! There is no artificial boundary between user and developer. This is very similar to how Excel enables business users to develop solutions, to be honest, but an order of magnitude more powerful (because Emacs exposes so much more to the user-programmer than does Excel).

I think a better way to describe Emacs buffers is: Emacs has a structure called buffer that logically contains a string and various fields useful for use in a text editor (e.g. to mark positions in the string) and many, many functions working on such a struct that keep the additional fields in sync with the string content (e.g. moving marked positions when characters are inserted) and are geared towards using that structure in a text editor.

(the ‘keep in sync’ part is the reason the additional fields are part of the buffer. Also, the actual implementation may use a gap buffer, rope, or whatever for the string, but that is an implementation detail)

In other words: the Emacs buffer, together with those functions, implements the backend of a text editor. Front-ends could be programming languages calling those functions or UIs that call those functions in response to user key presses, and may show buffer contents on the screen. The standard emacs UI is a mix of the two: a lisp that can call those functions and a binding of key presses to functions (either built-in ones or ones defined in that lisp) to call.

6,477 LOCs.

A wasted an opportunity to implement it using 7 or 8 microservices.

I think this is sarcasm. But talking to proponents of microservices I am not sure. Can you confirm please

Haha. It is.

But looking at the code made me think how we are capable of shipping software products that are hundred of thousand of lines of code and then we have people trying to break things some much and we end up in a spaghetti of microservices and their connections.

Nice to see that some code I wrote almost 15 years ago is still in use

For some extra fun, click on "Blame" and look at the age of commits on individual lines.

Pretty neat for a piece of software that's been around for 37 years.

The git history goes quite far back too. The first version and the current version look quite different.


I find the Emacs source code pretty nicely structured and easy to understand (the good commenting helps), especially for something so old.

I thought eMacs was lisp.

Depends on the Emacs, but GNU Emacs has a low-level core written in C with most high-level functionality implemented in various Elisp packages.

And to clarify, in Emacs "move the cursor forward one paragraph" or "kill (~cut) this text" counts as high-level functionality.

Sure, I meant things like interfacing with the OS, the Elisp interpreter itself, performance-sensitive tasks like redisplay, etc.

No, those were just slightly bigger iMacs.


“Buffer” is just as bad or good a name as “file” or “document”. Files and documents can be in-memory and temporary, and the primary purpose of an Emacs buffer arguably isn’t buffering.

> the primary purpose of an Emacs buffer arguably isn’t buffering.

Until you save the buffer to disk, that's exactly what it's doing.

I wouldn't consider an index of my email, or an ongoing IRC conversation, or a calculator to be a file or a document. Buffer is certainly a better name. There might be an even better name, but I don't know what a better name for "visual output from anything" is. Monitor?, but that would be kind of weird because people would think of the hardware.

Another name for the API provided by "buffet" could have been "console" (combination of monitor+keyboard: read/write API).

In the most general sense, “Document” is a better name, because more people understand what a document is than understand what a “buffer” is.

Maybe not for emacs?

You can have buffers that are views of many different things: directories, shells, REPLs, email boxes, etc. People might not know what a buffer is, but I think they'd be surprised to see those "in" a document, which might be worse.

"Document" can be puzzling when working on a directory.

A "buffer" is a transition zone the user can work/interact with (and sync to it's source if any)

Designs these days try to hide the distinction between volatile memory ("RAM") and mass storage ("disk"), so buffers can only be more and more an alien concept for common users, if not a design defect.

However, I think it doesn't really matters, because people who choose to use Emacs (or Vim) - even among developers - are much more likely to be aware of those "low level" details or understand and adapt easily. So how they are named doesn't matters more than the odd "frame" term Emacs uses instead of "Windows".

What happens if you open two buffers to view two parts of the same document (easy in Emacs, not possible in lots of editors)? If you switch focus or close one, what are you manipulating? Not the document.

You could choose a different name (view?) but it's a different abstraction compared to a "document".

Did you mean two windows in the same buffer? Or did you mean opening the same file in two buffers, each attached to its own window?

Off topic but …

> Most people have heard of files and documents in real life

I don’t think so. I’m late 30s and I learned of files and folders on windows long before I was introduced to the physical concept.

I still feel a small tickle in my funny bone when I see a real life version of the folder icon in an office.

Have you seen the little 3d-printed versions of the “save” icon? Some people call them “floppy disks” for unknown reasons. :)

Funny how there’s two levels of obsolescence here: I’ve used plenty of floppies, but none of them were actually floppy; I’ve had 8-inch floppies waved at me but I don’t think I’ve ever seen a powered-on 8-inch drive. (Yes, I’ve heard the nuclear launch story.)

I only used 5 1/4 and 3 1/2 floppy disks, and I assume you refer to the 3 1/2 ones as the cover on those was hard. I always thought the 'floppy' part was to refer to the disk itself, which, even on the 3 1/2 ones was flexible, but now you make me question that and perhaps they name just stuck even if they stopped being floppy? I never thought of this because in my country they were referred to as "diskettes" , a word that didn't imply (to me anyway) any flexibility on them.

It was indeed a reference to the physical substrate of the disk, not the bendability of the media per se. They were still truly floppy.

I think I stopped calling them floppies when we started using 3.5" disks and just called them disks. Because the 5.25's were floppy on the outside, but the 3.5's weren't.

The inside is floppy though. Disk implies solid in my world.

I'd venture to guess everyone here knows what YouTube is even if they never use it. I wonder how many (even on this list) have seen a CRT?

(I'm guessing the answer is "more than I think...")

I catch myself asking questions like this, but then I remember how much tech from before my lifetime I can identify. Carriages, crank-started automobiles, biplanes, muskets, fire-starting bows, spinning wheels, washboards, etc.

How do I know what it looks like to press a wax seal with a ring? Or send a message with a telegraph, or climb the rigging of a square-rigged ship?

I know there are people who carefully avoid any media that isn't obviously by and about the present moment, and they'll continue to be clueless about the past like they always have been, but there will also be plenty of people who can identify a CRT into the 22nd century.

I was going to agree "more than you think", but in tech years I'm also old. Maybe people have seen CRT tv's in parents' or grandparents' houses.

As someone who has used multiple CRT's and both 5.25" and 3.5" floppy disks, this thread makes me feel old.

I'm 33. The first time I felt truly old is when in Grown Ups (a comedy) the kid sees a CRT TV and asks "Dad, what is the box behind the TV for?".

> even if they never use it

Who would that be?

as you can see ive lost a case. have you seen my upper case by chance?

Must have been pretty hard to carry around a type case the weight of a wardrobe.

A pretty bad article to be honest, just creating confusion.

Emacs buffers are not the reason why functions like forward-paragraph exists.

That’s just normal abstraction (in programming).

> Emacs buffers are not the reason why functions like forward-paragraph exists.

A buffer is essentially a tuple

    (contents, point, mark, definition of a "paragraph", ...)
So the article is correct - without buffers 0-ary (or implicitly 1-ary, since all evaluation is in the context of a buffer) functions like `forward-paragraph` could not exist. And without a wide library of such functions, the correspondence between normal editing commands and idiomatic Emacs Lisp wouldn't exist, and most benefits of Emacs would be lost.

Thank you! This single comment explained why Emacs had buffers much more concisely than the article did (although I still liked the article).

I thought they were called buffers because they buffed and polished your text.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact