Deryks _dev

A file you didn't write

You have watched the editor put two kinds of line ending into a file that arrived with one.

No edits in this stage. It is an experiment, and it needs a file with Windows line endings. You don't need Windows to make one — printf will write whatever bytes you ask for:

cd editor
printf 'one\r\ntwo\r\n' > crlf.txt
file crlf.txt
→ crlf.txt: ASCII text, with CRLF line terminators
cat -A crlf.txt
→ one^M$
→ two^M$

\r is carriage return, byte 13, and \n is line feed, byte 10. The names are from teletypes, where one moved the print head back to the left and the other moved the paper up a line. Windows kept both; Unix kept the second.cat -A draws \r as ^M and the end of every line as$, so those two lines are the whole story: each line ends in both bytes. Six letters and two line breaks — ten bytes.

The character you can't see

Open it in the editor and click at the end of the first line:

cargo run -- crlf.txt
→ the status bar says 10 bytes
→ click after the e of one, or press End: the cursor sits one cell to the right of the e, with a gap between them, and the status bar saysLn 1, Col 5

Column five, on a three-letter line. The fourth character is \r. Rust gave it to egui as part of the text, because fs::read_to_string reads bytes and checks that they are UTF-8 and does nothing else. egui has no picture for it, but it still reserves a cell — measured: the same width as a letter, and no pixels in it. That is the gap. Your line_col is telling the truth about a character you cannot see.

Line numbers were right all along. ropey treats\r\n as a single line break, so char_to_line gave the right line even with the \r in the text. Only the column was off, and only by the one invisible character. Two libraries, two opinions about the same two bytes.

The line you add is different from the others

With the cursor still at the end of the first line, press Enter, typex, and save:

→ Enter, x, Ctrl+S
→ status: saved 12 bytes to crlf.txt

Then close the editor and look at the file again:

file crlf.txt
→ crlf.txt: ASCII text, with CRLF, LF line terminators
cat -A crlf.txt
→ one^M$
→ x$
→ two^M$

One line has lost its ^M. When you press Enter, egui inserts\n — one byte, the Unix ending, no matter what the surrounding text uses.Buffer::save then writes the buffer out exactly as it is. So the file now has two conventions in it, and neither file nor git diff will let that pass unremarked. (If your cursor landed on the other side of the invisible character, the bare $ is on a different line. Either way, one of the three has lost it.)

The fix is not to make egui insert \r\n. The buffer should hold one convention — \n, the one every Rust string function assumes — and the conversion belongs at the door, in open andsave, where the file's bytes come in and go out. That is what the next four stages build.

A name for the two endings

The editor can look at a piece of text and say which ending it uses, and four tests prove it.

This stage takes three edits: an enum, its tests, and then a repair the compiler will ask for.

First — the type. It goes between list_dir and the Pendingenum from lesson 15, which it resembles:

editor/src/main.rsbetween list_dir and the Pending enum

    Ok(entries)
}

/// How a file marks the end of a line. Unix writes `\n`; Windows writes `\r\n`.
/// The buffer only ever holds `\n` — this remembers what the file had, so
/// saving can put it back.
#[derive(Clone, Copy)]
enum LineEnding {
    Lf,
    CrLf,
}

impl LineEnding {
    /// Which ending the text uses, judged by its first line break. Text with
    /// no line break gets `Lf`, which is what this editor writes anyway.
    fn detect(text: &str) -> Self {
        match text.split_once('\n') {
            Some((first_line, _)) if first_line.ends_with('\r') => Self::CrLf,
            _ => Self::Lf,
        }
    }

    /// The name shown in the status bar.
    fn label(self) -> &'static str {
        match self {
            Self::Lf => "LF",
            Self::CrLf => "CRLF",
        }
    }
}

/// Something the editor is waiting for permission to do.

Two variants, no data in either, the same #[derive(Clone, Copy)] asPending. Then an impl block with two functions.

detect has no self, so it is called on the type — LineEnding::detect(text), the way Buffer::open is. It asks one question: does the first line end in \r? split_once('\n')returns an Option holding the text before the first \n and the text after it. If there is a first line and it ends in \r, the file isCrLf. The match guard — Some(…) if … — is the one from lesson 5's ErrorKind::NotFound. Everything else, including text with no line break at all, is Lf, which is what this editor would write anyway.

label takes self by value, which is fine for a Copy type, and returns &'static str. You saw'static in lesson 8. A string literal lives inside the compiled binary for the whole run of the program, so a reference to one never has an owner that could go away, and 'static is the name for that.

Second — four tests. Put them above the list_dir tests:

editor/src/main.rsin mod tests, above the list_dir tests

        assert_eq!(editor.unsaved(), 0);
    }

    #[test]
    fn detect_sees_lf() {
        assert_eq!(LineEnding::detect("one\ntwo\n"), LineEnding::Lf);
    }

    #[test]
    fn detect_sees_crlf() {
        assert_eq!(LineEnding::detect("one\r\ntwo\r\n"), LineEnding::CrLf);
    }

    #[test]
    fn detect_defaults_to_lf_without_a_line_break() {
        assert_eq!(LineEnding::detect("one"), LineEnding::Lf);
    }

    #[test]
    fn detect_goes_by_the_first_line_break() {
        assert_eq!(LineEnding::detect("one\ntwo\r\n"), LineEnding::Lf);
    }

    #[test]
    fn list_dir_finds_the_crate_manifest() {

cargo test

It doesn't compile. The first two errors, verbatim; the same pair repeats for each of the four tests:

error[E0369]: binary operation `==` cannot be applied to type `LineEnding`
   --> src/main.rs:753:9
    |
753 |         assert_eq!(LineEnding::detect("one\ntwo\n"), LineEnding::Lf);
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |         |
    |         LineEnding
    |         LineEnding
    |
note: an implementation of `PartialEq` might be missing for `LineEnding`
   --> src/main.rs:230:1
    |
230 | enum LineEnding {
    | ^^^^^^^^^^^^^^^ must implement `PartialEq`
help: consider annotating `LineEnding` with `#[derive(PartialEq)]`
    |
230 + #[derive(PartialEq)]
231 | enum LineEnding {
    |

error[E0277]: `LineEnding` doesn't implement `std::fmt::Debug`
   --> src/main.rs:753:9
    |
753 |         assert_eq!(LineEnding::detect("one\ntwo\n"), LineEnding::Lf);
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::fmt::Debug` is not implemented for `LineEnding`
    |
    = note: add `#[derive(Debug)]` to `LineEnding` or manually `impl std::fmt::Debug for LineEnding`
help: consider annotating `LineEnding` with `#[derive(Debug)]`
    |
230 + #[derive(Debug)]
231 | enum LineEnding {
    |

What a type has to be able to do before you can test it

assert_eq! needs two things from a type. It has to compare the two sides with ==, and if they differ it has to print both, so you can see what went wrong. A type you write yourself can do neither until you say so. PartialEqis the trait behind ==; Debug is the trait behind{:?}, which you used in lesson 17's assertion message. Pendingnever needed either, because no test ever compared one.

derive writes the implementation for you, and for an enum with no data it is exactly what you would write by hand: two values are equal if they are the same variant, and printing one prints its name. The compiler told you the fix twice over; third — take it:

editor/src/main.rsthe line above enum LineEnding

/// saving can put it back.
#[derive(Clone, Copy)]
#[derive(Clone, Copy, PartialEq, Debug)]
enum LineEnding {

cargo test
→ 26 passed

The buffer remembers what it was given

Opening crlf.txt shows CRLF in the status bar, the cursor sits against the last letter, and the column is right.

This stage takes five edits. The first three go in together and won't build until the fourth; the fifth is a test.

First — a field:

editor/src/main.rsthe Buffer struct

/// One open file.
struct Buffer {
    path: PathBuf,
    text: Text,
    /// How the file on disk ends its lines. The text above only ever holds `\n`.
    line_ending: LineEnding,
    status: String,

Second — set it. Buffer::open already has the file's text in aString called text. Detect the ending from that, then replace every \r\n with \n before the rope ever sees it:

editor/src/main.rsin Buffer::open, between the match and the Self

        };
        let line_ending = LineEnding::detect(&text);
        let text = text.replace("\r\n", "\n");
        Self {
            path: PathBuf::from(path),
            text: Text::new(&text),
            line_ending,
            status,

The second new line shadows text with a new String — the same trick main uses on paths. Order matters: detecthas to look at the text before replace takes the evidence away.replace builds a fresh String even when there is nothing to replace; on the 2.3 MB file from lesson 11 that is about a tenth of a millisecond, once, at open. From here on the rope holds \n and nothing else.

Third — show it. The status bar gets one more segment, between the saved state and the highlighting state:

editor/src/main.rsin fn ui, the status bar

            ui.label(format!(
                "{}  ·  {} bytes  ·  {saved}  ·  {highlighting}  ·  line_col {:.0} µs  ·  {:.2} ms/frame",
                "{}  ·  {} bytes  ·  {saved}  ·  {}  ·  {highlighting}  ·  line_col {:.0} µs  ·  {:.2} ms/frame",
                self.position,
                self.buffers[self.active].text.len(),
                self.buffers[self.active].line_ending.label(),
                self.line_col_us,

cargo test

error[E0063]: missing field `line_ending` in initializer of `Buffer`
   --> src/main.rs:628:9
    |
628 |         Buffer {
    |         ^^^^^^ missing `line_ending`

Not in open — you set it there. Line 628 is buffer_with, the test helper from lesson 10 that builds a Buffer by hand, and a struct literal must name every field. Fourth — give it one:

editor/src/main.rsin mod tests, buffer_with

            path: PathBuf::from("test.rs"),
            text: Text::new(text),
            line_ending: LineEnding::Lf,
            status: String::new(),

Fifth — a test that a Windows file arrives as bare newlines. It needs a real file, which means a helper that writes one into the temporary directory, the way lesson 17's last test did. The helper returns the path, so each test can open the file it asked for:

editor/src/main.rsin mod tests, between the detect tests and the list_dir tests

        assert_eq!(LineEnding::detect("one\ntwo\r\n"), LineEnding::Lf);
    }

    /// A file on disk with Windows line endings, for the round-trip tests.
    fn crlf_file(name: &str) -> PathBuf {
        let dir = env::temp_dir().join("editor-lesson-18");
        fs::create_dir_all(&dir).unwrap();
        let path = dir.join(name);
        fs::write(&path, "one\r\ntwo\r\n").unwrap();
        path
    }

    #[test]
    fn a_crlf_file_is_held_with_bare_newlines() {
        let b = Buffer::open(&crlf_file("held.txt"));
        assert_eq!(b.text.as_str(), "one\ntwo\n");
        assert_eq!(b.line_ending, LineEnding::CrLf);
    }

    #[test]
    fn list_dir_finds_the_crate_manifest() {

cargo test
→ 27 passed
cargo run -- crlf.txt
→ the status bar says 8 bytes and CRLF
→ press End on the first line: the cursor sits against the e, and the status bar says Ln 1, Col 4

Eight bytes, not ten: the count is of the buffer, which no longer holds the two\rs. Column four: there is no invisible character to stand on.

Writing back what you were given

A test proves that saving a CRLF file changes nothing on disk — and first, that it currently does.

This stage takes five edits: a test that fails, an import, a method, the save that uses it, and the save that compiles.

First — the test, straight after the one from stage 3. It writes the file, opens it, saves it without touching it, and reads the bytes back:

editor/src/main.rsin mod tests, after a_crlf_file_is_held_with_bare_newlines

        assert_eq!(b.line_ending, LineEnding::CrLf);
    }

    #[test]
    fn saving_a_crlf_file_writes_it_back_unchanged() {
        let path = crlf_file("unchanged.txt");
        Buffer::open(&path).save().unwrap();
        assert_eq!(fs::read_to_string(&path).unwrap(), "one\r\ntwo\r\n");
    }

    #[test]
    fn list_dir_finds_the_crate_manifest() {

cargo test

test tests::saving_a_crlf_file_writes_it_back_unchanged ... FAILED

failures:

---- tests::saving_a_crlf_file_writes_it_back_unchanged stdout ----

thread 'tests::saving_a_crlf_file_writes_it_back_unchanged' panicked at src/main.rs:798:9:
assertion `left == right` failed
  left: "one\ntwo\n"
 right: "one\r\ntwo\r\n"

That is the whole bug in two lines, and it is why the test compares strings rather than Vec<u8>s: Debug prints a string with its escapes showing, so the \rs that are missing on the left are visible as text. Open a Windows file, save it, and it is a Unix file.

One function, two kinds of answer

Second — an import:

editor/src/main.rsthe imports at the top

use ropey::Rope;
use std::borrow::Cow;
use std::char;

Third — the method that produces what goes to disk:

editor/src/main.rsin impl LineEnding, after label

            Self::CrLf => "CRLF",
        }
    }

    /// The text as it should be written to disk. For `Lf` that is the text
    /// itself, borrowed; for `CrLf` it is a new string with every line break
    /// rewritten.
    fn apply(self, text: &str) -> Cow<'_, str> {
        match self {
            Self::Lf => Cow::Borrowed(text),
            Self::CrLf => Cow::Owned(text.replace('\n', "\r\n")),
        }
    }
}

Cow is clone on write, and it is an enum with two variants:Borrowed holds a reference, Owned holds a value. For a&str the owned form is a String. It exists for exactly this situation — a function that usually has nothing to do and can hand back what it was given, but sometimes has to build something new. The two cases have different types, &str and String, and Cowis one type that holds either.

Why not return a String every time? Because most files areLf, and for them that would copy the entire buffer on every Ctrl+S to produce a string identical to the one you already had. With Cow, theLf arm costs nothing — no allocation, no copy — and only theCrLf arm pays for the replace.

The '_ in Cow<'_, str> says the borrowed variant, if that is what comes back, borrows from the argument. It is the same lifetime the compiler was filling in silently for Buffer::name back when it returned&str; Cow just makes you write the placeholder.

Fourth — use it in save. The obvious way:

editor/src/main.rsin Buffer::save

    fn save(&mut self) -> io::Result<()> {
        fs::write(&self.path, self.text.as_str())?;
        let contents = self.line_ending.apply(self.text.as_str());
        fs::write(&self.path, contents.as_bytes())?;
        self.text.mark_saved();
        self.dirty = false;
        self.status = format!("saved {} bytes to {}", self.text.len(), self.path.display());
        self.status = format!("saved {} bytes to {}", contents.len(), self.path.display());
        Ok(())

cargo test

error[E0502]: cannot borrow `self.text` as mutable because it is also borrowed as immutable
   --> src/main.rs:311:9
    |
309 |         let contents = self.line_ending.apply(self.text.as_str());
    |                                               --------- immutable borrow occurs here
310 |         fs::write(&self.path, contents.as_bytes())?;
311 |         self.text.mark_saved();
    |         ^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
312 |         self.dirty = false;
313 |         self.status = format!("saved {} bytes to {}", contents.len(), self.path.display());
    |                                                       -------- immutable borrow later used here

The borrow that outlived its usefulness

Read the three markers in order. contents might beCow::Borrowed, in which case it is a borrow of self.text — line 309. mark_saved needs &mut self.text — line 311. And the borrow is still alive at line 311, because line 313 uses contentsafterwards. Many readers or one writer, never both at once: lesson 1's rule, and this time the reader is a Cow.

The compiler doesn't know which variant apply returned, and it doesn't matter — a Cow that can be a borrow is treated as one. The fix is to finish with contents before mark_saved runs. Fifth — take the length first, and the borrow ends there:

editor/src/main.rsin Buffer::save, again

        fs::write(&self.path, contents.as_bytes())?;
        let written = contents.len();
        self.text.mark_saved();
        self.dirty = false;
        self.status = format!("saved {} bytes to {}", contents.len(), self.path.display());
        self.status = format!("saved {written} bytes to {}", self.path.display());
        Ok(())

cargo test
→ 28 passed
cargo run -- crlf.txt
→ Ctrl+S: saved 10 bytes to crlf.txt

Ten bytes written, eight in the status bar. The two numbers mean different things now — what is on disk and what is in the buffer — and for a Windows file they differ by one byte per line.

contents.as_bytes() works because a Cow<str>can be used wherever a &str can: it dereferences to one, the way a String does. fs::write wants bytes rather than text, andas_bytes is the same call you would make on either.

The line you typed gets the file's ending

Stage 1's experiment, repeated, leaves crlf.txt with one kind of line ending.

Stage 4 proved the untouched round trip. The bug in stage 1 needed an edit, so this stage takes two edits, both tests, and then you repeat the experiment.

First — the edit that started the lesson. When you press Enter, egui callsinsert_text on your Text with a bare \n; the test makes the same call directly. insert_text belongs to theTextBuffer trait, which mod tests hasn't imported, so the call is written with the trait's full name in front:

editor/src/main.rsin mod tests, after saving_a_crlf_file_writes_it_back_unchanged

        assert_eq!(fs::read_to_string(&path).unwrap(), "one\r\ntwo\r\n");
    }

    #[test]
    fn a_line_typed_into_a_crlf_file_gets_a_crlf_ending() {
        let path = crlf_file("typed.txt");
        let mut b = Buffer::open(&path);
        egui::TextBuffer::insert_text(&mut b.text, "x\n", CharIndex(4));
        b.save().unwrap();
        assert_eq!(fs::read_to_string(&path).unwrap(), "one\r\nx\r\ntwo\r\n");
    }

    #[test]
    fn list_dir_finds_the_crate_manifest() {

Second — two tests for apply on its own, next to the ones fordetect. The first checks the promise that made Cow worth using: for Lf, nothing is copied. matches! isassert!'s companion for enums — true if the value fits the pattern, andCow::Borrowed(_) is a pattern:

editor/src/main.rsin mod tests, after the detect tests

        assert_eq!(LineEnding::detect("one\ntwo\r\n"), LineEnding::Lf);
    }

    #[test]
    fn apply_lf_borrows_the_text_as_it_is() {
        assert!(matches!(LineEnding::Lf.apply("a\nb\n"), Cow::Borrowed(_)));
    }

    #[test]
    fn apply_crlf_rewrites_every_line_break() {
        assert_eq!(LineEnding::CrLf.apply("a\nb\n"), "a\r\nb\r\n");
    }

    /// A file on disk with Windows line endings, for the round-trip tests.

cargo test
→ 31 passed

Now the experiment. Make the file again, open it, and do exactly what you did in stage 1 — End on the first line, Enter, x, Ctrl+S:

printf 'one\r\ntwo\r\n' > crlf.txt
cargo run -- crlf.txt
→ End, Enter, x, Ctrl+S: saved 13 bytes to crlf.txt
file crlf.txt
→ crlf.txt: ASCII text, with CRLF line terminators
cat -A crlf.txt
→ one^M$
→ x^M$
→ two^M$

Thirteen bytes: three lines of one letter or three, each with a two-byte ending. The line you typed has the same ending as the two you didn't, and file has nothing to add.

Two endings this lesson does not handle. A lone \r, which classic Mac OS used until 2001, is neither detected nor normalised. And textpasted into the editor goes into the buffer as it is — egui only normalises pasted line breaks for single-line fields — so a paste from a Windows clipboard can put a\r into a buffer that is supposed to hold none. Both are onereplace away, in insert_text; neither has come up yet.

rm crlf.txt