⚡ Zig Guide LiveUnofficialbut fully verified
✓ Zig 0.17.0-dev.1503+1f1bee62eOn an older Zig?

The Lists in std Are Intrusive

If you have read a Zig tutorial about linked lists, it almost certainly showed this:

// Does not compile on current Zig.
var list = std.SinglyLinkedList(i32){};
var node = std.SinglyLinkedList(i32).Node{ .data = 5 };
list.prepend(&node);

std.SinglyLinkedList is no longer a type function and its nodes no longer carry a payload. On current master it is a plain type, and its node holds nothing but a next pointer:

var list: std.SinglyLinkedList = .{};
const std = @import("std");

/// The list node lives *inside* the thing being listed. That is what intrusive
/// means: the container does not wrap your value, your value contains the
/// container's bookkeeping.
const Task = struct {
    name: []const u8,
    priority: u8,
    node: std.SinglyLinkedList.Node = .{},

    /// Given a pointer to the embedded node, recover the struct it sits in.
    ///
    /// `@fieldParentPtr` subtracts the field's offset from the pointer. It is
    /// checked at compile time: the field name must exist on the result type
    /// and have the pointee's type, so the one dangerous-looking line in this
    /// file cannot silently point at the wrong thing.
    fn fromNode(node: *std.SinglyLinkedList.Node) *Task {
        return @fieldParentPtr("node", node);
    }
};

/// A doubly linked version, to show the same shape with a second list type.
const Job = struct {
    id: u32,
    node: std.DoublyLinkedList.Node = .{},

    fn fromNode(node: *std.DoublyLinkedList.Node) *Job {
        return @fieldParentPtr("node", node);
    }
};

pub fn main(init: std.process.Init) !void {
    var buf: [4096]u8 = undefined;
    var file_writer = std.Io.File.stdout().writerStreaming(init.io, &buf);
    const out = &file_writer.interface;

    // The tasks live wherever you put them. Here that is the stack, and the
    // list allocates nothing at all: no allocator is passed, and `prepend`
    // cannot fail. That is the whole reason the intrusive design exists.
    var deploy = Task{ .name = "deploy", .priority = 3 };
    var verify = Task{ .name = "verify", .priority = 1 };
    var build = Task{ .name = "build", .priority = 2 };

    var list: std.SinglyLinkedList = .{};
    list.prepend(&deploy.node);
    list.prepend(&verify.node);
    list.prepend(&build.node);

    try out.writeAll("singly linked, walked by node:\n");
    var it = list.first;
    while (it) |node| : (it = node.next) {
        const task = Task.fromNode(node);
        try out.print("  {s} (priority {d})\n", .{ task.name, task.priority });
    }
    try out.print("len = {d}\n", .{list.len()});

    // Removal takes the node, and the node is reachable from the task, so
    // "remove this task" needs no search at all. On the hand-written list two
    // chapters ago the same operation was a walk from the head.
    list.remove(&verify.node);
    try out.print("\nafter removing verify, len = {d}\n", .{list.len()});

    var jobs: std.DoublyLinkedList = .{};
    var a = Job{ .id = 1 };
    var b = Job{ .id = 2 };
    var c = Job{ .id = 3 };
    jobs.append(&a.node);
    jobs.append(&b.node);
    jobs.append(&c.node);

    try out.writeAll("\ndoubly linked, forwards:  ");
    var f = jobs.first;
    while (f) |node| : (f = node.next) try out.print("{d} ", .{Job.fromNode(node).id});
    try out.writeAll("\ndoubly linked, backwards: ");
    var r = jobs.last;
    while (r) |node| : (r = node.prev) try out.print("{d} ", .{Job.fromNode(node).id});
    try out.writeByte('\n');

    // A node is two words at most and carries no payload, so the cost of being
    // listable is fixed and visible in the struct that pays it. Counted in
    // pointer widths, so the numbers are the same as wasm32 and as a 64-bit
    // native build.
    try out.print("\nsizes in pointer widths\n", .{});
    try out.print("  SinglyLinkedList.Node = {d}\n", .{@sizeOf(std.SinglyLinkedList.Node) / @sizeOf(usize)});
    try out.print("  DoublyLinkedList.Node = {d}\n", .{@sizeOf(std.DoublyLinkedList.Node) / @sizeOf(usize)});
    try out.print("  Task                  = {d} (a slice, a byte, and the node, padded)\n", .{@sizeOf(Task) / @sizeOf(usize)});

    // A task can be on more than one list at a time by holding more than one
    // node. A wrapping list cannot express that without allocating a second
    // node per membership.
    try out.print("\nlists are types, not type functions: {s}, {s}\n", .{
        @typeName(std.SinglyLinkedList), @typeName(std.DoublyLinkedList),
    });

    try out.flush();
}

The node goes inside your struct

const Task = struct {
    name: []const u8,
    priority: u8,
    node: std.SinglyLinkedList.Node = .{},
};

That is what intrusive means. The container does not wrap your value; your value contains the container’s bookkeeping. The list then holds pointers to those embedded nodes, and to get from a node back to the task it lives in:

fn fromNode(node: *std.SinglyLinkedList.Node) *Task {
    return @fieldParentPtr("node", node);
}

@fieldParentPtr subtracts the field’s offset from the pointer. It looks dangerous and is checked at compile time: the named field must exist on the result type and its type must match the pointer’s, so this cannot silently produce a pointer to the wrong thing. The one thing it does not check is that the node really is inside a Task, which is why the conversion is written once, next to the struct that owns it.

Why std made this trade

The advantages are not aesthetic.

The list never allocates. In the snippet the tasks live on the stack and the list is built with no allocator anywhere. prepend cannot fail, so it returns void rather than !void. For an allocator’s own free list, or a scheduler’s run queue, “insertion cannot fail” is not a convenience but a requirement: those are the places you cannot allocate, because they are what allocation is built out of.

Removal needs no search. list.remove(&verify.node) is O(1), because the node is reachable from the task. The hand-written list two chapters ago had to walk from the head to find the node holding a value.

One value, several lists. A Task can hold two node fields and be on two lists at once. A wrapping list cannot express that without allocating a separate node per membership.

The cost is visible. The node is a field in your struct, so the memory a membership costs is written down where you can see it, in the type that pays it.

The price is that the value has to know it might be listed, which is a real coupling, and that a list of i32 now needs a wrapper struct.

Sizes

sizes in pointer widths
  SinglyLinkedList.Node = 1
  DoublyLinkedList.Node = 2
  Task                  = 4

One pointer for a forward link, two for forward and back. Sizes are printed in pointer widths because this page runs as wasm32 in your browser, where a pointer is four bytes, and natively on your machine, where it is eight; the relationships hold either way.

What to use instead for ordinary work

For “a growable sequence of values”, ArrayList. It is contiguous, cache-friendly, and it is what std expects you to reach for.

The intrusive lists are there for the cases in the previous chapter’s last section: pointer stability, O(1) splicing, and insertion that must not allocate. If you are writing an allocator, a scheduler, or an object pool, they are exactly right. If you are collecting user records, they are not.

The migration, if you have old code

  • std.SinglyLinkedList(T) becomes std.SinglyLinkedList, with a node field added to whatever T was.
  • node.data becomes @fieldParentPtr("node", node).<your field>.
  • list.prepend(node) no longer allocates, so wherever the old code allocated a node, allocate your own struct instead and pass &value.node.
  • std.DoublyLinkedList changed the same way, and still gives you first, last, prev and next for iteration in both directions.