5. Traps and Syscalls

Chapter 4 built page tables. This chapter uses them: the kernel enters user mode, a user program runs, and an ecall traps back to the kernel.

The hard part is switching page tables while code is executing. The solution is a trampoline — a page of assembly mapped at the same virtual address in every page table. The CPU can keep fetching instructions across a satp write because the code never moves.

jikei: booted under OpenSBI
Kernel: 0x80200000 - ...
  ...
Paging enabled
Hi
process exited

“Hi” comes from user-mode ecall syscalls. “process exited” comes from the kernel handling SYS_EXIT.

What Changes

File	Status
src/trap/mod.rs	new — TrapFrame, enter_usermode, trap classification
src/trap/trampoline.S	new — user↔kernel page table switch
src/trap/kernel.S	new — S-mode trap handler
src/utils/sbi.rs	add set_timer and Timer extension
src/memory/mod.rs	add alloc_zeroed_frame helper
src/main.rs	add mod trap, demo that enters user mode

No changes to the linker script, boot sequence, or paging module.

Trap Basics

A trap transfers control to supervisor mode. Three flavors:

• Interrupts — asynchronous (timer tick, external device). Bit 63 of scause is set.
• Exceptions — synchronous faults (page fault, illegal instruction). The CPU cannot continue past the faulting instruction.
• Syscalls — intentional ecall from user code. scause = 8. The handler advances sepc by 4 to skip past the ecall.

When a trap fires, the CPU saves the PC in sepc, records the cause in scause, writes any extra info to stval, and jumps to the address in stvec.

Add the SBI Timer Extension

The trap handler needs to clear timer interrupts. Update src/utils/sbi.rs:

#[repr(usize)]
enum SbiExt {
    ConsolePutchar = 0x01,
    Timer = 0x5449_4D45,
}

fn sbi_call(ext_id: SbiExt, fun_id: usize, args: [usize; 3]) -> (isize, usize) {
    let mut error: isize;
    let mut value: usize;
    unsafe {
        core::arch::asm!(
            "ecall",
            in("a7") ext_id as usize,
            in("a6") fun_id,
            inlateout("a0") args[0] => error,
            inlateout("a1") args[1] => value,
            in("a2") args[2],
        )
    }
    (error, value)
}

pub fn console_write(buf: &[u8]) {
    for &byte in buf {
        sbi_call(SbiExt::ConsolePutchar, 0, [byte as usize, 0, 0]);
    }
}

pub fn set_timer(val: u64) {
    sbi_call(SbiExt::Timer, 0, [val as usize, 0, 0]);
}

set_timer programs the timer to fire at val ticks. Setting it to u64::MAX effectively disarms it.

Add a Zeroed Frame Helper

The user demo maps an entire page for code, but the program bytes do not fill the whole page. Clear the frame first so bytes after the program are deterministic instead of stale contents from a previous owner.

Add this next to alloc_frame in src/memory/mod.rs:

/// Allocate a single physical frame and clear it before use.
pub fn alloc_zeroed_frame() -> Option<usize> {
    let pa = alloc_frame()?;
    unsafe { core::ptr::write_bytes(pa as *mut u8, 0, PAGE_SIZE) };
    Some(pa)
}

The Trap Frame

The TrapFrame stores all the state needed to suspend a user process and resume it later. Create src/trap/mod.rs — the sections below build the file.

use core::arch::global_asm;

const INTERRUPT_BIT: usize = 1 << 63;
const SUPERVISOR_TIMER_INTERRUPT: usize = 5;
const USER_ECALL: usize = 8;

global_asm!(include_str!("kernel.S"));
global_asm!(
    include_str!("trampoline.S"),
    TRAMPOLINE = const crate::paging::TRAMPOLINE,
);

The trampoline assembly needs the TRAMPOLINE address as a constant. The const argument in global_asm! substitutes it at compile time.

/// Saved register state for user ↔ kernel transitions.
///
/// Layout (304 bytes, 16-byte aligned):
///   0: sepc, 8: sstatus, 16: scause, 24: stval
///  32..279: x[0..30] = x1..x31 (31 GPRs × 8)
/// 280: kernel_satp, 288: kernel_sp, 296: kernel_trap_ra
#[repr(C, align(16))]
pub struct TrapFrame {
    pub sepc: usize,
    pub sstatus: usize,
    pub scause: usize,
    pub stval: usize,
    pub x: [usize; 31],
    kernel_satp: usize,
    kernel_sp: usize,
    kernel_trap_ra: usize,
}

const _: () = assert!(core::mem::size_of::<TrapFrame>() == 304);

The x array holds 31 general-purpose registers: x[0] is x1/ra, x[1] is x2/sp, and so on up to x[30] for x31/t6. Register x0 is hardwired to zero and not saved.

The three kernel_* fields are written by _enter_usermode before jumping to user mode, and read by _user_trap_entry to restore kernel state when the user traps.

impl TrapFrame {
    pub fn new(entry: usize, sp: usize) -> Self {
        let mut tf = TrapFrame {
            sepc: entry,
            sstatus: 1 << 5, // SPIE — enable interrupts after sret
            scause: 0,
            stval: 0,
            x: [0; 31],
            kernel_satp: 0,
            kernel_sp: 0,
            kernel_trap_ra: 0,
        };
        tf.x[1] = sp; // x[1] = x2 = sp
        tf
    }

    pub fn a0(&self) -> usize {
        self.x[9]
    }
    pub fn a7(&self) -> usize {
        self.x[16]
    }
    pub fn set_a0(&mut self, val: usize) {
        self.x[9] = val;
    }
}

new creates a trap frame for a fresh process: sepc is the entry point, the user stack pointer goes in x2, and SPIE is set so interrupts are enabled when sret drops to U-mode.

The a0/a7 accessors index into the x array. a0 = x10 → x[10-1] = x[9]. a7 = x17 → x[17-1] = x[16].

Kernel Trap Entry

Traps taken while already in S-mode (e.g., a timer interrupt during kernel code) use a simpler handler. It does not switch page tables — the kernel identity map is already active.

Create src/trap/kernel.S:

# _kernel_trap_entry — handles traps taken while in S-mode

.balign 4
.globl _kernel_trap_entry
_kernel_trap_entry:
    addi sp, sp, -144

    .set _off, 0
    .irp reg, ra, t0, t1, t2, t3, t4, t5, t6, a0, a1, a2, a3, a4, a5, a6, a7
        sd \reg, _off(sp)
        .set _off, _off + 8
    .endr

    csrr t0, sepc
    csrr t1, sstatus
    sd t0, 128(sp)
    sd t1, 136(sp)

    mv a0, t0
    csrr a1, scause
    csrr a2, stval
    call handle_kernel_trap

    ld t0, 128(sp)
    ld t1, 136(sp)
    csrw sepc, t0
    csrw sstatus, t1

    .set _off, 0
    .irp reg, ra, t0, t1, t2, t3, t4, t5, t6, a0, a1, a2, a3, a4, a5, a6, a7
        ld \reg, _off(sp)
        .set _off, _off + 8
    .endr

    addi sp, sp, 144
    sret

Only caller-saved registers are saved: ra, t0–t6, a0–a7 (16 registers × 8 bytes = 128 bytes), plus sepc and sstatus (16 more). The callee-saved registers (s0–s11) are safe because the Rust function handle_kernel_trap preserves them by calling convention.

A trap is like an involuntary function call that can land anywhere. The caller-saved registers might be in use, so the trap handler must save them. The callee-saved registers are guaranteed stable across the call into Rust.

The Trampoline Page

This is the most intricate part of the kernel. The trampoline page is mapped at TRAMPOLINE (0x3F_FFFF_F000) in every page table — kernel and user. Code on this page can execute across a satp switch because its virtual address does not change.

Create src/trap/trampoline.S. The file contains three parts: helper macros, _enter_usermode (in normal .text), and the trampoline-page code (in .trampoline).

Register Save/Restore Macros

# sd/ld all GPRs except x10 (a0, used as base pointer) to/from TrapFrame
.macro gprs op
    .irp n, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
        \op x\n, (32 + (\n - 1) * 8)(a0)
    .endr
.endm

# Preserve kernel callee-saved registers across the user-mode round trip.
.macro kernel_sregs op
    .set _off, 0
    .irp reg, s0, s1, s2, s3, s4, s5, s6, s7, s8, s9, s10, s11
        \op \reg, _off(sp)
        .set _off, _off + 8
    .endr
.endm

gprs saves or loads all 31 GPRs (except a0, which holds the trap frame pointer). The offset formula 32 + (n-1) * 8 matches the TrapFrame layout: 4 CSR fields (32 bytes) then 31 registers.

kernel_sregs saves or loads the 12 callee-saved s-registers. These must survive the entire user-mode round trip because _enter_usermode is called from Rust and must return with s-registers intact.

Enter User Mode

_enter_usermode is in normal .text — it is NOT on the trampoline page. It sets up the kernel state in the trap frame, then jumps to the trampoline to finish the transition.

# a0 = *mut TrapFrame (kernel VA)
# a1 = user satp
# a2 = TrapFrame VA in user page table

.balign 4
.globl _enter_usermode
_enter_usermode:
    addi sp, sp, -96
    kernel_sregs sd

    sd sp, 288(a0)              # tf.kernel_sp
    sd ra, 296(a0)              # tf.kernel_trap_ra
    csrr t0, satp
    sd t0, 280(a0)              # tf.kernel_satp

    li t0, {TRAMPOLINE}
    csrw stvec, t0

    ld t0, 0(a0)
    csrw sepc, t0

    li t0, (1 << 8)
    csrc sstatus, t0
    li t0, (1 << 5)
    csrs sstatus, t0

    # Jump to _user_trap_return via its trampoline VA
    la t0, _user_trap_return
    la t1, _user_trap_entry
    sub t0, t0, t1
    li t1, {TRAMPOLINE}
    add t0, t0, t1

    mv a0, a2
    jr t0

Step by step:

1. Save kernel s-registers on the kernel stack (96 bytes for 12 registers).
2. Stash kernel state in the trap frame: stack pointer, return address, and current satp. When the user traps back, the trampoline reads these to restore the kernel.
3. Point stvec at the trampoline so user traps land on the shared page.
4. Load sepc from the trap frame — this is the user PC.
5. Clear SPP (bit 8 of sstatus) so sret drops to U-mode. Set SPIE (bit 5) so interrupts are enabled after sret.
6. Compute the trampoline VA of _user_trap_return. Since _user_trap_return is in the .trampoline section, its link-time address differs from TRAMPOLINE. The subtraction and addition rebase it.
7. Switch a0 to the trap frame’s user-page-table VA (a2), since after the page table switch the kernel VA in a0 would be invalid.
8. Jump to the trampoline.

The Trampoline Code

Everything below lives in the .trampoline section — the page that Chapter 4 reserved and mapped at TRAMPOLINE in every page table.

.pushsection .trampoline, "ax"
.balign 4

.globl _user_trap_entry
_user_trap_entry:
    csrrw a0, sscratch, a0

    gprs sd                      # save x1-x9, x11-x31
    csrr t0, sscratch
    sd t0, 104(a0)              # save the real user a0 from sscratch

    csrr t0, sepc
    sd t0,  0(a0)
    csrr t0, sstatus
    sd t0,  8(a0)
    csrr t0, scause
    sd t0, 16(a0)
    csrr t0, stval
    sd t0, 24(a0)

    ld t0, 280(a0)
    ld sp, 288(a0)
    ld ra, 296(a0)
    csrw satp, t0
    sfence.vma
    kernel_sregs ld
    addi sp, sp, 96
    ret

_user_trap_entry — the CPU jumps here on any trap from U-mode (because stvec points to TRAMPOLINE):

1. Swap a0 with sscratch. Before the trap, sscratch held the trap frame pointer (user VA). After the swap, a0 points to the trap frame, and sscratch holds the user’s a0.
2. Save all GPRs into the trap frame (except a0 which is our base pointer). Then recover the real user a0 from sscratch and save it too (offset 104 = x10’s slot).
3. Save CSRs: sepc, sstatus, scause, stval.
4. Restore kernel state: load kernel_satp, kernel_sp, kernel_trap_ra from the trap frame. Write kernel_satp to satp and flush the TLB.
5. Restore kernel s-registers from the kernel stack, clean up the stack frame.
6. ret — returns to the address saved in kernel_trap_ra, which is the call site in enter_usermode.

.globl _user_trap_return
_user_trap_return:
    csrw satp, a1
    sfence.vma
    csrw sscratch, a0

    gprs ld                      # restore x1-x9, x11-x31
    ld x10, 104(a0)             # restore a0 last (was our base ptr)

    sret

.popsection

_user_trap_return — entered from _enter_usermode to drop into U-mode:

1. Switch to user page table (a1 = user satp). Flush the TLB.
2. Store the trap frame pointer in sscratch so _user_trap_entry can find it on the next trap.
3. Restore all user GPRs from the trap frame. a0 is restored last since it is the base pointer.
4. sret — jumps to sepc in U-mode with interrupts enabled (SPIE was set).

The Rust Trap Interface

Back in src/trap/mod.rs, add the Rust side:

pub enum TrapCause {
    Syscall,
    TimerInterrupt,
    Exception {
        scause: usize,
        stval: usize,
        sepc: usize,
    },
}

unsafe extern "C" {
    fn _enter_usermode(tf: *mut TrapFrame, user_satp: usize, tf_user_va: usize);
}

/// Enter user mode. Returns when the process traps back to S-mode.
pub fn enter_usermode(
    tf: &mut TrapFrame,
    user_satp: usize,
    tf_user_va: usize,
) -> TrapCause {
    unsafe { _enter_usermode(tf as *mut TrapFrame, user_satp, tf_user_va) }

    // Trampoline returned. Restore kernel stvec.
    unsafe {
        core::arch::asm!(
            "la {tmp}, _kernel_trap_entry",
            "csrw stvec, {tmp}",
            tmp = out(reg) _,
        );
    }

    let scause = tf.scause;
    if scause & INTERRUPT_BIT != 0 {
        match scause & !INTERRUPT_BIT {
            SUPERVISOR_TIMER_INTERRUPT => TrapCause::TimerInterrupt,
            _ => exception(tf),
        }
    } else if scause == USER_ECALL {
        tf.sepc += 4; // advance past ecall
        TrapCause::Syscall
    } else {
        exception(tf)
    }
}

fn exception(tf: &TrapFrame) -> TrapCause {
    TrapCause::Exception {
        scause: tf.scause,
        stval: tf.stval,
        sepc: tf.sepc,
    }
}

enter_usermode is the key abstraction: call it with a trap frame and user page table, and it returns a TrapCause when the user traps. The entire user-mode round trip — trampoline, sret, user code, ecall, trampoline back — looks like a single function call to Rust.

After the trampoline returns, stvec is switched back to _kernel_trap_entry so kernel-mode traps use the simpler handler.

For syscalls, sepc is advanced by 4 so the process resumes at the instruction after ecall.

#[unsafe(no_mangle)]
extern "C" fn handle_kernel_trap(sepc: usize, scause: usize, stval: usize) {
    if scause & INTERRUPT_BIT != 0 {
        match scause & !INTERRUPT_BIT {
            SUPERVISOR_TIMER_INTERRUPT => {
                crate::utils::sbi::set_timer(u64::MAX);
            }
            cause => panic!(
                "unhandled kernel interrupt: cause={cause}, sepc={sepc:#x}"
            ),
        }
    } else {
        panic!(
            "kernel exception: scause={scause:#x}, stval={stval:#x}, sepc={sepc:#x}"
        );
    }
}

pub fn init_hart() {
    crate::utils::sbi::set_timer(u64::MAX);
    unsafe {
        core::arch::asm!(
            "la {tmp}, _kernel_trap_entry",
            "csrw stvec, {tmp}",
            "li {tmp}, (1 << 5)",
            "csrs sie, {tmp}",
            tmp = out(reg) _,
        );
    }
}

handle_kernel_trap handles traps taken while already in S-mode. Timer interrupts are cleared. Everything else panics — a kernel exception is unrecoverable.

init_hart sets up the trap system for one hart:

1. Disarm the timer (set it far in the future).
2. Point stvec at _kernel_trap_entry.
3. Enable the supervisor timer interrupt in sie (bit 5 = STIE).

A First User Program

To prove the trap system works, embed a tiny assembly program that prints “Hi” via putchar syscalls and exits.

Add this to src/main.rs — it is the same technique the real kernel uses to embed user programs in .rodata:

unsafe extern "C" {
    fn _demo_start() -> u8;
    fn _demo_end() -> u8;
}

core::arch::global_asm!(
    r#"
    .pushsection .rodata.user_prog, "a"
    .globl _demo_start
    _demo_start:
        li a7, 100
        li a0, 'H'
        ecall
        li a7, 100
        li a0, 'i'
        ecall
        li a7, 100
        li a0, '\n'
        ecall
        li a7, 1
        ecall
    .globl _demo_end
    _demo_end:
    .popsection
    "#
);

Syscall 100 is SYS_PUTCHAR (print one character). Syscall 1 is SYS_EXIT. The program is placed in .rodata so it is included in the kernel binary but not executed from there — it is copied into a user page at runtime.

Wire It Together

Replace src/main.rs:

#![no_std]
#![no_main]

#[macro_use]
mod utils;
mod boot;
mod memory;
mod paging;
mod trap;

core::arch::global_asm!(include_str!("../boot.S"));

#[unsafe(no_mangle)]
pub extern "C" fn kernel_main(hartid: usize, dtb_ptr: usize) -> ! {
    boot::init(hartid, dtb_ptr);
    trap::init_hart();

    run_demo();

    println!("all done");
    loop {
        unsafe { core::arch::asm!("wfi") }
    }
}

fn run_demo() {
    use memory::{alloc_frame, alloc_zeroed_frame, PAGE_SIZE};
    use paging::*;

    // Build a user address space.
    let pt = PageTable::alloc();

    // Map one page of user code. Use a zeroed frame so any unused bytes
    // past the end of the program decode to a deterministic illegal
    // instruction instead of stale frame contents.
    let code_pa = alloc_zeroed_frame().expect("oom");
    let code = demo_code();
    unsafe {
        core::ptr::copy_nonoverlapping(code.as_ptr(), code_pa as *mut u8, code.len());
    }
    pt.map_at(0x10000, code_pa, PTE_U | PTE_R | PTE_X, 0);

    // Map a user stack (4 pages, guard page below).
    let stack_base = 0x0100_1000;
    for i in 0..4 {
        let pa = alloc_frame().expect("oom");
        pt.map_at(
            stack_base + i * PAGE_SIZE,
            pa,
            PTE_U | PTE_R | PTE_W,
            0,
        );
    }
    let stack_top = stack_base + 4 * PAGE_SIZE;

    // Map the trampoline at its canonical VA.
    pt.map_trampoline();

    // Map a trap frame page (kernel-only — no PTE_U).
    let tf_va = TRAMPOLINE - PAGE_SIZE;
    let tf_pa = alloc_frame().expect("oom");
    pt.map_at(tf_va, tf_pa, PTE_R | PTE_W, 0);

    // Initialize the trap frame.
    let tf = unsafe { &mut *(tf_pa as *mut trap::TrapFrame) };
    *tf = trap::TrapFrame::new(0x10000, stack_top);

    let user_satp = pt.satp();

    // Run: enter user mode, handle syscalls, repeat until exit.
    loop {
        match trap::enter_usermode(tf, user_satp, tf_va) {
            trap::TrapCause::Syscall => match tf.a7() {
                1 => {
                    println!("process exited");
                    return;
                }
                100 => {
                    print!("{}", tf.a0() as u8 as char);
                }
                nr => {
                    println!("unknown syscall {nr}");
                    return;
                }
            },
            trap::TrapCause::TimerInterrupt => {} // re-enter
            trap::TrapCause::Exception {
                scause,
                stval,
                sepc,
            } => {
                println!(
                    "exception: scause={scause:#x} stval={stval:#x} sepc={sepc:#x}"
                );
                return;
            }
        }
    }
}

fn demo_code() -> &'static [u8] {
    let start = _demo_start as *const u8 as usize;
    let end = _demo_end as *const u8 as usize;
    unsafe { core::slice::from_raw_parts(start as *const u8, end - start) }
}

unsafe extern "C" {
    fn _demo_start() -> u8;
    fn _demo_end() -> u8;
}

core::arch::global_asm!(
    r#"
    .pushsection .rodata.user_prog, "a"
    .globl _demo_start
    _demo_start:
        li a7, 100
        li a0, 'H'
        ecall
        li a7, 100
        li a0, 'i'
        ecall
        li a7, 100
        li a0, '\n'
        ecall
        li a7, 1
        ecall
    .globl _demo_end
    _demo_end:
    .popsection
    "#
);

run_demo builds a user address space from scratch:

1. Allocate a page table.
2. Copy the demo program into a fresh frame and map it at 0x10000 as user R+X.
3. Allocate 4 stack pages starting at 0x01001000. The unmapped page at 0x01000000 acts as a guard — a stack overflow faults instead of silently corrupting memory.
4. Map the trampoline (same physical page, same VA as in the kernel page table).
5. Map a trap frame page just below the trampoline. No PTE_U — user code cannot read it, but the trampoline can (it runs in S-mode).
6. Initialize the trap frame with the entry point and stack pointer.
7. Loop: enter user mode, handle the trap, re-enter until exit.

The loop is a minimal scheduler. Each enter_usermode call sends the CPU to user mode and returns when a trap occurs. Syscalls are dispatched inline. Timer interrupts are ignored (we re-enter immediately). Exceptions are fatal.

Run It

cargo run --release

After the paging and memory output from previous chapters:

Paging enabled
Hi
process exited
all done

The kernel:

1. Entered user mode via sret on the trampoline
2. User code ran at 0x10000 in its own page table
3. Each ecall trapped to _user_trap_entry, switched back to the kernel page table, and returned into enter_usermode
4. Rust code handled the syscall and re-entered user mode
5. SYS_EXIT broke the loop

If you see “kernel exception” instead, the user program or trap frame is set up wrong. If the kernel hangs, stvec probably does not point at the trampoline — check init_hart.

Checkpoint

OpenSBI
  -> boot::init (memory, paging)
  -> trap::init_hart (stvec, sie)
  -> run_demo
     -> build user page table
     -> enter_usermode (trampoline -> sret -> U-mode)
     -> user ecall -> _user_trap_entry -> kernel
     -> handle syscall, re-enter
     -> SYS_EXIT -> done

The kernel can now move safely between supervisor and user mode. The trampoline handles the page-table switch. The trap frame preserves all register state. Kernel s-registers survive the round trip.

What Comes Next

The demo hardcodes one process with an inline scheduler. Chapter 6 builds a real process table, spawn_with_args, and a proper scheduler loop that picks from multiple runnable processes. Chapter 7 adds timer-driven preemption and SMP.