kvmem.c File Reference

kmalloc() and kfree(): the kernel's memory for its own structures. More...

#include <core/system.h>
#include <core/panic.h>
#include <core/string.h>
#include <mm/buddy.h>
#include <mm/mm.h>
#include <mm/pmap.h>
#include <mm/vm.h>
#include <mm/vmem.h>

Go to the source code of this file.

Macros
#define	KVMEM_GUARD   PAGE_SIZE
#define	TEST_OBJS   48

Functions
void *	slab_alloc (size_t size, struct malloc_type *type)
	Allocates a small object, from a slab of its malloc type and size class, making a new slab if none has a free object.
int	slab_owns (void *ptr)
	Says whether an object is a slab's, for kfree().
void	slab_free (void *ptr)
	Frees an object of a slab.
void	kvmem_setup (void)
	Sets up the kernel's allocator at boot, once physical memory and the page tables are ready.
static void	kvmem_unmap (vaddr_t base, size_t size)
	Unmaps the pages of an object at base, and gives them back to the buddy allocator.
static void *	kvmem_alloc (size_t size, struct malloc_type *type)
	Allocates an object of whole pages, with its guard page after it.
void *	kmalloc (size_t size, struct malloc_type *type, int flags)
	Allocates memory for one of the kernel's structures.
void	kfree (void *ptr)
	Frees an object that kmalloc() gave.
static size_t	kvmem_segs (void)
	Returns how many segments, used and free, the arena has.
static const char *	kvmem_selftest_run (void)
	Allocates objects of many sizes and checks that each is on pages of its own, that its guard page isn't mapped, and that none overwrites another.
void	kvmem_selftest (void)
	Runs the allocator's self-test, for the kvmem.test kernel argument, and prints whether it passed.
void	kvmem_guardtest (void)
	Writes one byte past the end of an object, for the kvmem.guard kernel argument: the guard page should fault, and the kernel panic.

Variables
struct queue *	malloc_types = QUEUE_NEW()
static struct vmem	kvmem
size_t	kvmem_used
size_t	kvmem_obj_cnt

Detailed Description

kmalloc() and kfree(): the kernel's memory for its own structures.

Objects of up to SLAB_MAX bytes, most of them, come from slabs (mm/slab.c). Bigger objects get whole pages of their own, in the kvmem region of the kernel's address space (ARCH_KVMEM_BASE): a vmem arena (mm/vmem.h) gives the range of addresses, pages of memory are mapped there, and the arena's segment keeps the object's size and malloc type for kfree().

Each object's range ends in a guard page, which is never mapped. A write past an object's end faults, and so does a write below its start (the page there is another object's guard, or isn't allocated). So a kernel stack that overflows, or a buffer overrun, ends in a page fault instead of corrupting another object. It costs addresses, not memory.

Every allocation is counted by its malloc type (/proc/kvmem).

Definition in file kvmem.c.

Macro Definition Documentation

KVMEM_GUARD

#define KVMEM_GUARD   PAGE_SIZE

Definition at line 40 of file kvmem.c.

Referenced by kvmem_alloc().

TEST_OBJS

#define TEST_OBJS   48

Definition at line 194 of file kvmem.c.

Referenced by kvmem_selftest_run().

Function Documentation

kfree()

void kfree

(

void *

ptr

)

Frees an object that kmalloc() gave.

Freeing anything else (a pointer into an object, or one freed already) is a bug, and panics.

Parameters

ptr	the object, or NULL to do nothing

Definition at line 146 of file kvmem.c.

References kfree(), kvmem, kvmem_obj_cnt, kvmem_unmap(), kvmem_used, malloc_type::nr, panic, printk(), slab_free(), slab_owns(), malloc_type::total, vmem_find(), and vmem_free().

Referenced by __arch_cur_thread_kill(), aarch64_fpsimd_drop(), aarch64_thread_new(), acpi_map(), arch_kthread_create(), arch_proc_fork(), arch_sigcode_map(), arch_thread_create(), arch_thread_kill(), at_lookup(), at_path(), ata_partition(), bcache_remove(), binfmt_elf_load_segments(), binfmt_fmt_load(), blkcache_read(), blkcache_write(), cpio_new_node(), cpio_root_node(), creat_path(), dentry_update(), dequeue(), devpts_init(), do_linkat(), do_mknodat(), do_openat(), do_unlinkat(), entry_remove(), exec_script(), exec_script_args(), exec_strings_free(), ext2_block_alloc(), ext2_block_free(), ext2_close(), ext2_dentry_create(), ext2_dentry_find(), ext2_indirect_block_get(), ext2_inode_alloc(), ext2_inode_block_read(), ext2_inode_free(), ext2_load(), ext2_read(), ext2_readdir(), ext2_trunc(), ext2_vmknod(), ext2_write(), file_put(), flush_block(), free_tokens(), free_tree(), hashmap_free(), hashmap_grow(), hashmap_insert(), hashmap_new(), hashmap_node_remove(), irq_uninstall(), kdev_blk_io(), kdev_bread(), kdev_bwrite(), kfree(), kthread_create(), kvmem_selftest_run(), link_path(), minix_inode_build(), minix_load(), minix_read(), minix_trunc(), minix_write(), pf_anon(), pf_object(), pf_zero(), pgrp_release(), pipefs_close(), pipefs_pfree(), pipefs_pipe(), pmap_create(), pmap_release(), pmap_release(), pmap_release(), poll_unregister(), proc_execve(), proc_fork(), proc_kill(), proc_new(), proc_reap(), proc_setpgid(), proc_threads_end(), procfs_close(), procfs_read(), pseudofs_close(), pseudofs_vlink(), pseudofs_vmknod(), pseudofs_vrename(), ptm_close(), pty_free(), pty_new(), queue_node_remove(), queue_remove(), ringbuf_free(), ringbuf_new(), seg_merge(), session_new(), sigcode_in(), socket_inet_create(), socket_inet_shutdown(), socket_tcp_create(), socket_tcp_shutdown(), socket_unix_accept(), socket_unix_bind(), socket_unix_connect(), socket_unix_shutdown(), sys_accept(), sys_chdir(), sys_execve(), sys_getcwd(), sys_mmap(), sys_mount(), sys_munmap(), sys_pipe(), sys_poll(), sys_readlinkat(), sys_renameat(), sys_socket(), sys_symlinkat(), sys_thread_join(), table_alloc(), tcp_pcb_free(), tcp_pcb_new(), thread_create(), thread_new(), tmpfs_close(), tmpfs_mount(), tmpfs_trunc(), tmpfs_write(), tokenize(), tty_new(), tty_release(), uart_8250_attach(), upath_get(), vfs_file_lock_release(), vfs_link(), vfs_lookup_parent(), vfs_mknod(), vfs_mount(), vfs_mount_root_dev(), vfs_remove(), vfs_rename(), vm_anon_copy(), vm_anon_decref(), vm_anon_destroy(), vm_anon_new(), vm_object_vnode(), vm_space_destroy(), vm_space_fork(), vnode_page_in(), vnode_pages_free(), vnode_pages_update(), vt_term_init(), walk(), and x86_smp_init().

kmalloc()

void * kmalloc	(	size_t	size,
		struct malloc_type *	type,
		int	flags )

Allocates memory for one of the kernel's structures.

Parameters

size	the size wanted, in bytes
type	the malloc type it's counted as (MALLOC_DEFINE())
flags	M_ZERO to have it zeroed, or 0

Returns

the object, or NULL if there was no memory for it

Definition at line 123 of file kvmem.c.

References kmalloc(), kvmem_alloc(), M_ZERO, malloc_types, memset(), NULL, malloc_type::qnode, slab_alloc(), and SLAB_MAX.

Referenced by aarch64_fpsimd_fork(), aarch64_fpsimd_trap(), aarch64_thread_new(), acpi_map(), arch_kthread_create(), arch_proc_fork(), arch_proc_init(), arch_sigcode_map(), arch_thread_create(), ata_partition(), bcache_insert(), binfmt_fmt_load(), blkcache_read(), blkcache_write(), block_new(), copy_fds(), cpio_new_node(), cpio_root_node(), creat_path(), dentry_update(), devfs_init(), devpts_init(), enqueue(), enqueue_head(), exec_script_args(), exec_strings_get(), ext2_block_alloc(), ext2_block_free(), ext2_dentry_create(), ext2_dentry_find(), ext2_indirect_block_get(), ext2_inode_alloc(), ext2_inode_block_read(), ext2_inode_build(), ext2_inode_free(), ext2_load(), ext2_read(), ext2_readdir(), ext2_trunc(), ext2_vmknod(), ext2_write(), file_new(), flush_block(), free_tree(), hashmap_grow(), hashmap_insert(), hashmap_new(), irq_install(), kdev_blk_io(), kdev_bread(), kdev_bwrite(), kmalloc(), kproc_init(), kvmem_guardtest(), kvmem_selftest_run(), link_path(), load_ramdisk(), minix_bcache_get(), minix_inode_build(), minix_load(), minix_read(), minix_trunc(), minix_write(), pf_anon(), pf_object(), pf_zero(), pipefs_end(), pipefs_mkpipe(), pmap_alloc(), pmap_alloc(), pmap_create(), poll_register(), proc_execve(), proc_init(), proc_new(), proc_setpgid(), procfs_init(), procfs_read(), procfs_vget(), pseudofs_vlink(), pseudofs_vmknod(), ptm_new(), pty_new(), queue_new(), ringbuf_new(), session_new(), sigcode_in(), socket_inet_create(), socket_tcp_accept(), socket_tcp_create(), socket_unix_accept(), socket_unix_bind(), socket_unix_connect(), socket_unix_create(), strdup(), sys_getcwd(), sys_poll(), sys_readlinkat(), table_alloc(), tcp_pcb_new(), thread_new(), tmpfs_mount(), tmpfs_trunc(), tmpfs_write(), tokenize(), tty_new(), uart_8250_attach(), upath_get(), vfs_file_flock(), vfs_install(), vfs_mount(), vfs_mount_root_dev(), vm_anon_new(), vm_entry_new(), vm_object_vnode(), vm_space_fork(), vm_space_insert(), vmem_alloc(), vmem_init(), vnode_page_in(), vnode_pages_update(), vt_probe(), vt_term_init(), walk(), x86_fpu_load_from(), x86_fpu_trap(), and x86_smp_init().

kvmem_alloc()

void * kvmem_alloc ( size_t size, struct malloc_type * type )

static

Allocates an object of whole pages, with its guard page after it.

Returns NULL if there is no memory, or no room left in the region.

Definition at line 82 of file kvmem.c.

References vmem_seg::base, kvm_space, kvmem, kvmem_alloc(), KVMEM_GUARD, kvmem_obj_cnt, kvmem_unmap(), kvmem_used, mm_page_alloc(), mm_page_dealloc(), malloc_type::nr, NULL, vm_page::paddr, PAGE_ROUND, PAGE_SIZE, vm_space::pmap, pmap_add(), vmem::size, vmem_seg::size, malloc_type::total, VM_KRW, vmem_alloc(), and vmem_free().

Referenced by kmalloc(), and kvmem_alloc().

kvmem_guardtest()

void kvmem_guardtest

(

void

)

Writes one byte past the end of an object, for the kvmem.guard kernel argument: the guard page should fault, and the kernel panic.

Definition at line 278 of file kvmem.c.

References kmalloc(), kvmem_guardtest(), PAGE_ROUND, and printk().

Referenced by kmain(), and kvmem_guardtest().

kvmem_segs()

size_t kvmem_segs ( void )

static

Returns how many segments, used and free, the arena has.

Definition at line 184 of file kvmem.c.

References kvmem, kvmem_segs(), and vmem::segs.

Referenced by kvmem_segs(), and kvmem_selftest_run().

kvmem_selftest()

void kvmem_selftest

(

void

)

Runs the allocator's self-test, for the kvmem.test kernel argument, and prints whether it passed.

Definition at line 264 of file kvmem.c.

References kvmem_selftest(), kvmem_selftest_run(), and printk().

Referenced by kmain(), and kvmem_selftest().

kvmem_selftest_run()

const char * kvmem_selftest_run ( void )

static

Allocates objects of many sizes and checks that each is on pages of its own, that its guard page isn't mapped, and that none overwrites another.

Then frees them out of order, and checks that the arena is as it was. Returns NULL if all is well, or what went wrong.

Definition at line 202 of file kvmem.c.

References arch_page_get_mapping(), vmem::inuse, kfree(), kmalloc(), kvm_space, kvmem, kvmem_obj_cnt, kvmem_segs(), kvmem_selftest_run(), memset(), NULL, PAGE_ROUND, PAGE_SIZE, vm_space::pmap, SLAB_MAX, and TEST_OBJS.

Referenced by kvmem_selftest(), and kvmem_selftest_run().

kvmem_setup()

void kvmem_setup

(

void

)

Sets up the kernel's allocator at boot, once physical memory and the page tables are ready.

Definition at line 55 of file kvmem.c.

References ARCH_KVMEM_BASE, ARCH_KVMEM_SIZE, kvmem, kvmem_setup(), malloc_types, PAGE_SIZE, panic, and vmem_init().

Referenced by aarch64_init(), kvmem_setup(), and x86_cpu_init().

kvmem_unmap()

void kvmem_unmap ( vaddr_t base, size_t size )

static

Unmaps the pages of an object at base, and gives them back to the buddy allocator.

Pages that aren't mapped are skipped.

Definition at line 68 of file kvmem.c.

References arch_page_get_mapping(), kvm_space, kvmem_unmap(), mm_page_dealloc(), PAGE_SIZE, vm_space::pmap, pmap_remove(), and vmem::size.

Referenced by kfree(), kvmem_alloc(), and kvmem_unmap().

slab_alloc()

void * slab_alloc	(	size_t	size,
		struct malloc_type *	type )

Allocates a small object, from a slab of its malloc type and size class, making a new slab if none has a free object.

kmalloc() calls it for objects of up to SLAB_MAX bytes.

Parameters

size	the size wanted, in bytes, at most SLAB_MAX
type	the malloc type it's counted as

Returns

the object, or NULL if there was no memory for a new slab

Definition at line 142 of file slab.c.

References slab::free, malloc_type::nr, NULL, PAGE_SIZE, slab::size, slab_alloc(), SLAB_HEADER, slab_link(), SLAB_MIN, slab_page_alloc(), slab_unlink(), malloc_type::slabs, malloc_type::total, slab::type, and slab::used.

Referenced by kmalloc(), and slab_alloc().

slab_free()

void slab_free

(

void *

ptr

)

Frees an object of a slab.

The slab goes back on its list if it was full, and gives its page back if it is empty now.

Parameters

ptr	the object, which slab_alloc() gave

Definition at line 199 of file slab.c.

References slab::free, malloc_type::nr, PAGE_ALIGN, slab::size, slab_class(), slab_free(), slab_link(), slab_page_free(), slab_unlink(), malloc_type::slabs, malloc_type::total, slab::type, and slab::used.

Referenced by kfree(), and slab_free().

slab_owns()

int slab_owns

(

void *

ptr

)

Says whether an object is a slab's, for kfree().

Parameters

ptr	the object

Returns

whether it is in the slab region

Definition at line 187 of file slab.c.

References ARCH_SLAB_BASE, ARCH_SLAB_SIZE, and slab_owns().

Referenced by kfree(), and slab_owns().

Variable Documentation

kvmem

struct vmem kvmem

static

Definition at line 42 of file kvmem.c.

Referenced by kfree(), kvmem_alloc(), kvmem_segs(), kvmem_selftest_run(), and kvmem_setup().

kvmem_obj_cnt

size_t kvmem_obj_cnt

Definition at line 45 of file kvmem.c.

Referenced by kfree(), kvmem_alloc(), kvmem_selftest_run(), and procfs_meminfo().

kvmem_used

size_t kvmem_used

Definition at line 44 of file kvmem.c.

Referenced by kfree(), kvmem_alloc(), and procfs_meminfo().

malloc_types

struct queue* malloc_types = QUEUE_NEW()

Definition at line 30 of file kvmem.c.

Referenced by kmalloc(), kvmem_setup(), and procfs_kvmem().