x86: starting the other CPUs, the application processors (APs). More...

#include <core/system.h>
#include <core/kargs.h>
#include <core/string.h>
#include <core/arch.h>
#include <cpu/cpu.h>
#include <cpu/msr.h>
#include <cpu/smp.h>
#include <mm/mm.h>
#include <mm/pmap.h>
#include <mm/vm.h>
#include <platform/misc.h>
#include <sys/cpu.h>

Data Structures
struct	trampoline_data
	What an AP needs to reach the kernel, at x86_trampoline_data (the fields are at trampoline.S's TR_ offsets). More...

Macros
#define	IDLE_STACK_SIZE 8192
#define	ICR_INIT 0x00000500
#define	ICR_STARTUP 0x00000600
#define	ICR_ASSERT 0x00004000
#define	TABLES (X86_TRAMPOLINE + 0x1000)
#define	PG_PRESENT 0x001
#define	PG_WRITE 0x002
#define	PG_LARGE 0x080
#define	MSR_EFER 0xC0000080

Functions
static void	x86_ap_start (struct cpu *cpu)
	Where an AP enters the kernel, from the trampoline: it sets up its own CPU structures, says that it's there, and idles.
void	arch_cpu_kick (struct cpu *cpu)
	Makes another CPU enter the kernel now, with an inter-processor interrupt: to run a thread that just became ready if it's idle, or so that its thread sees it was killed or takes its signals.
static paddr_t	tables_build (void)
	Builds the trampoline's page tables, which map the trampoline's page where it is and the kernel's half as the kernel's tables do.
static int	ap_boot (struct cpu *cpu, uint32_t id)
	Starts the CPU whose local APIC is id, as cpu.
void	x86_smp_init (void)
	Starts the other CPUs, at boot, unless the kernel's command line says smp=off.

Variables
char	x86_trampoline []
char	x86_trampoline_data []
char	x86_trampoline_end []
static volatile int	ap_started

Detailed Description

x86: starting the other CPUs, the application processors (APs).

The local APICs that ACPI's MADT lists are the CPUs (apic.c). The boot CPU sends each other one an INIT IPI, then a startup IPI (twice if it doesn't answer). The AP starts in real mode at the trampoline's page (trampoline.S), which brings it into the kernel, at x86_ap_start(), on its own idle stack. There it gets its own GDT, TSS and gs (curcpu()), sets up its local APIC, says that it's there, and idles until an interrupt gives it something to run.

The trampoline's page tables map its own page where it is, and the kernel's half as the kernel's tables do (the kernel's entries of the top table never change). The AP switches to the kernel's tables right away, so nothing of the trampoline's stays in a TLB.

With smp=off on the kernel's command line, only the boot CPU runs.

References: [1] - Intel(R) 64 and IA-32 Architectures Software Developer's Manual, Volume 3, 9.4 (Multiple-Processor Initialization) and 11.6 (Issuing Interprocessor Interrupts)

Definition in file smp.c.

Macro Definition Documentation

◆ ICR_ASSERT

#define ICR_ASSERT 0x00004000

Definition at line 46 of file smp.c.

Referenced by ap_boot(), arch_cpu_kick(), and tlb_shootdown().

◆ ICR_INIT

#define ICR_INIT 0x00000500

Definition at line 44 of file smp.c.

Referenced by ap_boot().

◆ ICR_STARTUP

#define ICR_STARTUP 0x00000600

Definition at line 45 of file smp.c.

Referenced by ap_boot().

◆ IDLE_STACK_SIZE

#define IDLE_STACK_SIZE 8192

Definition at line 40 of file smp.c.

Referenced by x86_smp_init().

◆ MSR_EFER

#define MSR_EFER 0xC0000080

Definition at line 67 of file smp.c.

Referenced by x86_smp_init().

◆ PG_LARGE

#define PG_LARGE 0x080

Definition at line 65 of file smp.c.

Referenced by pmap_init(), pte_get(), and tables_build().

◆ PG_PRESENT

#define PG_PRESENT 0x001

Definition at line 63 of file smp.c.

Referenced by arch_page_get_mapping(), pmap_add(), pmap_init(), pmap_protect(), pmap_remove(), pmap_remove_all(), pte_flags(), pte_flags(), pte_get(), pte_get(), tables_build(), and tables_free().

◆ PG_WRITE

#define PG_WRITE 0x002

Definition at line 64 of file smp.c.

Referenced by pmap_init(), pte_flags(), pte_flags(), pte_get(), pte_get(), and tables_build().

◆ TABLES

#define TABLES (X86_TRAMPOLINE + 0x1000)

Definition at line 61 of file smp.c.

Referenced by tables_build().

Function Documentation

◆ ap_boot()

int ap_boot	(	struct cpu *	cpu,
		uint32_t	id )

static

Starts the CPU whose local APIC is id, as cpu.

Returns 0, or -1 if it didn't answer.

Definition at line 141 of file smp.c.

References ap_boot(), ap_started, cpu::arch, trampoline_data::cpu, trampoline_data::entry, ICR_ASSERT, ICR_INIT, ICR_STARTUP, arch_cpu::idle_stack, PHYS2VIRT, trampoline_data::stack, x86_ap_start(), x86_lapic_ipi(), x86_pit_delay(), X86_TRAMPOLINE, x86_trampoline, and x86_trampoline_data.

Referenced by ap_boot(), and x86_smp_init().

◆ arch_cpu_kick()

void arch_cpu_kick ( struct cpu * cpu )

Makes another CPU enter the kernel now, with an inter-processor interrupt: to run a thread that just became ready if it's idle, or so that its thread sees it was killed or takes its signals.

Each architecture implements it; with one CPU, it does nothing.

Parameters

cpu the CPU

Definition at line 97 of file smp.c.

References arch_cpu::apic_id, cpu::arch, arch_cpu_kick(), ICR_ASSERT, x86_lapic_ipi(), and X86_WAKE_VECTOR.

◆ tables_build()

paddr_t tables_build ( void )

static

Builds the trampoline's page tables, which map the trampoline's page where it is and the kernel's half as the kernel's tables do.

Returns the top table's physical address.

Definition at line 105 of file smp.c.

References kvm_space, pmap::map, memset(), PAGE_SIZE, PG_LARGE, PG_PRESENT, PG_WRITE, PHYS2VIRT, vm_space::pmap, TABLES, tables_build(), and X86_TRAMPOLINE.

Referenced by tables_build(), and x86_smp_init().

◆ x86_ap_start()

void x86_ap_start ( struct cpu * cpu )

static

Where an AP enters the kernel, from the trampoline: it sets up its own CPU structures, says that it's there, and idles.

Definition at line 74 of file smp.c.

References ap_started, arch_idle(), cpu::idle, kvm_space, vm_space::pmap, pmap_switch(), VMA, x86_ap_start(), x86_gdt_setup(), x86_idt_setup(), and x86_lapic_cpu_setup().

Referenced by ap_boot(), and x86_ap_start().

◆ x86_smp_init()

void x86_smp_init ( void )

Starts the other CPUs, at boot, unless the kernel's command line says smp=off.

A CPU that doesn't answer is left out.

Definition at line 182 of file smp.c.

References ap_boot(), arch_cpu::apic_id, cpu::arch, cpus, cpus_count, CPUS_MAX, trampoline_data::cr0, trampoline_data::cr3, trampoline_data::cr4, trampoline_data::efer, cpu::id, arch_cpu::idle_stack, IDLE_STACK_SIZE, kargs_get(), kfree(), kmalloc(), memcpy(), memset(), MSR_EFER, NULL, PAGE_SIZE, PHYS2VIRT, printk(), read_cr4(), tables_build(), x86_apic_cpus(), x86_smp_init(), X86_TRAMPOLINE, x86_trampoline, x86_trampoline_data, and x86_trampoline_end.

Referenced by platform_init(), and x86_smp_init().

Variable Documentation

◆ ap_started

volatile int ap_started

static

Definition at line 70 of file smp.c.

Referenced by ap_boot(), and x86_ap_start().

◆ x86_trampoline

char x86_trampoline[]

extern

Referenced by ap_boot(), and x86_smp_init().

◆ x86_trampoline_data

char x86_trampoline_data[]

Definition at line 49 of file smp.c.

Referenced by ap_boot(), and x86_smp_init().

◆ x86_trampoline_end

char x86_trampoline_end[]

Definition at line 49 of file smp.c.

Referenced by x86_smp_init().

Data Structures

Macros

Functions

Variables

Detailed Description

Macro Definition Documentation

◆ ICR_ASSERT

◆ ICR_INIT

◆ ICR_STARTUP

◆ IDLE_STACK_SIZE

◆ MSR_EFER

◆ PG_LARGE

◆ PG_PRESENT

◆ PG_WRITE

◆ TABLES

Function Documentation

◆ ap_boot()

◆ arch_cpu_kick()

◆ tables_build()

◆ x86_ap_start()

◆ x86_smp_init()

Variable Documentation

◆ ap_started

◆ x86_trampoline

◆ x86_trampoline_data

◆ x86_trampoline_end