online, possible and disabled cpus
Robert P. J. Day
rpjday at crashcourse.ca
Mon Oct 22 17:33:34 EDT 2012
On Mon, 22 Oct 2012, Srivatsa Bhat wrote:
> Hi,
>
> You might want to take a look at Documentation/cpu-hotplug.txt for
> some details regarding online/possible CPUs...
i'll do one more post on this, even though i never imagined it was
going to get this complicated. to recap, what i'm trying to
understand is this output from dmesg:
[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x08] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x09] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0a] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0b] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0c] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0d] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0e] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x0f] disabled)
... snip ...
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[ 0.000000] smpboot: Allowing 16 CPUs, 8 hotplug CPUs
... snip ...
[ 0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:16 nr_node_ids:1
where my quad-core laptop (which should technically have eight
processors) apparently has 16 "possible" CPUs, 8 of which are
apparently available for hotplug.
first, it seems clear that why the above is happening is that a lot
of kernel code that allocates per-CPU variables needs to do that
*statically* and can't cope with hotplugged CPUs, so the kernel boot
code will automatically add some float to the alleged number of CPUs,
just to play it safe. as i read it, then, the kernel code will use
the value of "nr_cpu_ids" above equal to 16 on my system, even though
only the first 8 correspond to physical processors. so far, so good.
from "Documentation/cpu-hotplug.txt":
"cpu_possible_mask: Bitmap of possible CPUs that can ever be available
in the system. This is used to allocate some boot time memory for
per_cpu variables that aren't designed to grow/shrink as CPUs are made
available or removed. Once set during boot time discovery phase, the
map is static, i.e no bits are added or removed anytime. Trimming it
accurately for your system needs upfront can save some boot time
memory. See below for how we use heuristics in x86_64 case to keep
this under check...
"Q: How do we determine how many CPUs are available for hotplug.
"A: There is no clear spec defined way from ACPI that can give us that
information today. Based on some input from Natalie of Unisys, that
the ACPI MADT (Multiple APIC Description Tables) marks those possible
CPUs in a system with disabled status.
"Andi implemented some simple heuristics that count the number of
disabled CPUs in MADT as hotpluggable CPUS. In the case there are no
disabled CPUS we assume 1/2 the number of CPUs currently present can
be hotplugged."
so the above suggests that the number of "possible" CPUs on my
system is based on the ACPI values, which will already incorporate
some extras just to play it safe.
in the file "arch/x86/kernel/smpboot.c", we further read:
/*
* cpu_possible_mask should be static, it cannot change as cpu's
* are onlined, or offlined. The reason is per-cpu data-structures
* are allocated by some modules at init time, and dont expect to
* do this dynamically on cpu arrival/departure.
* cpu_present_mask on the other hand can change dynamically.
* In case when cpu_hotplug is not compiled, then we resort to current
* behaviour, which is cpu_possible == cpu_present.
* - Ashok Raj
*
* Three ways to find out the number of additional hotplug CPUs:
* - If the BIOS specified disabled CPUs in ACPI/mptables use that.
* - The user can overwrite it with possible_cpus=NUM
* - Otherwise don't reserve additional CPUs.
... snip ....
so it looks like the value of 16 comes from the ACPI/mptables.
finally, from "drivers/acpi/tables.c", this looks like the code that's
called for each possible CPU, that matches the output from "dmesg":
void acpi_table_print_madt_entry(struct acpi_subtable_header *header)
{
if (!header)
return;
switch (header->type) {
case ACPI_MADT_TYPE_LOCAL_APIC:
{
struct acpi_madt_local_apic *p =
(struct acpi_madt_local_apic *)header;
printk(KERN_INFO PREFIX
"LAPIC (acpi_id[0x%02x] lapic_id[0x%02x] %s)\n",
p->processor_id, p->id,
(p->lapic_flags & ACPI_MADT_ENABLED) ? "enabled" : "disabled");
}
break;
and at that point, i think i'm just going to let it go. but that
still brings me back to my original question regarding the code for
/proc/softirqs in fs/proc/softirqs.c -- why iterate through all
*possible* CPUs when some of them will clearly be disabled and not
represent actual CPUs?
static int show_softirqs(struct seq_file *p, void *v)
{
int i, j;
seq_puts(p, " ");
for_each_possible_cpu(i)
seq_printf(p, "CPU%-8d", i);
seq_putc(p, '\n');
for (i = 0; i < NR_SOFTIRQS; i++) {
seq_printf(p, "%12s:", softirq_to_name[i]);
for_each_possible_cpu(j)
seq_printf(p, " %10u", kstat_softirqs_cpu(i, j));
seq_putc(p, '\n');
}
return 0;
}
anyway, i think i've flogged this enough.
rday
--
========================================================================
Robert P. J. Day Ottawa, Ontario, CANADA
http://crashcourse.ca
Twitter: http://twitter.com/rpjday
LinkedIn: http://ca.linkedin.com/in/rpjday
========================================================================
More information about the Kernelnewbies
mailing list