Tag Archives: proc

Debugging, Embedded Linux, Linux Kernel, Linux Programming, SysAd

Exploring Linux procfs via shell scripts

4 Comments

Very often, while working on a Linux project, we’d like information about the system we’re working on: both at a global scope and a local (process) scope.

Have we not wondered: is there a quick way to query which kernel version am using, what interrupts are enabled & hit, what my processor(s) are, details about kernel subsystems, memory usage, file, network, IPC usage, etc etc. Linux’s proc filesystem makes this easy.

So what exactly is the proc filesystem all about?

Essentially, some quick salient points about the proc filesystem:

  • it’s a RAM-based filesystem (think ramdisk; yup, it’s volatile)
  • it’s a kernel feature, not userspace – proc is a filesystem supported by the Linux kernel VFS
  • it serves two primary purposes
    • proc serves as a “view” deep into the kernel internals; we can see details about hardware and software subsystems that userspace otherwise would have no access to (no syscalls)
    • certain “files” under proc, typically anchored under /proc/sys, can be written into: these basically are the “tuning knobs” of the Linux kernel. Sysads, developers, apps, etc exploit this feature
  • proc is mounted on start-up under /proc
  • a quick peek under /proc will show you several “files” and “folders”. These are pseudo-entries in the sense that they exist only in RAM while power is applied. The “folders” that are numbers are in fact the PID of each process that’s alive when you typed ‘ls’! it’s a snapshot of the system at that moment in time..
  • in fact, the name “proc” suggests “process”

At this point, and if you’re not really familiar with this stuff, I’d urge you to peek around /proc on your Linux box, cat-ting stuff as you go. (Also, lest i forget, it’s better to run as root (sudo /bin/bash) so that we don’t get annoying ‘permission denied’ messages). Of course, be careful when you run as root!!!

For example, to get one started off:

Continue reading Exploring Linux procfs via shell scripts →

Debugging, Device Drivers, Embedded Linux, Linux Kernel, Linux Programming

kmalloc and vmalloc : Linux kernel memory allocation API Limits

13 Comments

The Intent

To determine how large a memory allocation can be made from within the kernel, via the “usual suspects” – the kmalloc and vmalloc kernel memory allocation APIs, in a single call.

Lets answer this question using two approaches:

  1. By reading the source, and
  2. By trying it out empirically on the system.

(Kernel source from kernel ver 3.0.2; tried out on kernel ver 2.6.35 on an x86 PC and 2.6.33.2 on the (ARM) BeagleBoard).

Quick Summary

For the impatient:

The upper limit (number of bytes that can be allocated in a single kmalloc request), is a function of:

  • the processor – really, the page size – and
  • the number of buddy system freelists (MAX_ORDER).

On both x86 and ARM, with a standard page size of 4 Kb and MAX_ORDER of 11, the kmalloc upper limit is 4 MB!

The vmalloc upper limit is, in theory, the amount of physical RAM on the system.
[EDIT/UPDATE #2]
In practice, the kernel allocates an architecture (cpu) specific “range” of virtual memory for the purpose of vmalloc: from VMALLOC_START to VMALLOC_END.

[EDIT/UPDATE #1]
In practice, it’s usually a lot less. A useful comment by ugoren points out that:
” in 32bit systems, vmalloc is severely limited by its virtual memory area. For a 32bit x86 machine, with 1GB RAM or more, vmalloc is limited to 128MB (for all allocations together, not just for one).

[EDIT/UPDATE #3 : July ’17]
I wrote a simple kernel module (can download the source code, see the link at the end of this article), to test the kmalloc/vmalloc upper limits; the results are what we expect:
for kmalloc, 4 MB is the upper limit with a single call; for vmalloc, it depends on the vmalloc range.

Also, please realize, the actual amount you can acquire at runtime depends on the amount of physically contiguous RAM available at that moment in time; this can and does vary widely.

Finally, what if one require more than 4 MB of physically contiguous memory? That’s pretty much exactly the reason behind CMA – the Contiguous Memory Allocator! Details on CMA and using it are in this excellent LWN article here. Note that CMA was integrated into mainline Linux in v3.17 (05 Oct 2014). Also, the recommended API interface to use CMA is the ‘usual’ DMA [de]alloc APIs (kernel documentation here and here); don’t try and use them directly.

I kmalloc Limit Tests

First, lets check out the limits for kmalloc :

Continue reading kmalloc and vmalloc : Linux kernel memory allocation API Limits →

Misc, SysAd

LINUX CHEATSHEET – vmstat, ps, top

Leave a comment

LINUX Quick Reference Cheat Sheet

vmstat, ps, top


v 0.1 : Last Updated: March 2009 : <kaiwan at designergraphix dot com>
(c) kaiwan billimoria.

Much of the information below gleaned from various Linux man pages.

 

———————————————————-

vmstat

vmstat fields quick reference

The -a switch displays active/inactive memory, given a 2.5.41 kernel or better.
The -f switch displays the number of forks since boot. This includes the fork, vfork, and clone system calls, and is equivalent to the total number of tasks created. Each process is represented by one or more tasks, depending on thread usage. This display does not repeat.
The -m displays slabinfo.
The -n switch causes the header to be displayed only once rather than periodically.
The -s switch displays a table of various event counters and memory statistics. This display does not repeat.

delay is the delay between updates in seconds. If no delay is specified, only one report is printed with
the average values since boot.
count is the number of updates. If no count is specified and delay is defined, count defaults to infinity.

The -d reports disk statistics (2.5.70 or above required)
The -p followed by some partition name for detailed statistics (2.5.70 or above required)
The -S followed by k or K or m or M switches outputs between 1000, 1024, 1000000, or 1048576 bytes
The -V switch results in displaying version information.

FIELD DESCRIPTION FOR VM MODE

Procs

r: The number of processes waiting for run time << ready-to-run >>.
b: The number of processes in uninterruptible sleep << blocked >>.

Memory << (default) in kilobytes >>
swpd: the amount of virtual memory used.

free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache << page cache, not incl. swap cache>> .
inact: the amount of inactive memory. (-a option)
active: the amount of active memory. (-a option)

Swap << in kilobytes/second >>

si: Amount of memory swapped in from disk (/s).
so: Amount of memory swapped to disk (/s).
 Continue reading LINUX CHEATSHEET – vmstat, ps, top →