Wednesday, August 14, 2013

Finding Linux Syscall Source Code

One of the joys of developing for Linux is the ability to look at the source code for syscalls.

Example 1: What happens when you call fsync() on a loopback fs?


First, check the glibc source for fsync(), to see if glibc adds any code, or directly calls into the kernel.

alex@laptop:~/src/glibc-2.15$ grep -r -n "^fsync" *
CANCEL-FCT-WAIVE:231:fsync
misc/fsync.c:24:fsync (fd)
sysdeps/unix/syscalls.list:16:fsync - fsync Ci:i __libc_fsync fsync
sysdeps/mach/hurd/fsync.c:26:fsync (fd)

Opening misc/fsync.c, we see that this is just a stub file, and therefore fsync is a thin wrapper around the syscall.

#include <errno.h>
#include <unistd.h>

/* Make all changes done to FD actually appear on disk.  */
int
fsync (fd)
     int fd;
{
  __set_errno (ENOSYS);
  return -1;
}


stub_warning (fsync)
#include <stub-tag.h>

The wrapper calls into the kernel, returns -1 on an error and sets errno with the return value from the kernel. See comments in sysdeps/unix/syscall-template.S for more information about glibc syscall stubs.

From here, we look for the syscall source code in the kernel.  The syscall source is wrapped with the SYSCALL_DEFINEn() macro, where n is the number of arguments.  In this case, SYSCALL_DEFINE1(fsync, unsigned int, fd):

alex@laptop:~src/kernel$ grep -r -n -I "SYSCALL.*fsync" *
arch/xtensa/include/asm/unistd.h:74:__SYSCALL( 26, sys_fsync, 1)
arch/x86/include/asm/unistd_64.h:173:__SYSCALL(__NR_fsync, sys_fsync)
arch/powerpc/include/asm/systbl.h:124:SYSCALL_SPU(fsync)
arch/s390/kernel/syscalls.S:129:SYSCALL(sys_fsync,sys_fsync,sys32_fsync_wrapper)
fs/sync.c:201:SYSCALL_DEFINE1(fsync, unsigned int, fd)
include/asm-generic/unistd.h:253:__SYSCALL(__NR_fsync, sys_fsync)

Opening fs/sync.c, we see the actual source of fsync (function comments removed for brevity):

int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync)
{
if (!file->f_op || !file->f_op->fsync)
return -EINVAL;
return file->f_op->fsync(file, start, end, datasync);
}
EXPORT_SYMBOL(vfs_fsync_range);

int vfs_fsync(struct file *file, int datasync)
{
return vfs_fsync_range(file, 0, LLONG_MAX, datasync);
}
EXPORT_SYMBOL(vfs_fsync);

static int do_fsync(unsigned int fd, int datasync)
{
struct file *file;
int ret = -EBADF;

file = fget(fd);
if (file) {
ret = vfs_fsync(file, datasync);
fput(file);
}
return ret;
}

SYSCALL_DEFINE1(fsync, unsigned int, fd)
{
return do_fsync(fd, 0);
}

And down the rabbit hole we go.

Monday, October 25, 2010

Ubuntu 10.10 Install Notes

  1. System -> Preferences -> Keyboard Shortcuts
    • Lock Screen: Super + L
    • Run a terminal: Super + `
    • Move Window: Shift + Super + arrows
    • Switch to Workspace: Super + arrows
    • gksu synaptic: Super + S
  2. System -> Preferences -> Keyboard
    • Layout -> Options -> Ctrl Key Position -> Make Caps Lock an additional Ctrl
  3. System -> Preferences -> Appearance
    • Fonts -> Details -> Resolution: 86 DPI
  4. Synaptic (add Canonical Partners Repo)
    • emacs
    • emacs-goodies-el
    • compizconfig-settings-manager
    • compiz-fusion-plugins-extra       -- for the grid plugin
    • inkscape
    • gimp
    • build-essential
    • git
    • mercurial
    • subversion
    • gparted
    • sun-java6-jdk
    • sun-java6-plugin
    • vlc
    • nautilus-open-terminal    -- adds open in terminal to right-click in nautilus
    • wireshark
      • change menu item to "Wireshark (as root)" and "gksu wireshark"
  5. Other software
    • Google Chrome
    • Dropbox
    • Adobe Flash Plugin
    • Kicad nightly
      • add repo: deb http://ppa.launchpad.net/paxer/ppa/ubuntu lucid main
      • install kicad package
  6. home directory structure
    • progs/
      • bin/
      • src/
  7. Emacs setup:
    1. ln -s /home/alex/Dropbox/Public/.emacs .emacs
    2. Install http://code.google.com/p/js2-mode/
  8. System -> Preferences -> CompizConfig Settings Manager
    • Enable Grid
      • Change shortcuts to Super + numpads

Thursday, October 14, 2010

Recovering a fakeRAID0 in Ubuntu

A few years back my motherboard died. I had two 74GB raptor hard drives set up in a RAID0 (striped) array using the motherboard's built in fakeRAID controller. I tried to recover some of the data today and managed to get the old ntfs partition mounted.

I plugged in the drives to the standard SATA ports on my new motherboard. I have Ubuntu installed on a third hard drive. The two raptor hard drives showed up as /dev/sdb and /dev/sdc.

First, make sure you have mdadm installed. mdadm is the linux software RAID management tool:

$ sudo apt-get install mdadm

Now we will try to create a RAID0 array that matches the original fakeRAID one. RAID0 splits data between the drives in chunks, and to read back the data, we need to know which hard drive had the first chunk and how big the chunks were.

To figure which hard drive was first, run fdisk:

$ sudo fdisk -l
Disk /dev/sdb: 74.4 GB, 74355769344 bytes
255 heads, 63 sectors/track, 9039 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7962aa02

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 74.4 GB, 74355769344 bytes
255 heads, 63 sectors/track, 9039 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3e344803

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *           1       18070   145147243+   7  HPFS/NTFS

At least /dev/sdc has a partition table, so it was probably the first hard drive in the array.

To find the chunk size, I just guessed. Usually it will be one of 64K, 32K, 16K, 128K, etc. I vaguely remember setting it to 16K when I created the fakeRAID. The moral of the story is write down the chunk size when creating a RAID0 array so that recovery is easier.

Now to create the array. /dev/md0 will be the name of the new array, and the hard drive with the first chunk should be listed first (in my case /dev/sdc):

$ sudo mdadm --create /dev/md0 --verbose --level=0 --raid-devices=2 --chunk=64 /dev/sdc /dev/sdb

Try and mount the new raid array (in my case with NTFS, to /media/old_raid0):

$ sudo mkdir /media/old_raid0
$ sudo mount.ntfs /dev/md0 /media/old_raid0

If the mount fails, try a different chunk size:

$ sudo mdadm --stop /dev/md0 && sudo mdadm --remove /dev/md0
$ sudo mdadm --create /dev/md0 --verbose --level=0 --raid-devices=2 --chunk=32 /dev/sdc /dev/sdb

After finding the right chunk size, the mount should be successful, and a new file system should show up in nautilus. Recovery is as simple as copying files to a different hard drive.

Friday, July 16, 2010

A Foray into Emacs

I spent a day learning emacs.

Remapping CapsLock

One of the first things that I did was remap CapsLock as another Control button (in Gnome, so this applies system wide):

System -> Preferences -> Keyboard -> Layouts Tab -> Options -> Ctrl key position -> Make CapsLock an additional Ctrl

The only problem is that I keep hitting CapsLock instead of shift and accidentally starting commands. It's definitely a relief on the pinky though.

~/.emacs

I went through the tutorial and proceeded to edit ~/.emacs for 6 hours (learning lisp in the process too).

  1. Change the default font-size to 10pt (height is in units of 1/10pt):
    (set-face-attribute 'default nil :height 100)
  2. Change the vertical scroll bars to the right side:
    (set-scroll-bar-mode 'right)
  3. Hide the top scroll bar:
    (tool-bar-mode -1) ;; hide top tool-bar
  4. Remap Alt+Tab to Ctrl+Tab (this does completion in a lot of modes):
    ;; remap alt-tab to control-tab
    (define-key function-key-map [(control tab)] [?\M-\t])
  5. Copying/Pasting from other applications - I used the shortcuts from gnome-terminal:
    • Ctrl+Shift+x to cut (clipboard-kill-region)
    • Ctrl+Shift+c to copy (clipboard-kill-ring-save)
    • Ctrl+Shift+v to paste (clipboard-yank)
    • Disable immediate copy into the kill ring with the mouse: (setq mouse-drag-copy-region nil)
    ;; stops mouse selection from copying immediately
    (setq mouse-drag-copy-region nil)
    
    (global-set-key [(control shift x)] 'clipboard-kill-region)
    (global-set-key [(control shift c)] 'clipboard-kill-ring-save)
    (global-set-key [(control shift v)] 'clipboard-yank)
  6. Open the bookmarks list on startup, unless we're opening a file explicitly:
    (setq bookmark-save-flag 1) ;; save every new bookmark
    
    ;; show bookmarks at start-up
    (when (= 1 (length command-line-args))
      (setq inhibit-startup-screen 1)
      (add-hook 'emacs-startup-hook 
         '(lambda ()
            (bookmark-bmenu-list)
            (switch-to-buffer "*Bookmark List*"))))

Here's my .emacs: http://dl.dropbox.com/u/2628537/.emacs

(tip: keep your configuration files in your Dropbox and link to them from your home folder)

Saturday, January 30, 2010

Python AST Pretty Printer

Description

This is a python module that provides a dump function, identical to ast.dump, except the returned string has newlines and indentation.

Download: astpp.py

Example

import ast
import astpp

tree = ast.parse(
"""
print "Hello World!"
s = "I'm a string!"
print s
""")
print astpp.dump(tree)
Which prints
Module(body=[
    Print(dest=None, values=[
        Str(s='Hello World!'),
      ], nl=True),
    Assign(targets=[
        Name(id='s', ctx=Store()),
      ], value=Str(s="I'm a string!")),
    Print(dest=None, values=[
        Name(id='s', ctx=Load()),
      ], nl=True),
  ])