Linux peformance tips: disk i/o scheduler

The disk i/o scheduler is the method that Linux uses to describe how data will be submitted to the storage devices.  They apply to disk devices and not partitions.

On Linux, the main algorithms are these three:

• CFQ (Completely fair Queuing)
• Noop
• Deadline

There is another popular scheduler called Anticipatory, but as from kernel version 2.6.33 it has been removed.

CFQ places synchronous requests on per-process queues and allocates time slices for each one of the queues to access the disk. Length of each time slice depends on the process priority, and allows process to idle at the end of the I/O call in anticipation of a close-by read request (another read on the same sector). You can use ionice to give priorities.

Tuning parameters can be given at /sys/block/<device>/queue/iosched/slice_idle, /sys/block/<device>/queue/iosched/quantum and /sys/block/<device>/queue/iosched/low_latency.

Noop operates as a simple FIFO queue, first in first out.

Deadline imposes a deadline to all requests, to prevent processes to "hang" waiting for disk. In addition of the read and write queues, it does maintain two deadline queues (one for read, one for write), so the deadline scheduler will check if the requests have expired in the deadline queues. Read queues have higher priority.

Tuning parameters can be given at /sys/block/<device>/queue/iosched/writes_starved, /sys/block/<device>/queue/iosched/read_expire and /sys/block/<device>/queue/iosched/write_expire.

To check what scheduler we are using we can query the block device (sda in my case):

$ cat /sys/block/sda/queue/scheduler
noop deadline [cfq]

I'm using CFQ. To change it to noop or deadline,  we can insert the desired scheduler into the same file we use to query:

(as root)

# echo deadline > /sys/block/sda/queue/scheduler
# cat /sys/block/sda/queue/scheduler
noop [deadline] cfq

For any other device the route would be /sys/block/<device name>/queue/scheduler. You can change it anytime, without crashing the system.

Which one is better? it depends on your environment. If you have a proper process priority scheme in your server, CFQ could be the best. For backup servers with low performance disks, deadline worked pretty fine for me in the past.

To put some numbers, on my desktop using default scheduler, dd operations timing to write 1 GB are as follows:

$ dd if=/dev/zero of=tmp1 bs=512 count=2000000

• CFQ 10.8294 s, 94.6 MB/s
• Deadline 9.90455 s, 103 MB/s
• Noop 10.0025 s, 102 MB/s

Reading + writing :

$ dd if=tmp1 of=tmp2

• CFQ 26.3413 s, 38.9 MB/s
• Deadline 30.449 s, 33.6 MB/s
• Noop 28.9345 s, 35.4 MB/s

Reading, deadline is ahead due to the prioritized read queue - however not so far from a FIFO algorithm. Read + Write makes CFQ more advantageous not because the algorithm itself (as I have not given high priority to the dd command) but just as result of performance degradation of deadline and noop.