Prev Next

5. Understanding results¶

Fio will start displaying results after you execute the benchmarking command. The more nodes participate in the test, the more results will be output by fio. However, a summary will be provided at the end indicated by the words “All clients”. For example:

All clients: (groupid=0, jobs=80): err= 0: pid=0: Mon Jan 28 19:18:23 2019
 read: IOPS=198k, BW=772Mi (809M)(45.3GiB/60078msec)
  slat (usec): min=2, max=1510, avg= 6.77, stdev= 3.29
  clat (nsec): min=1567, max=287148k, avg=3787988.85, stdev=6333559.68
   lat (usec): min=50, max=287152, avg=3794.83, stdev=6333.46
 bw (  KiB/s): min= 2048, max=113536, per=0.08%, avg=9880.44, stdev=4052.50, samples=9600
 iops        : min=  512, max=28384, avg=2470.06, stdev=1013.12, samples=9600
write: IOPS=84.8k, BW=331Mi (347M)(19.4GiB/60078msec)
  slat (usec): min=2, max=1055, avg= 7.21, stdev= 3.33
  clat (usec): min=138, max=498280, avg=21347.45, stdev=40779.33
   lat (usec): min=144, max=498296, avg=21354.72, stdev=40779.01
 bw (  KiB/s): min=  968, max=49824, per=0.08%, avg=4240.17, stdev=1738.29, samples=9600
 iops        : min=  242, max=12456, avg=1059.99, stdev=434.58, samples=9600
lat (usec)   : 2=0.01%, 20=0.01%, 50=0.01%, 100=0.71%, 250=9.39%
lat (usec)   : 500=7.81%, 750=3.96%, 1000=3.80%
lat (msec)   : 2=12.40%, 4=20.13%, 10=24.61%, 20=9.37%, 50=4.52%
lat (msec)   : 100=1.43%, 250=1.70%, 500=0.17%
cpu          : usr=0.68%, sys=3.44%, ctx=15563500, majf=0, minf=491
IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
   submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
   complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
   issued rwt: total=11866050,5093585,0, short=0,0,0, dropped=0,0,0

This is a sample output from a randrw.fio (random read/write) job run on an all-flash cluster of five nodes. The main metrics are IOPS (iops) and bandwidth (bw):

read: IOPS=198k, BW=772Mi (809M)(45.3GiB/60078msec)
write: IOPS=84.8k, BW=331Mi (347M)(19.4GiB/60078msec)

Pay attention to IOPS when you benchmark random workloads and observe bandwidth when you test sequential operations. When you test random reads/writes also note the iodepth parameter that indicates how many requests are queued and waiting.

In addition:

slat is submission latency that indicates how much time it took to submit I/O to the kernel.
clat is completion latency that indicates how much time passed between submission to the kernel and completing the I/O (excluding submission latency).
lat is the sum of slat and clat.

Version 4.7 — Oct 18, 2021

Edit Print

Prev Next