5. Understanding results¶
Fio will start displaying results after you execute the benchmarking command. The more nodes participate in the test, the more results will be output by fio. However, a summary will be provided at the end indicated by the words “All clients”. For example:
All clients: (groupid=0, jobs=80): err= 0: pid=0: Mon Jan 28 19:18:23 2019 read: IOPS=198k, BW=772Mi (809M)(45.3GiB/60078msec) slat (usec): min=2, max=1510, avg= 6.77, stdev= 3.29 clat (nsec): min=1567, max=287148k, avg=3787988.85, stdev=6333559.68 lat (usec): min=50, max=287152, avg=3794.83, stdev=6333.46 bw ( KiB/s): min= 2048, max=113536, per=0.08%, avg=9880.44, stdev=4052.50, samples=9600 iops : min= 512, max=28384, avg=2470.06, stdev=1013.12, samples=9600 write: IOPS=84.8k, BW=331Mi (347M)(19.4GiB/60078msec) slat (usec): min=2, max=1055, avg= 7.21, stdev= 3.33 clat (usec): min=138, max=498280, avg=21347.45, stdev=40779.33 lat (usec): min=144, max=498296, avg=21354.72, stdev=40779.01 bw ( KiB/s): min= 968, max=49824, per=0.08%, avg=4240.17, stdev=1738.29, samples=9600 iops : min= 242, max=12456, avg=1059.99, stdev=434.58, samples=9600 lat (usec) : 2=0.01%, 20=0.01%, 50=0.01%, 100=0.71%, 250=9.39% lat (usec) : 500=7.81%, 750=3.96%, 1000=3.80% lat (msec) : 2=12.40%, 4=20.13%, 10=24.61%, 20=9.37%, 50=4.52% lat (msec) : 100=1.43%, 250=1.70%, 500=0.17% cpu : usr=0.68%, sys=3.44%, ctx=15563500, majf=0, minf=491 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued rwt: total=11866050,5093585,0, short=0,0,0, dropped=0,0,0
This is a sample output from a
randrw.fio (random read/write) job run on an all-flash cluster of five nodes. The main metrics are IOPS (
iops) and bandwidth (
read: IOPS=198k, BW=772Mi (809M)(45.3GiB/60078msec) write: IOPS=84.8k, BW=331Mi (347M)(19.4GiB/60078msec)
Pay attention to IOPS when you benchmark random workloads and observe bandwidth when you test sequential operations. When you test random reads/writes also note the
iodepth parameter that indicates how many requests are queued and waiting.
slatis submission latency that indicates how much time it took to submit I/O to the kernel.
clatis completion latency that indicates how much time passed between submission to the kernel and completing the I/O (excluding submission latency).
latis the sum of