Linux 系统中提供CPU性能分析工具整理
Linux 系统中提供CPU性能分析工具整理
汇总
查看CPU信息
在linux操作系统中，CPU的信息在启动的过程中被装载到虚拟目录/proc下的cpuinfo文件中，我们可以通过 cat /proc/cpuinfo 查看一下：
cat /proc/cpuinfo
显示如下：
root@thead-910:~# cat /proc/cpuinfo
processor : 0
hart : 0
isa : rv64imafdcsu
mmu : sv39
model name : T-HEAD C910
freq : 1.2GHz
icache : 64kB
dcache : 64kB
l2cache : 2MB
tlb : 1024 4-ways
cache line : 64Bytes
address sizes : 40 bits physical, 39 bits virtual
vector version : 0.7.1
processor : 1
hart : 1
isa : rv64imafdcsu
mmu : sv39
model name : T-HEAD C910
freq : 1.2GHz
icache : 64kB
dcache : 64kB
l2cache : 2MB
tlb : 1024 4-ways
cache line : 64Bytes
address sizes : 40 bits physical, 39 bits virtual
vector version : 0.7.1
通过 cpuinfo 可以看到主要信息：
两颗 RISC-V 64位处理器，型号 T-HEAD C910
CPU 主频：1.2GHz
支持的 ISA 特性：
i: 基础整形指令集
m: 整型乘除法指令扩展
a: 原子操作指令扩展
f: 单精度指令扩展
d: 双精度指令扩展
c: 16位压缩指令扩展
s: 特权态支持
u: 用户态支持
I/D CACHE: 64KB
二级 CACHE： 2MB
MMU TLB：4路1024个表项
物理地址位宽：40位
虚拟地址位宽: 39位
矢量加速指令集版本: 0.7.1
测量处理器运算能力 dhrystone
Dhrystone是测量处理器运算能力的最常见基准程序之一，常用于处理器的整型运算性能的测量。Dhrystone的计量单位为每秒计算多少次Dhrystone，后来把在VAX-11/780机器上的测试结果1757 Dhrystones/s定义为1 Dhrystone MIPS(百万条指令每秒)。但其也有许多不足，Dhrystone不仅不适合于作为嵌入式系统的测试向量，甚至在其大多数场合下都不适合进行应用。
安装 dhrystone：
apt install -y dhrystone
ICE-EVB 开发板测试：
dhrystone
Copy
在 ICE-EVB 开发板上运行结果如下：
Dhrystone Benchmark, Version 2.1 (Language: C)
Execution starts, 600000000 runs through Dhrystone
Execution ends
Final values of the variables used in the benchmark:
Int_Glob: 5
should be: 5
Bool_Glob: 1
should be: 1
Ch_1_Glob: A
should be: A
Ch_2_Glob: B
should be: B
Arr_1_Glob[8]: 7
should be: 7
Arr_2_Glob[8][7]: 600000010
should be: Number_Of_Runs + 10
Ptr_Glob->
Ptr_Comp: 481216
should be: (implementation-dependent)
Discr: 0
should be: 0
Enum_Comp: 2
should be: 2
Int_Comp: 17
should be: 17
Str_Comp: DHRYSTONE PROGRAM, SOME STRING
should be: DHRYSTONE PROGRAM, SOME STRING
Next_Ptr_Glob->
Ptr_Comp: 481216
should be: (implementation-dependent), same as above
Discr: 0
should be: 0
Enum_Comp: 1
should be: 1
Int_Comp: 18
should be: 18
Str_Comp: DHRYSTONE PROGRAM, SOME STRING
should be: DHRYSTONE PROGRAM, SOME STRING
Int_1_Loc: 5
should be: 5
Int_2_Loc: 13
should be: 13
Int_3_Loc: 7
should be: 7
Enum_Loc: 1
should be: 1
Str_1_Loc: DHRYSTONE PROGRAM, 1'ST STRING
should be: DHRYSTONE PROGRAM, 1'ST STRING
Str_2_Loc: DHRYSTONE PROGRAM, 2'ND STRING
should be: DHRYSTONE PROGRAM, 2'ND STRING
Microseconds for one run through Dhrystone: 0.1
Dhrystones per Second: 12329495.0
换算：
12329495/1757/1200=5.8 DMIPS/MHz
衡量CPU性能 coremark
CoreMark是用来衡量CPU性能的标准。包含如下的运算法则：列举（寻找并排序），数学矩阵操作（普通矩阵运算）和状态机（用来确定输入流中是否包含有效数字），最后还包括CRC（循环冗余校验）。
安装 coremark：
apt install -y coremark
ICE-EVB 开发板测试命令：
coremark
在 ICE-EVB 开发板上运行结果如下：
2K performance run parameters for coremark.
CoreMark Size : 666
Total ticks : 13193
Total time (secs): 13.193000
Iterations/Sec : 8337.754870
Iterations : 110000
Compiler version : GCC8.4.0
Compiler flags : -O2 -O3 -static -funroll-all-loops -finline-limit=500 -fgcse-sm -fno-schedule-insns --param max-rtl-if-conversion-unpredictable-cost=100 -msignedness-cmpiv -fno-code-hoisting -mno-thread-jumps1 -mno-iv-adjust-addr-cost -mno-expand-split-imm -lrt
Memory location : Please put data memory location here
(e.g. code in flash, data on heap etc)
seedcrc : 0xe9f5
[0]crclist : 0xe714
[0]crcmatrix : 0x1fd7
[0]crcstate : 0x8e3a
[0]crcfinal : 0x33ff
Correct operation validated. See readme.txt for run and reporting rules.
CoreMark 1.0 : 8337.754870 / GCC8.4.0 -O2 -O3 -static -funroll-all-loops -finline-limit=500 -fgcse-sm -fno-schedule-insns --param max-rtl-if-conversion-unpredictable-cost=100 -msignedness-cmpiv -fno-code-hoisting -mno-thread-jumps1 -mno-iv-adjust-addr-cost -mno-expand-split-imm -lrt / Heap
换算：
8337.754870 / 1200 = 6.948
其中 1200表示 1200MHz，也就是 1.2GHz，也即 ICE 开发板的 CPU 频率。
whetstone
whetstone是一个用于测量CPU浮点计算性能的工具。
安装 whetstone：
apt install -y whetstone
ICE-EVB 开发板测试：
root@thead-910:~# whetstone 1000000
Loops: 1000000, Iterations: 1, Duration: 19.778 sec.
C Converted Double Precision Whetstones: 5056.029 MIPS
换算：
5056.029/1200=4.2 MIPS/MHz
其中 1200表示 1200MHz，也就是 1.2GHz，也即 ICE 开发板的 CPU 频率。
linpack
通过利用高性能计算机，用高斯消元法求解N元一次稠密线性代数方程组的测试，评价高性能计算机的浮点性能。
安装 linpack：
apt install -y linpack
ICE-EVB 开发板测试：
root@thead-910:~# linpack
Unrolled Double Precision Linpack
Unrolled Double Precision Linpack
norm. resid resid machep x[0]-1 x[n-1]-1
2.7 2.40728912e-13 2.22044605e-16 4.08562073e-14 -2.94209102e-14
times are reported for matrices of order 200
dgefa dgesl total kflops unit ratio
times for array with leading dimension of 201
0.01 0.00 0.01 644521 0.00 0.15
0.01 0.00 0.01 646445 0.00 0.15
0.01 0.00 0.01 647683 0.00 0.15
0.01 0.00 0.01 651032 0.00 0.15
times for array with leading dimension of 200
0.01 0.00 0.01 691623 0.00 0.14
0.01 0.00 0.01 693661 0.00 0.14
0.01 0.00 0.01 696428 0.00 0.14
0.01 0.00 0.01 696383 0.00 0.14
Unrolled Double Precision 651032 Kflops ; 10 Reps
换算：
651032/1200=542.5 KFLOPS/MHz
其中 1200表示 1200MHz，也就是 1.2GHz，也即 ICE 开发板的 CPU 频率。
stream
Stream测试是内存测试中业界公认的内存带宽性能测试基准工具。
安装 stream：
apt install -y stream
ICE-EVB 开发板测试：
root@thead-910:~# stream
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 10000000 (elements), Offset = 0 (elements)
Memory per array = 76.3 MiB (= 0.1 GiB).
Total memory required = 228.9 MiB (= 0.2 GiB).
Each kernel will be executed 10 times.
The *best* time for each kernel (excluding the first iteration)
will be used to compute the reported bandwidth.
-------------------------------------------------------------
Your clock granularity/precision appears to be 1 microseconds.
Each test below will take on the order of 30088 microseconds.
(= 30088 clock ticks)
Increase the size of the arrays if this shows that
you are not getting at least 20 clock ticks per test.
-------------------------------------------------------------
WARNING -- The above is only a rough guideline.
For best results, please be sure you know the
precision of your system timer.
-------------------------------------------------------------
Function Best Rate MB/s Avg time Min time Max time
Copy: 6965.0 0.022986 0.022972 0.023003
Scale: 7020.0 0.022831 0.022792 0.022890
Add: 6095.7 0.039484 0.039372 0.039601
Triad: 6105.9 0.040019 0.039306 0.042522
-------------------------------------------------------------
Solution Validates: avg error less than 1.000000e-13 on all three arrays
-------------------------------------------------------------
sysbench
sysbench是一个开源的、模块化的、跨平台的多线程性能测试工具，可以用来进行CPU、内存、磁盘I/O、线程、数据库的性能测试。
安装 sysbench：
apt install -y sysbench
ICE-EVB 开发板测试：
测试线程：
bash root@thead-910:/# ./sysbench --test=threads --num-threads=32 --thread-yields=100 --thread-lock=8 run sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options: Number of threads: 32 Random number generator seed is 0 and will be ignored
Threads started!
General statistics: total time: 2.9138s total number of events: 10000 total time taken by event execution: 93.1165s response time: min: 0.14ms avg: 9.31ms max: 82.38ms approx. 95 percentile: 32.83ms
Threads fairness: events (avg/stddev): 312.5000/14.83 execution time (avg/stddev): 2.9099/0.00
换算：
14.83/2.9099=5.1
其他测试项目：
-
测试CPU:
root@thead-910:~# ./sysbench --test=cpu --cpu-max-prime=20000 --num-threads=2 run sysbench 0.5: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 2 Random number generator seed is 0 and will be ignored Primer numbers limit: 20000 Threads started! General statistics: total time: 15.0576s total number of events: 10000 total time taken by event execution: 30.1038s response time: min: 3.00ms avg: 3.01ms max: 3.12ms approx. 95 percentile: 3.01ms Threads fairness: events (avg/stddev): 5000.0000/0.00 execution time (avg/stddev): 15.0519/0.00 -
测试IO：
root@thead-910:~# .sysbench --test=fileio --num-threads=16 --file-total-size=2G --file-test-mode=rndrw prepare
sysbench 0.5: multi-threaded system evaluation benchmark
128 files, 16384Kb each, 2048Mb total
Creating files for the test...
Extra file open flags: 0
Creating file test_file.0
Creating file test_file.1
...
Creating file test_file.125
Creating file test_file.126
Creating file test_file.127
2147483648 bytes written in 110.50 seconds (18.53 MB/sec).
root@thead-910:~# sysbench --test=fileio --num-threads=20 --file-total-size=2G --file-test-mode=rndrw run sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options: Number of threads: 20 Random number generator seed is 0 and will be ignored
Extra file open flags: 0 128 files, 16Mb each 2Gb total file size Block size 16Kb Number of IO requests: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started!
Operations performed: 6000 reads, 4000 writes, 12800 Other = 22800 Total Read 93.75Mb Written 62.5Mb Total transferred 156.25Mb (34.834Mb/sec) 2229.38 Requests/sec executed
General statistics: total time: 4.4856s total number of events: 10000 total time taken by event execution: 0.8171s response time: min: 0.01ms avg: 0.08ms max: 20.83ms approx. 95 percentile: 0.08ms
Threads fairness: events (avg/stddev): 500.0000/179.75 execution time (avg/stddev): 0.0409/0.02
root@thead-910:~# ./sysbench --test=fileio --num-threads=20 --file-total-size=2G --file-test-mode=rndrw cleanup
sysbench 0.5: multi-threaded system evaluation benchmark
Removing test files...
- 测试内存：
root@thead-910:~# sysbench --test=memory --memory-block-size=8k --memory-total-size=1G run sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options: Number of threads: 1 Random number generator seed is 0 and will be ignored
Threads started!
Operations performed: 131072 (374615.49 ops/sec)
1024.00 MB transferred (2926.68 MB/sec)
General statistics: total time: 0.3499s total number of events: 131072 total time taken by event execution: 0.2768s response time: min: 0.00ms avg: 0.00ms max: 0.06ms approx. 95 percentile: 0.00ms
Threads fairness: events (avg/stddev): 131072.0000/0.00 execution time (avg/stddev): 0.2768/0.00
- 测试mutex：
root@thead-910:~# sysbench --test=mutex --num-threads=64 run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Random number generator seed is 0 and will be ignored
Threads started!
General statistics:
total time: 1.2616s
total number of events: 64
total time taken by event execution: 76.6870s
response time:
min: 1066.46ms
avg: 1198.23ms
max: 1254.10ms
approx. 95 percentile: 1251.30ms
Threads fairness:
events (avg/stddev): 1.0000/0.00
execution time (avg/stddev): 1.1982/0.05
nbench
nbench是一个简单的用于测试处理器、存储器性能的基准测试程序。
安装 nbench：
apt install -y nbench
ICE-EVB 开发板测试：
root@thead-910:~# nbench
BYTEmark* Native Mode Benchmark ver. 2 (10/95)
Index-split by Andrew D. Balsa (11/97)
Linux/Unix* port by Uwe F. Mayer (12/96,11/97)
TEST : Iterations/sec. : Old Index : New Index
: : Pentium 90* : AMD K6/233*
--------------------:------------------:-------------:------------
NUMERIC SORT : 676.01 : 17.34 : 5.69
STRING SORT : 148.55 : 66.38 : 10.27
BITFIELD : 1.6208e+08 : 27.80 : 5.81
FP EMULATION : 154.38 : 74.08 : 17.09
FOURIER : 21743 : 24.73 : 13.89
ASSIGNMENT : 17.95 : 68.30 : 17.72
IDEA : 3738.9 : 57.19 : 16.98
HUFFMAN : 1657.3 : 45.96 : 14.68
NEURAL NET : 14.84 : 23.84 : 10.03
LU DECOMPOSITION : 649.13 : 33.63 : 24.28
==========================ORIGINAL BYTEMARK RESULTS==========================
INTEGER INDEX : 45.842
FLOATING-POINT INDEX: 27.063
Baseline (MSDOS*) : Pentium* 90, 256 KB L2-cache, Watcom* compiler 10.0
==============================LINUX DATA BELOW===============================
CPU : Dual T-HEAD C910
L2 Cache :
OS : Linux 5.10.4
C compiler : riscv64-unknown-linux-gnu-gcc
libc : static
MEMORY INDEX : 10.186
INTEGER INDEX : 12.479
FLOATING-POINT INDEX: 15.010
Baseline (LINUX) : AMD K6/233*, 512 KB L2-cache, gcc 2.7.2.3, libc-5.4.38
* Trademarks are property of their respective holder.
cache_calibrator
cache_calibrator 可以测量 cache 相关的性能参数。
安装 cache_calibrator：
apt install -y cache-calibrator
ICE-EVB 开发板测试：
cache_calibrator 1200 10M file
在 ICE-EVB 开发板上运行结果如下：
Calibrator v0.9e
(by Stefan.Manegold@cwi.nl, http://www.cwi.nl/~manegold/)
e16a9010 274364796944 4096 16
e16a9fff 274364801023 4096 4095
e16aa000 274364801024 4096 0
MINTIME = 10000
analyzing cache throughput...
range stride spots brutto- netto-time
10485760 8 1310720 17472 273
analyzing cache latency...
range stride spots brutto- netto-time
10485760 8 1310720 139776 273
analyzing TLB latency...
range stride spots brutto- netto-time
1474560 1152 1280 18176 1136
CPU loop + L1 access: 2.50 ns = 3 cy
( delay: 0.00 ns = 0 cy )
caches:
level size linesize miss-latency replace-time
1 64 KB 128 bytes 9.19 ns = 11 cy 9.16 ns = 11 cy
2 2 MB 128 bytes 19.05 ns = 23 cy 19.09 ns = 23 cy
TLBs:
level #entries pagesize miss-latency
1 20 4 KB 3.34 ns = 4 cy
# ls
file.cache-replace-time.gp
file.TLB-miss-latency.data
file.TLB-miss-latency.gp
file.cache-miss-latency.data
file.cache-miss-latency.gp
file.cache-replace-time.data
其中 1200表示 1200MHz，也就是 1.2GHz，也即 ICE 开发板的 CPU 频率；跑完后生成了很多以“file”开头的文件，可以观测 cache 各个方面的性能。
iperf3
iperf 用于测量网络性能。
安装iperf3：
apt install -y iperf3
启动 perf3 server：
iperf3 -s > /dev/null &
开始测试：
iperf3 -c 127.0.0.1
在ICE 开发板上，测试数组如下：
root@thead-910:~# iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 51114 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 454 MBytes 3.80 Gbits/sec 0 1.12 MBytes
[ 5] 1.00-2.00 sec 452 MBytes 3.79 Gbits/sec 0 1.12 MBytes
[ 5] 2.00-3.00 sec 430 MBytes 3.61 Gbits/sec 0 1.12 MBytes
[ 5] 3.00-4.00 sec 445 MBytes 3.73 Gbits/sec 0 1.12 MBytes
[ 5] 4.00-5.00 sec 452 MBytes 3.79 Gbits/sec 0 1.50 MBytes
[ 5] 5.00-6.00 sec 442 MBytes 3.71 Gbits/sec 0 1.50 MBytes
[ 5] 6.00-7.00 sec 448 MBytes 3.76 Gbits/sec 0 1.94 MBytes
[ 5] 7.00-8.00 sec 451 MBytes 3.78 Gbits/sec 0 2.12 MBytes
[ 5] 8.00-9.00 sec 464 MBytes 3.90 Gbits/sec 0 2.81 MBytes
[ 5] 9.00-10.00 sec 458 MBytes 3.84 Gbits/sec 0 3.00 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 4.39 GBytes 3.77 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 4.39 GBytes 3.77 Gbits/sec receiver
如果需要测试物理网卡的速度，可以在远程电脑上启动 iperf3 服务，然后再通过 iperf3 -c <host_ip> 来测试网速。
压力测试神器 stress_ng
stress-ng是stress的加强版，完全兼容stress，并在此基础上增加了几百个参数，堪称压测工具中的瑞士军刀。
安装 stress-ng：
apt install -y stress-ng
应用场景1：CPU 密集型进程（使用CPU的进程）
stress-ng --cpu 2 --cpu-method all
应用场景2：I/O 密集型进程（等待IO的进程）
stress-ng -i 4 --hdd 1 --timeout 600
应用场景3：大量进程的场景（等待CPU的进程->进程间会争抢CPU）
stress-ng -c 16 --timeout 600
性能分析工具 – perf
Perf 是内置于 Linux 内核源码树中的性能剖析(profiling)工具，它基于事件采样原理，以性能事件为基础，支持针对处理器相关性能指标与操作系统相关性能指标的性能剖析，常用于性能瓶颈的查找与热点代码的定位。
通过它，应用程序可以利用 PMU，tracepoint 和内核中的特殊计数器来进行性能统计。它不但可以分析指定应用程序的性能问题 (per thread)，也可以用来分析内核的性能问题，当然也可以同时分析应用代码和内核，从而全面理解应用程序中的性能瓶颈。
使用 perf，您可以分析程序运行期间发生的硬件事件，比如 instructions retired ，processor clock cycles 等；您也可以分析软件事件，比如 Page Fault 和进程切换。这使得 Perf 拥有了众多的性能分析能力，举例来说，使用 Perf 可以计算每个时钟周期内的指令数，称为 IPC，IPC 偏低表明代码没有很好地利用 CPU。Perf 还可以对程序进行函数级别的采样，从而了解程序的性能瓶颈究竟在哪里等等。Perf 还可以替代 strace，可以添加动态内核 probe 点，还可以做 benchmark 衡量调度器的好坏。
安装perf
apt install perf
装完之后就可以使用perf命令了。
usage: perf [--version] [--help] [OPTIONS] COMMAND [ARGS]
The most commonly used perf commands are:
annotate Read perf.data (created by perf record) and display annotatedd code
archive Create archive with object files with build-ids found in perff.data file
bench General framework for benchmark suites
buildid-cache Manage build-id cache.
buildid-list List the buildids in a perf.data file
c2c Shared Data C2C/HITM Analyzer.
config Get and set variables in a configuration file.
data Data file related processing
diff Read perf.data files and display the differential profile
evlist List the event names in a perf.data file
ftrace simple wrapper for kernel's ftrace functionality
inject Filter to augment the events stream with additional informatiion
kallsyms Searches running kernel for symbols
kmem Tool to trace/measure kernel memory properties
kvm Tool to trace/measure kvm guest os
list List all symbolic event types
lock Analyze lock events
mem Profile memory accesses
record Run a command and record its profile into perf.data
report Read perf.data (created by perf record) and display the profiile
sched Tool to trace/measure scheduler properties (latencies)
script Read perf.data (created by perf record) and display trace outtput
stat Run a command and gather performance counter statistics
test Runs sanity tests.
timechart Tool to visualize total system behavior during a workload
top System profiling tool.
version display the version of perf binary
See 'perf help COMMAND' for more information on a specific command.
- 安装flamegraph
火焰图是脚本，只需要下载:
git clone https://github.com/brendangregg/FlameGraph.git
查看当前软硬件环境、支持的性能事件
查看所有分类事件的个数:
perf list | awk -F: '/Tracepoint event/ { lib[$1]++ } END { for (l in lib) { printf " %-16s %d\n", l, lib[l] } }' | sort | column
在 ICE 芯片运行结果如下：
root@thead-910:~# perf list | awk -F: '/Tracepoint event/ { lib[$1]++ } END { for (l in lib) { printf " %-16s %d\n", l, lib[l] } }' | sort | column
9p 3 iomap 9 random 16
alarmtimer 4 irq 5 ras 4
block 19 jbd2 17 raw_syscalls 2
bpf_test_run 1 kmem 13 rcu 1
bpf_trace 1 kyber 3 regmap 15
cgroup 13 mdio 1 rpcgss 26
clk 16 migrate 1 sched 20
compaction 14 mmap 1 signal 2
cpuhp 3 mmc 2 skb 3
dma_fence 7 module 5 smbus 4
dwc3 15 napi 1 sock 3
ext4 117 neigh 7 spi 7
fib 1 net 18 sunrpc 127
fib6 1 nfs 58 swiotlb 1
filelock 12 nfs4 87 task 2
filemap 4 oom 8 tcp 7
ftrace 1 page_pool 4 timer 12
gadget 24 pagemap 2 udp 1
gpio 2 percpu 5 vmscan 14
i2c 4 power 22 workqueue 4
initcall 3 printk 1 writeback 30
io_uring 14 qdisc 4 xdp 12
- flamegraph火焰图
如果希望了解CPU在一段时间内的都运行了哪些函数以及各函数都消耗了多少时间，就可以使用On CPU火焰图，这种火焰图基于cpu-cycles事件进行采样，然后通过svg图片格式展现出来
dd if=/dev/zero of=/tmp/testfile bs=4K count=102400 &
perf record -e cpu-cycles -a -g sleep 1
perf script > perf.unfold
cd FlameGraph
./stackcollapse-perf.pl < ../perf.unfold | ./flamegraph.pl > ../perf.svg
首先在后台启动一个dd命令，让它持续运行一段时间，然后开启perf record，记录一秒钟内cpu都运行了多少个cpu-cycles，也就是时间（同时使能-g，就会一并记录运行的函数以及调用关系），再利用perf script命令将perf.data转成perf.unfold，最后利用FlameGraph工具将其转换成一个perf.svg，这是一个图形文件，用浏览器打开后会得到这样图记录着函数调用关系及其cpu-cycles（时间）占比，就像一缕缕升起的火苗，所以被称之为火焰图。
火焰图还可以通过鼠标点击放大，观察其细节。