OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 20 9 466 22 21 23.46 1.10
bin-true no-host 665 533 2,377 798 789 3.57 1.20
configure.cpython no-host 14,602 14,846 29,125 14,938 15,197 1.99 1.02
configure.ocaml no-host 3,738 3,682 4,643 3,806 3,881 1.24 1.02
configure.tcc no-host 46 41 205 49 49 4.42 1.04
configure.util-linux no-host 13,633 14,477 31,615 14,067 14,454 2.32 1.03
configure.yash no-host 1,827 1,777 2,456 1,823 1,864 1.34 1.00
hello-world no-host 1 1 56 3 3 44.32 2.21

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,527 1,100 24,098 3,157 3,101 9.54 1.25
bin-true no-host 108,344 65,477 805,767 127,181 126,174 7.44 1.17
configure.cpython no-host 1,993,075 2,025,063 6,102,321 2,014,102 2,004,557 3.06 1.01
configure.ocaml no-host 372,975 353,665 617,087 377,006 376,148 1.65 1.01
configure.tcc no-host 6,097 5,032 23,935 6,514 6,428 3.93 1.07
configure.util-linux no-host 1,915,818 2,077,823 5,278,043 1,809,138 1,810,764 2.75 0.94
configure.yash no-host 233,114 222,316 368,929 235,857 233,904 1.58 1.01
hello-world no-host 148 79 3,961 344 336 26.76 2.32

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.5 22.9 9.2 8.7 5.13 2.06
bin-true no-host 3.0 1.4 18.5 8.1 7.6 6.15 2.70
configure.cpython no-host 29.2 29.0 29.2 29.3 29.1 1.00 1.00
configure.ocaml no-host 22.5 22.5 22.5 22.6 22.6 1.00 1.01
configure.tcc no-host 19.6 19.7 19.7 19.7 19.6 1.00 1.01
configure.util-linux no-host 30.3 30.3 30.2 30.3 30.5 1.00 1.00
configure.yash no-host 26.4 26.4 26.5 26.3 26.5 1.00 0.99
hello-world no-host 2.9 1.4 18.3 6.3 5.9 6.34 2.18

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.3 6.3 3,799 3,799 0 699 1,867 1,233 2 0 50,000 0
bin-true no-host 798 1 2 2.6 8.1 153,766 12,331 2,895 8,751 76,334 68,681 3,008 3 50,000 0
configure.cpython no-host 14,938 1 67 15.3 29.3 1,773,757 49,822 28,520 91,504 881,462 800,791 25,736 60 50,000 0
configure.util-linux no-host 14,067 3 66 21.6 30.3 2,704,668 53,751 65,719 114,371 1,293,954 1,296,343 32,241 41 131,438 1
configure.ocaml no-host 3,806 1 2 2.6 22.6 149,782 11,148 5,215 7,732 70,841 71,209 1,695 3 50,000 0
configure.tcc no-host 49 0 0 1.3 19.7 30,567 30,567 0 1,541 14,632 14,394 271 0 50,000 0
configure.yash no-host 1,823 1 2 2.6 26.3 139,678 8,035 6,694 6,393 68,250 65,035 2,095 3 50,000 0
abuild-print-help no-host 22 1 2 2.8 9.2 116,315 64,413 42,621 4,543 54,451 57,321 277 3 85,242 1
hello-world no-host 3 0 0 0.3 5.9 3,798 3,798 0 700 1,867 1,231 2 0 50,000 0
bin-true no-host 789 1 2 2.6 7.6 153,765 12,326 2,890 8,752 76,334 68,679 3,008 3 50,000 0
configure.cpython no-host 15,197 1 66 15.3 29.1 1,773,756 49,817 28,515 91,505 881,462 800,789 25,736 60 50,000 0
configure.util-linux no-host 14,454 3 66 21.6 30.5 2,704,668 53,734 65,714 114,373 1,293,954 1,296,341 32,241 41 131,428 1
configure.ocaml no-host 3,881 1 2 2.6 22.6 149,782 11,143 5,210 7,734 70,841 71,207 1,695 3 50,000 0
configure.tcc no-host 49 0 0 1.3 19.6 30,567 30,567 0 1,543 14,632 14,392 271 0 50,000 0
configure.yash no-host 1,864 1 2 2.6 26.5 139,678 8,030 6,689 6,395 68,250 65,033 2,095 3 50,000 0
abuild-print-help no-host 21 1 2 2.8 8.7 116,314 64,408 42,616 4,544 54,451 57,319 277 3 85,232 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 20 10 10 4.5 bash
15 no-host abuild-print-help 9 7 3 3.5 dash
23 no-host abuild-print-help 466 412 58 22.9 osh-cpython
31 no-host abuild-print-help 22 15 7 9.2 osh-native
39 no-host abuild-print-help 21 10 12 8.7 osh-native-souffle
1 no-host bin-true 665 295 404 3.0 bash
9 no-host bin-true 533 276 268 1.4 dash
17 no-host bin-true 2,377 813 1,826 18.5 osh-cpython
25 no-host bin-true 798 304 509 8.1 osh-native
33 no-host bin-true 789 289 517 7.6 osh-native-souffle
2 no-host configure.cpython 14,602 6,670 8,356 29.2 bash
10 no-host configure.cpython 14,846 6,639 8,660 29.0 dash
18 no-host configure.cpython 29,125 14,466 16,621 29.2 osh-cpython
26 no-host configure.cpython 14,938 6,582 8,652 29.3 osh-native
34 no-host configure.cpython 15,197 6,614 8,737 29.1 osh-native-souffle
4 no-host configure.ocaml 3,738 1,278 1,503 22.5 bash
12 no-host configure.ocaml 3,682 1,274 1,430 22.5 dash
20 no-host configure.ocaml 4,643 1,860 2,002 22.5 osh-cpython
28 no-host configure.ocaml 3,806 1,344 1,504 22.6 osh-native
36 no-host configure.ocaml 3,881 1,331 1,591 22.6 osh-native-souffle
5 no-host configure.tcc 46 25 22 19.6 bash
13 no-host configure.tcc 41 17 25 19.7 dash
21 no-host configure.tcc 205 136 73 19.7 osh-cpython
29 no-host configure.tcc 49 25 25 19.7 osh-native
37 no-host configure.tcc 49 26 23 19.6 osh-native-souffle
3 no-host configure.util-linux 13,633 5,809 8,324 30.3 bash
11 no-host configure.util-linux 14,477 5,853 9,149 30.3 dash
19 no-host configure.util-linux 31,615 18,190 14,528 30.2 osh-cpython
27 no-host configure.util-linux 14,067 5,636 8,664 30.3 osh-native
35 no-host configure.util-linux 14,454 5,621 9,063 30.5 osh-native-souffle
6 no-host configure.yash 1,827 912 923 26.4 bash
14 no-host configure.yash 1,777 862 919 26.4 dash
22 no-host configure.yash 2,456 1,347 1,152 26.5 osh-cpython
30 no-host configure.yash 1,823 910 919 26.3 osh-native
38 no-host configure.yash 1,864 932 938 26.5 osh-native-souffle
0 no-host hello-world 1 0 1 2.9 bash
8 no-host hello-world 1 0 1 1.4 dash
16 no-host hello-world 56 30 27 18.3 osh-cpython
24 no-host hello-world 3 2 1 6.3 osh-native
32 no-host hello-world 3 1 1 5.9 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,527 0 0 0 16 0 40 1 bash
15 no-host abuild-print-help 1,100 0 0 0 8 0 39 0 dash
23 no-host abuild-print-help 24,098 0 0 0 8 0 45 1 osh-cpython
31 no-host abuild-print-help 3,157 0 0 0 16 0 39 0 osh-native
39 no-host abuild-print-help 3,101 0 0 0 16 0 39 1 osh-native-souffle
1 no-host bin-true 108,344 0 0 0 8 0 1,993 137 bash
9 no-host bin-true 65,477 0 0 0 8 0 1,972 76 dash
17 no-host bin-true 805,767 0 0 0 8 0 1,984 239 osh-cpython
25 no-host bin-true 127,181 0 0 0 16 0 1,985 164 osh-native
33 no-host bin-true 126,174 0 0 0 16 0 1,981 98 osh-native-souffle
2 no-host configure.cpython 1,993,075 0 0 0 32,928 0 14,208 1,089 bash
10 no-host configure.cpython 2,025,063 0 0 0 32,992 0 14,328 1,172 dash
18 no-host configure.cpython 6,102,321 1 0 0 27,512 0 13,048 1,123 osh-cpython
26 no-host configure.cpython 2,014,102 0 0 0 27,520 0 13,092 1,040 osh-native
34 no-host configure.cpython 2,004,557 0 0 0 27,520 0 13,038 1,076 osh-native-souffle
4 no-host configure.ocaml 372,975 0 0 0 5,120 0 1,940 199 bash
12 no-host configure.ocaml 353,665 0 0 0 5,112 0 1,925 179 dash
20 no-host configure.ocaml 617,087 0 0 0 5,112 0 1,851 197 osh-cpython
28 no-host configure.ocaml 377,006 0 0 0 5,120 0 1,920 197 osh-native
36 no-host configure.ocaml 376,148 0 0 0 5,120 0 1,924 202 osh-native-souffle
5 no-host configure.tcc 6,097 0 0 0 104 0 48 5 bash
13 no-host configure.tcc 5,032 0 0 0 96 0 47 13 dash
21 no-host configure.tcc 23,935 0 0 0 96 0 53 5 osh-cpython
29 no-host configure.tcc 6,514 0 0 0 104 0 47 3 osh-native
37 no-host configure.tcc 6,428 0 0 0 104 0 47 11 osh-native-souffle
3 no-host configure.util-linux 1,915,818 2 0 0 42,408 0 18,466 1,271 bash
11 no-host configure.util-linux 2,077,823 2 0 0 42,592 0 17,323 1,392 dash
19 no-host configure.util-linux 5,278,043 4 0 0 39,112 0 17,002 1,364 osh-cpython
27 no-host configure.util-linux 1,809,138 1 0 0 39,120 0 17,018 1,246 osh-native
35 no-host configure.util-linux 1,810,764 1 0 0 39,128 0 17,065 1,074 osh-native-souffle
6 no-host configure.yash 233,114 0 0 0 4,120 0 867 101 bash
14 no-host configure.yash 222,316 0 0 0 3,720 0 866 100 dash
22 no-host configure.yash 368,929 0 0 0 3,728 0 852 126 osh-cpython
30 no-host configure.yash 235,857 0 0 0 3,736 0 847 80 osh-native
38 no-host configure.yash 233,904 0 0 0 3,736 0 845 96 osh-native-souffle
0 no-host hello-world 148 0 0 0 8 0 1 0 bash
8 no-host hello-world 79 0 0 0 8 0 1 2 dash
16 no-host hello-world 3,961 0 0 0 8 0 7 1 osh-cpython
24 no-host hello-world 344 0 0 0 16 0 1 1 osh-native
32 no-host hello-world 336 0 0 0 16 0 1 0 osh-native-souffle

Shell and Host

shell label shell id
bash bash-2b5a2078
dash dash-0ebcf347
osh-cpython osh-2e612a42
osh-native osh-3686bfc4
osh-native-souffle osh-28394e70
host label host id
no-host no-host-e4839999