OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 20 10 467 22 20 23.39 1.08
bin-true no-host 660 528 2,212 794 797 3.35 1.20
configure.cpython no-host 14,761 14,725 28,332 14,855 14,971 1.92 1.01
configure.ocaml no-host 3,763 3,679 4,581 3,774 3,754 1.22 1.00
configure.tcc no-host 46 41 194 48 47 4.21 1.04
configure.util-linux no-host 13,565 14,403 31,217 13,948 13,932 2.30 1.03
configure.yash no-host 1,818 1,744 2,418 1,790 1,789 1.33 0.98
hello-world no-host 1 1 48 3 3 36.77 2.13

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,526 1,080 23,107 3,174 3,101 9.15 1.26
bin-true no-host 106,383 65,481 758,182 125,098 125,137 7.13 1.18
configure.cpython no-host 1,997,606 2,026,739 5,969,231 2,007,547 1,988,491 2.99 1.00
configure.ocaml no-host 373,049 353,408 604,541 376,095 375,683 1.62 1.01
configure.tcc no-host 6,091 4,997 22,978 6,524 6,483 3.77 1.07
configure.util-linux no-host 1,906,192 2,065,274 5,123,977 1,813,197 1,810,528 2.69 0.95
configure.yash no-host 232,599 222,385 363,738 235,481 235,389 1.56 1.01
hello-world no-host 148 76 3,704 347 343 25.03 2.34

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.2 3.4 21.8 9.0 8.4 5.19 2.16
bin-true no-host 2.9 1.6 17.7 8.1 7.6 6.14 2.82
configure.cpython no-host 29.3 29.3 29.3 29.2 29.2 1.00 1.00
configure.ocaml no-host 22.6 22.5 22.6 22.6 22.6 1.00 1.00
configure.tcc no-host 19.7 19.8 19.5 19.7 19.7 0.99 1.00
configure.util-linux no-host 30.5 30.3 30.3 30.2 30.3 1.00 0.99
configure.yash no-host 26.4 26.4 26.4 26.3 26.3 1.00 1.00
hello-world no-host 2.9 1.4 17.4 6.3 5.9 6.05 2.18

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.2 6.3 3,693 3,693 0 690 1,822 1,181 2 0 50,000 0
bin-true no-host 794 1 2 2.6 8.1 154,658 12,892 2,794 9,740 78,291 66,627 3,008 3 50,000 0
configure.cpython no-host 14,855 1 66 15.0 29.2 1,772,640 49,084 28,504 86,393 908,043 778,204 22,694 60 50,000 0
configure.util-linux no-host 13,948 3 63 21.3 30.2 2,703,376 42,721 65,808 109,655 1,320,541 1,273,180 27,391 41 131,616 1
configure.ocaml no-host 3,774 1 2 2.6 22.6 150,010 10,969 5,139 7,525 71,579 70,906 1,448 3 50,000 0
configure.tcc no-host 48 0 0 1.3 19.7 30,499 30,499 0 1,516 14,746 14,237 226 0 50,000 0
configure.yash no-host 1,790 1 2 2.6 26.3 139,667 7,937 6,596 6,332 70,251 63,084 1,803 3 50,000 0
abuild-print-help no-host 22 1 2 2.8 9.0 116,242 64,447 42,657 4,567 54,491 57,184 270 3 85,314 1
hello-world no-host 3 0 0 0.2 5.9 3,692 3,692 0 691 1,822 1,179 2 0 50,000 0
bin-true no-host 797 1 2 2.6 7.6 154,657 12,680 2,732 9,741 78,291 66,625 3,008 3 50,000 0
configure.cpython no-host 14,971 2 66 15.0 29.2 1,772,639 44,180 28,442 86,394 908,043 778,202 22,694 60 50,000 0
configure.util-linux no-host 13,932 3 64 21.3 30.3 2,703,376 44,622 65,745 109,657 1,320,541 1,273,178 27,391 41 131,490 1
configure.ocaml no-host 3,754 1 2 2.6 22.6 150,010 10,906 5,076 7,527 71,579 70,904 1,448 3 50,000 0
configure.tcc no-host 47 0 0 1.3 19.7 30,499 30,499 0 1,518 14,746 14,235 226 0 50,000 0
configure.yash no-host 1,789 1 2 2.6 26.3 139,667 7,875 6,534 6,334 70,251 63,082 1,803 3 50,000 0
abuild-print-help no-host 20 1 2 2.8 8.4 116,241 64,385 42,595 4,568 54,491 57,182 270 3 85,190 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 20 11 10 4.2 bash
15 no-host abuild-print-help 10 4 6 3.4 dash
23 no-host abuild-print-help 467 416 55 21.8 osh-cpython
31 no-host abuild-print-help 22 10 12 9.0 osh-native
39 no-host abuild-print-help 20 8 13 8.4 osh-native-souffle
1 no-host bin-true 660 311 382 2.9 bash
9 no-host bin-true 528 277 261 1.6 dash
17 no-host bin-true 2,212 761 1,700 17.7 osh-cpython
25 no-host bin-true 794 289 521 8.1 osh-native
33 no-host bin-true 797 287 526 7.6 osh-native-souffle
2 no-host configure.cpython 14,761 6,676 8,514 29.3 bash
10 no-host configure.cpython 14,725 6,642 8,530 29.3 dash
18 no-host configure.cpython 28,332 14,289 15,921 29.3 osh-cpython
26 no-host configure.cpython 14,855 6,507 8,642 29.2 osh-native
34 no-host configure.cpython 14,971 6,533 8,620 29.2 osh-native-souffle
4 no-host configure.ocaml 3,763 1,297 1,506 22.6 bash
12 no-host configure.ocaml 3,679 1,247 1,451 22.5 dash
20 no-host configure.ocaml 4,581 1,868 1,925 22.6 osh-cpython
28 no-host configure.ocaml 3,774 1,303 1,511 22.6 osh-native
36 no-host configure.ocaml 3,754 1,302 1,491 22.6 osh-native-souffle
5 no-host configure.tcc 46 23 24 19.7 bash
13 no-host configure.tcc 41 15 26 19.8 dash
21 no-host configure.tcc 194 137 61 19.5 osh-cpython
29 no-host configure.tcc 48 20 28 19.7 osh-native
37 no-host configure.tcc 47 20 28 19.7 osh-native-souffle
3 no-host configure.util-linux 13,565 5,737 8,337 30.5 bash
11 no-host configure.util-linux 14,403 5,801 9,129 30.3 dash
19 no-host configure.util-linux 31,217 17,800 14,525 30.3 osh-cpython
27 no-host configure.util-linux 13,948 5,661 8,510 30.2 osh-native
35 no-host configure.util-linux 13,932 5,633 8,522 30.3 osh-native-souffle
6 no-host configure.yash 1,818 912 916 26.4 bash
14 no-host configure.yash 1,744 875 874 26.4 dash
22 no-host configure.yash 2,418 1,317 1,143 26.4 osh-cpython
30 no-host configure.yash 1,790 921 874 26.3 osh-native
38 no-host configure.yash 1,789 868 926 26.3 osh-native-souffle
0 no-host hello-world 1 0 1 2.9 bash
8 no-host hello-world 1 0 1 1.4 dash
16 no-host hello-world 48 29 20 17.4 osh-cpython
24 no-host hello-world 3 1 2 6.3 osh-native
32 no-host hello-world 3 0 3 5.9 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,526 0 0 0 16 0 41 0 bash
15 no-host abuild-print-help 1,080 0 0 0 8 0 39 0 dash
23 no-host abuild-print-help 23,107 0 0 0 8 0 45 10 osh-cpython
31 no-host abuild-print-help 3,174 0 0 0 16 0 39 0 osh-native
39 no-host abuild-print-help 3,101 0 0 0 16 0 38 3 osh-native-souffle
1 no-host bin-true 106,383 0 0 0 8 0 1,990 184 bash
9 no-host bin-true 65,481 0 0 0 8 0 1,968 119 dash
17 no-host bin-true 758,182 0 0 0 8 0 1,985 150 osh-cpython
25 no-host bin-true 125,098 0 0 0 16 0 1,984 165 osh-native
33 no-host bin-true 125,137 0 0 0 16 0 1,983 275 osh-native-souffle
2 no-host configure.cpython 1,997,606 0 0 0 32,928 0 14,273 1,161 bash
10 no-host configure.cpython 2,026,739 0 0 0 32,992 0 14,330 1,304 dash
18 no-host configure.cpython 5,969,231 0 0 0 27,512 0 13,050 1,311 osh-cpython
26 no-host configure.cpython 2,007,547 0 0 0 27,520 0 13,082 1,169 osh-native
34 no-host configure.cpython 1,988,491 0 0 0 27,520 0 13,023 1,105 osh-native-souffle
4 no-host configure.ocaml 373,049 0 0 0 5,120 0 1,945 184 bash
12 no-host configure.ocaml 353,408 0 0 0 5,112 0 1,935 180 dash
20 no-host configure.ocaml 604,541 0 0 0 5,112 0 1,858 186 osh-cpython
28 no-host configure.ocaml 376,095 0 0 0 5,120 0 1,927 196 osh-native
36 no-host configure.ocaml 375,683 0 0 0 5,120 0 1,929 198 osh-native-souffle
5 no-host configure.tcc 6,091 0 0 0 104 0 49 1 bash
13 no-host configure.tcc 4,997 0 0 0 96 0 47 2 dash
21 no-host configure.tcc 22,978 0 0 0 96 0 52 5 osh-cpython
29 no-host configure.tcc 6,524 0 0 0 104 0 47 2 osh-native
37 no-host configure.tcc 6,483 0 0 0 104 0 47 4 osh-native-souffle
3 no-host configure.util-linux 1,906,192 0 0 0 42,400 0 18,475 1,330 bash
11 no-host configure.util-linux 2,065,274 3 0 0 42,584 0 17,333 1,497 dash
19 no-host configure.util-linux 5,123,977 9 0 0 39,112 0 17,005 1,429 osh-cpython
27 no-host configure.util-linux 1,813,197 0 0 0 39,120 0 17,016 1,197 osh-native
35 no-host configure.util-linux 1,810,528 1 0 0 39,128 0 17,030 1,174 osh-native-souffle
6 no-host configure.yash 232,599 0 0 0 4,120 0 869 94 bash
14 no-host configure.yash 222,385 0 0 0 3,720 0 869 85 dash
22 no-host configure.yash 363,738 0 0 0 3,728 0 857 77 osh-cpython
30 no-host configure.yash 235,481 0 0 0 3,736 0 850 84 osh-native
38 no-host configure.yash 235,389 0 0 0 3,736 0 852 133 osh-native-souffle
0 no-host hello-world 148 0 0 0 8 0 1 0 bash
8 no-host hello-world 76 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,704 0 0 0 8 0 7 1 osh-cpython
24 no-host hello-world 347 0 0 0 16 0 1 1 osh-native
32 no-host hello-world 343 0 0 0 16 0 1 1 osh-native-souffle

Shell and Host

shell label shell id
bash bash-3cb426d1
dash dash-0ebcf347
osh-cpython osh-3ac01879
osh-native osh-bdfd8f15
osh-native-souffle osh-5b2737d8
host label host id
no-host no-host-e5b8d3ee