OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 20 9 468 23 21 23.93 1.18
bin-true no-host 660 534 2,243 795 807 3.40 1.20
configure.cpython no-host 15,127 15,116 28,895 15,284 15,380 1.91 1.01
configure.ocaml no-host 3,851 3,748 4,682 3,849 3,858 1.22 1.00
configure.tcc no-host 47 42 199 48 49 4.24 1.03
configure.util-linux no-host 13,995 14,921 32,034 14,295 14,300 2.29 1.02
configure.yash no-host 1,854 1,812 2,473 1,849 1,842 1.33 1.00
hello-world no-host 1 1 50 3 3 36.03 2.05

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,525 1,130 23,059 3,207 3,113 9.13 1.27
bin-true no-host 105,389 66,423 759,238 126,149 128,100 7.20 1.20
configure.cpython no-host 2,000,701 2,025,664 5,980,063 2,001,315 1,994,959 2.99 1.00
configure.ocaml no-host 372,989 353,503 604,882 376,469 376,111 1.62 1.01
configure.tcc no-host 6,065 5,004 22,997 6,529 6,484 3.79 1.08
configure.util-linux no-host 1,904,696 2,072,003 5,130,925 1,806,541 1,815,301 2.69 0.95
configure.yash no-host 232,881 222,214 362,835 234,945 235,280 1.56 1.01
hello-world no-host 147 76 3,704 348 342 25.20 2.37

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.4 21.8 9.0 8.5 4.88 2.03
bin-true no-host 2.9 1.4 17.4 8.1 7.5 6.05 2.82
configure.cpython no-host 29.2 29.1 28.9 29.1 29.3 0.99 1.00
configure.ocaml no-host 22.6 22.6 22.5 22.6 22.6 0.99 1.00
configure.tcc no-host 19.5 19.7 19.6 19.6 19.5 1.00 1.00
configure.util-linux no-host 30.3 30.2 30.2 30.2 30.2 1.00 1.00
configure.yash no-host 26.4 26.4 26.2 26.4 26.4 0.99 1.00
hello-world no-host 2.8 1.4 17.6 6.2 6.0 6.38 2.24

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.2 6.2 3,693 3,693 0 690 1,822 1,181 2 0 50,000 0
bin-true no-host 795 1 2 2.6 8.1 154,658 12,892 2,794 9,740 78,291 66,627 3,008 3 50,000 0
configure.cpython no-host 15,284 1 65 15.0 29.1 1,772,640 49,084 28,504 86,393 908,043 778,204 22,694 60 50,000 0
configure.util-linux no-host 14,295 3 65 21.3 30.2 2,703,376 42,721 65,808 109,655 1,320,541 1,273,180 27,391 41 131,616 1
configure.ocaml no-host 3,849 1 2 2.6 22.6 150,010 10,969 5,139 7,525 71,579 70,906 1,448 3 50,000 0
configure.tcc no-host 48 0 0 1.3 19.6 30,499 30,499 0 1,516 14,746 14,237 226 0 50,000 0
configure.yash no-host 1,849 1 2 2.6 26.4 139,667 7,937 6,596 6,332 70,251 63,084 1,803 3 50,000 0
abuild-print-help no-host 23 1 2 2.8 9.0 116,242 64,447 42,657 4,567 54,491 57,184 270 3 85,314 1
hello-world no-host 3 0 0 0.2 6.0 3,692 3,692 0 691 1,822 1,179 2 0 50,000 0
bin-true no-host 807 1 2 2.6 7.5 154,657 12,680 2,732 9,741 78,291 66,625 3,008 3 50,000 0
configure.cpython no-host 15,380 1 66 15.0 29.3 1,772,639 44,180 28,442 86,394 908,043 778,202 22,694 60 50,000 0
configure.util-linux no-host 14,300 3 66 21.3 30.2 2,703,376 44,622 65,745 109,657 1,320,541 1,273,178 27,391 41 131,490 1
configure.ocaml no-host 3,858 1 2 2.6 22.6 150,010 10,906 5,076 7,527 71,579 70,904 1,448 3 50,000 0
configure.tcc no-host 49 0 0 1.3 19.5 30,499 30,499 0 1,518 14,746 14,235 226 0 50,000 0
configure.yash no-host 1,842 1 2 2.6 26.4 139,667 7,875 6,534 6,334 70,251 63,082 1,803 3 50,000 0
abuild-print-help no-host 21 1 2 2.8 8.5 116,241 64,385 42,595 4,568 54,491 57,182 270 3 85,190 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 20 11 8 4.5 bash
15 no-host abuild-print-help 9 2 8 3.4 dash
23 no-host abuild-print-help 468 419 52 21.8 osh-cpython
31 no-host abuild-print-help 23 10 14 9.0 osh-native
39 no-host abuild-print-help 21 13 9 8.5 osh-native-souffle
1 no-host bin-true 660 297 396 2.9 bash
9 no-host bin-true 534 275 272 1.4 dash
17 no-host bin-true 2,243 783 1,719 17.4 osh-cpython
25 no-host bin-true 795 290 521 8.1 osh-native
33 no-host bin-true 807 292 532 7.5 osh-native-souffle
2 no-host configure.cpython 15,127 6,892 8,653 29.2 bash
10 no-host configure.cpython 15,116 6,844 8,726 29.1 dash
18 no-host configure.cpython 28,895 14,351 16,419 28.9 osh-cpython
26 no-host configure.cpython 15,284 6,750 8,828 29.1 osh-native
34 no-host configure.cpython 15,380 6,756 8,809 29.3 osh-native-souffle
4 no-host configure.ocaml 3,851 1,370 1,521 22.6 bash
12 no-host configure.ocaml 3,748 1,297 1,473 22.6 dash
20 no-host configure.ocaml 4,682 1,897 2,003 22.5 osh-cpython
28 no-host configure.ocaml 3,849 1,340 1,549 22.6 osh-native
36 no-host configure.ocaml 3,858 1,335 1,563 22.6 osh-native-souffle
5 no-host configure.tcc 47 22 25 19.5 bash
13 no-host configure.tcc 42 20 22 19.7 dash
21 no-host configure.tcc 199 140 64 19.6 osh-cpython
29 no-host configure.tcc 48 23 25 19.6 osh-native
37 no-host configure.tcc 49 24 25 19.5 osh-native-souffle
3 no-host configure.util-linux 13,995 6,064 8,434 30.3 bash
11 no-host configure.util-linux 14,921 6,043 9,399 30.2 dash
19 no-host configure.util-linux 32,034 18,358 14,824 30.2 osh-cpython
27 no-host configure.util-linux 14,295 5,670 8,845 30.2 osh-native
35 no-host configure.util-linux 14,300 5,685 8,836 30.2 osh-native-souffle
6 no-host configure.yash 1,854 919 938 26.4 bash
14 no-host configure.yash 1,812 936 880 26.4 dash
22 no-host configure.yash 2,473 1,370 1,146 26.2 osh-cpython
30 no-host configure.yash 1,849 914 941 26.4 osh-native
38 no-host configure.yash 1,842 895 953 26.4 osh-native-souffle
0 no-host hello-world 1 0 1 2.8 bash
8 no-host hello-world 1 0 1 1.4 dash
16 no-host hello-world 50 36 14 17.6 osh-cpython
24 no-host hello-world 3 0 3 6.2 osh-native
32 no-host hello-world 3 2 1 6.0 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,525 0 0 0 16 0 42 1 bash
15 no-host abuild-print-help 1,130 0 0 0 8 0 39 0 dash
23 no-host abuild-print-help 23,059 0 0 0 8 0 45 5 osh-cpython
31 no-host abuild-print-help 3,207 0 0 0 16 0 39 5 osh-native
39 no-host abuild-print-help 3,113 0 0 0 16 0 39 5 osh-native-souffle
1 no-host bin-true 105,389 0 0 0 8 0 1,980 134 bash
9 no-host bin-true 66,423 0 0 0 8 0 1,974 167 dash
17 no-host bin-true 759,238 0 0 0 8 0 1,979 214 osh-cpython
25 no-host bin-true 126,149 0 0 0 16 0 1,973 110 osh-native
33 no-host bin-true 128,100 0 0 0 16 0 1,989 292 osh-native-souffle
2 no-host configure.cpython 2,000,701 0 0 0 32,928 0 14,200 1,166 bash
10 no-host configure.cpython 2,025,664 0 0 0 32,992 0 14,333 1,113 dash
18 no-host configure.cpython 5,980,063 0 0 0 27,512 0 13,051 1,209 osh-cpython
26 no-host configure.cpython 2,001,315 0 0 0 27,520 0 13,111 1,189 osh-native
34 no-host configure.cpython 1,994,959 0 0 0 27,520 0 13,012 1,205 osh-native-souffle
4 no-host configure.ocaml 372,989 0 0 0 5,120 0 1,953 201 bash
12 no-host configure.ocaml 353,503 0 0 0 5,112 0 1,923 171 dash
20 no-host configure.ocaml 604,882 0 0 0 5,112 0 1,858 186 osh-cpython
28 no-host configure.ocaml 376,469 0 0 0 5,120 0 1,924 207 osh-native
36 no-host configure.ocaml 376,111 0 0 0 5,120 0 1,936 174 osh-native-souffle
5 no-host configure.tcc 6,065 0 0 0 104 0 49 1 bash
13 no-host configure.tcc 5,004 0 0 0 96 0 47 2 dash
21 no-host configure.tcc 22,997 0 0 0 96 0 53 4 osh-cpython
29 no-host configure.tcc 6,529 0 0 0 104 0 46 3 osh-native
37 no-host configure.tcc 6,484 0 0 0 104 0 47 2 osh-native-souffle
3 no-host configure.util-linux 1,904,696 1 0 0 42,400 0 18,465 1,275 bash
11 no-host configure.util-linux 2,072,003 0 0 0 42,584 0 17,319 1,475 dash
19 no-host configure.util-linux 5,130,925 10 0 0 39,112 0 17,013 1,412 osh-cpython
27 no-host configure.util-linux 1,806,541 1 0 0 39,120 0 16,991 1,224 osh-native
35 no-host configure.util-linux 1,815,301 0 0 0 39,136 0 16,943 1,273 osh-native-souffle
6 no-host configure.yash 232,881 0 0 0 4,120 0 871 99 bash
14 no-host configure.yash 222,214 0 0 0 3,720 0 869 100 dash
22 no-host configure.yash 362,835 0 0 0 3,728 0 855 94 osh-cpython
30 no-host configure.yash 234,945 0 0 0 3,736 0 848 103 osh-native
38 no-host configure.yash 235,280 0 0 0 3,736 0 851 86 osh-native-souffle
0 no-host hello-world 147 0 0 0 8 0 1 0 bash
8 no-host hello-world 76 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,704 0 0 0 8 0 7 1 osh-cpython
24 no-host hello-world 348 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 342 0 0 0 16 0 1 9 osh-native-souffle

Shell and Host

shell label shell id
bash bash-48d9e5ab
dash dash-0ebcf347
osh-cpython osh-a6f9c77f
osh-native osh-b7662302
osh-native-souffle osh-2ad88969
host label host id
no-host no-host-e5b8d3ee