OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 19 10 483 22 21 24.93 1.16
bin-true no-host 691 554 2,339 846 826 3.38 1.22
configure.cpython no-host 15,533 15,580 30,601 15,549 15,879 1.97 1.00
configure.ocaml no-host 3,904 3,863 4,795 3,918 3,909 1.23 1.00
configure.tcc no-host 49 43 201 52 49 4.10 1.06
configure.util-linux no-host 14,382 15,430 33,150 14,751 14,643 2.30 1.03
configure.yash no-host 1,892 1,841 2,576 1,875 1,878 1.36 0.99
hello-world no-host 2 1 52 3 3 34.06 1.97

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,540 1,092 23,398 3,215 3,148 9.21 1.27
bin-true no-host 107,324 66,412 770,058 127,834 125,892 7.18 1.19
configure.cpython no-host 2,221,124 2,244,082 6,232,186 2,136,809 2,116,702 2.81 0.96
configure.ocaml no-host 410,495 391,462 646,154 412,598 411,938 1.57 1.01
configure.tcc no-host 6,539 5,478 23,729 7,027 6,977 3.63 1.07
configure.util-linux no-host 2,104,117 2,268,061 5,367,896 1,935,284 1,920,436 2.55 0.92
configure.yash no-host 255,877 244,996 386,726 256,539 255,717 1.51 1.00
hello-world no-host 152 79 3,749 354 350 24.66 2.33

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.5 22.2 9.0 8.9 4.97 2.03
bin-true no-host 3.0 1.4 17.8 8.1 7.9 5.91 2.70
configure.cpython no-host 27.5 27.4 27.3 27.5 29.1 0.99 1.00
configure.ocaml no-host 20.6 20.6 20.6 20.6 20.6 1.00 1.00
configure.tcc no-host 19.3 19.3 19.3 19.3 19.4 1.00 1.00
configure.util-linux no-host 30.0 30.1 30.3 30.1 30.3 1.01 1.00
configure.yash no-host 24.4 24.4 24.4 24.5 24.4 1.00 1.01
hello-world no-host 2.8 1.3 17.6 6.4 5.9 6.38 2.33

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num in pool 3 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.3 6.4 3,780 3,780 0 272 1,857 1,228 423 2 0 50,000 0
bin-true no-host 846 1 2 2.3 8.1 154,745 13,282 2,884 1,277 78,326 66,674 8,468 3,008 3 50,000 0
configure.cpython no-host 15,549 1 65 9.5 27.5 1,772,727 49,861 28,594 14,578 908,078 778,251 71,820 22,694 60 50,000 0
configure.util-linux no-host 14,751 3 65 15.0 30.1 2,703,495 29,752 65,877 13,595 1,320,584 1,273,235 96,081 27,391 41 131,754 1
configure.ocaml no-host 3,918 1 2 2.5 20.6 150,101 11,058 5,228 1,100 71,611 70,958 6,432 1,448 3 50,000 0
configure.tcc no-host 52 0 0 1.4 19.3 30,586 30,586 0 329 14,781 14,284 1,192 226 0 50,000 0
configure.yash no-host 1,875 1 2 2.4 24.5 139,754 8,027 6,686 650 70,286 63,131 5,687 1,803 3 50,000 0
abuild-print-help no-host 22 1 2 2.9 9.0 116,329 64,537 42,747 391 54,526 57,231 4,181 270 3 85,494 1
hello-world no-host 3 0 0 0.3 5.9 3,779 3,779 0 271 1,857 1,226 425 2 0 50,000 0
bin-true no-host 826 1 2 2.3 7.9 154,744 13,070 2,822 1,276 78,326 66,672 8,470 3,008 3 50,000 0
configure.cpython no-host 15,879 1 66 9.5 29.1 1,772,726 49,112 28,532 14,577 908,078 778,249 71,822 22,694 60 50,000 0
configure.util-linux no-host 14,643 3 66 14.7 30.3 2,703,495 42,748 65,835 13,595 1,320,584 1,273,233 96,083 27,391 41 131,670 1
configure.ocaml no-host 3,909 1 2 2.5 20.6 150,101 10,996 5,166 1,100 71,611 70,956 6,434 1,448 3 50,000 0
configure.tcc no-host 49 0 0 1.4 19.4 30,586 30,586 0 329 14,781 14,282 1,194 226 0 50,000 0
configure.yash no-host 1,878 1 2 2.4 24.4 139,754 7,965 6,624 650 70,286 63,129 5,689 1,803 3 50,000 0
abuild-print-help no-host 21 1 2 2.9 8.9 116,328 64,475 42,685 390 54,526 57,229 4,183 270 3 85,370 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 19 19 1 4.5 bash
15 no-host abuild-print-help 10 9 1 3.5 dash
23 no-host abuild-print-help 483 444 42 22.2 osh-cpython
31 no-host abuild-print-help 22 16 7 9.0 osh-native
39 no-host abuild-print-help 21 16 5 8.9 osh-native-souffle
1 no-host bin-true 691 513 210 3.0 bash
9 no-host bin-true 554 437 125 1.4 dash
17 no-host bin-true 2,339 1,497 1,121 17.8 osh-cpython
25 no-host bin-true 846 573 285 8.1 osh-native
33 no-host bin-true 826 575 266 7.9 osh-native-souffle
2 no-host configure.cpython 15,533 9,297 6,635 27.5 bash
10 no-host configure.cpython 15,580 9,431 6,567 27.4 dash
18 no-host configure.cpython 30,601 18,546 14,068 27.3 osh-cpython
26 no-host configure.cpython 15,549 9,171 6,639 27.5 osh-native
34 no-host configure.cpython 15,879 9,305 6,577 29.1 osh-native-souffle
4 no-host configure.ocaml 3,904 1,709 1,237 20.6 bash
12 no-host configure.ocaml 3,863 1,666 1,216 20.6 dash
20 no-host configure.ocaml 4,795 2,362 1,650 20.6 osh-cpython
28 no-host configure.ocaml 3,918 1,665 1,294 20.6 osh-native
36 no-host configure.ocaml 3,909 1,725 1,220 20.6 osh-native-souffle
5 no-host configure.tcc 49 28 22 19.3 bash
13 no-host configure.tcc 43 24 20 19.3 dash
21 no-host configure.tcc 201 141 64 19.3 osh-cpython
29 no-host configure.tcc 52 34 19 19.3 osh-native
37 no-host configure.tcc 49 37 13 19.4 osh-native-souffle
3 no-host configure.util-linux 14,382 8,988 5,861 30.0 bash
11 no-host configure.util-linux 15,430 9,179 6,742 30.1 dash
19 no-host configure.util-linux 33,150 22,104 12,201 30.3 osh-cpython
27 no-host configure.util-linux 14,751 8,920 6,028 30.1 osh-native
35 no-host configure.util-linux 14,643 8,770 6,049 30.3 osh-native-souffle
6 no-host configure.yash 1,892 1,042 858 24.4 bash
14 no-host configure.yash 1,841 1,035 809 24.4 dash
22 no-host configure.yash 2,576 1,568 1,053 24.4 osh-cpython
30 no-host configure.yash 1,875 1,134 744 24.5 osh-native
38 no-host configure.yash 1,878 1,106 775 24.4 osh-native-souffle
0 no-host hello-world 2 2 0 2.8 bash
8 no-host hello-world 1 0 1 1.3 dash
16 no-host hello-world 52 28 23 17.6 osh-cpython
24 no-host hello-world 3 0 3 6.4 osh-native
32 no-host hello-world 3 0 3 5.9 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,540 0 0 0 16 0 41 1 bash
15 no-host abuild-print-help 1,092 0 0 0 8 0 39 1 dash
23 no-host abuild-print-help 23,398 0 0 0 8 0 45 7 osh-cpython
31 no-host abuild-print-help 3,215 0 0 0 16 0 39 0 osh-native
39 no-host abuild-print-help 3,148 0 0 0 16 0 39 1 osh-native-souffle
1 no-host bin-true 107,324 0 0 0 8 0 1,991 196 bash
9 no-host bin-true 66,412 0 0 0 8 0 1,988 94 dash
17 no-host bin-true 770,058 0 0 0 8 0 1,997 193 osh-cpython
25 no-host bin-true 127,834 0 0 0 16 0 1,992 67 osh-native
33 no-host bin-true 125,892 0 0 0 16 0 1,991 57 osh-native-souffle
2 no-host configure.cpython 2,221,124 0 0 0 32,928 0 14,506 1,164 bash
10 no-host configure.cpython 2,244,082 0 0 0 32,992 0 14,625 1,015 dash
18 no-host configure.cpython 6,232,186 0 0 0 27,512 0 13,150 1,218 osh-cpython
26 no-host configure.cpython 2,136,809 0 0 0 27,520 0 13,515 888 osh-native
34 no-host configure.cpython 2,116,702 0 0 0 27,520 0 13,402 948 osh-native-souffle
4 no-host configure.ocaml 410,495 0 0 0 5,120 0 1,948 151 bash
12 no-host configure.ocaml 391,462 0 0 0 5,112 0 1,917 136 dash
20 no-host configure.ocaml 646,154 0 0 0 5,112 0 1,861 179 osh-cpython
28 no-host configure.ocaml 412,598 0 0 0 5,120 0 1,931 154 osh-native
36 no-host configure.ocaml 411,938 0 0 0 5,120 0 1,932 144 osh-native-souffle
5 no-host configure.tcc 6,539 0 0 0 104 0 48 14 bash
13 no-host configure.tcc 5,478 0 0 0 96 0 47 3 dash
21 no-host configure.tcc 23,729 0 0 0 96 0 53 8 osh-cpython
29 no-host configure.tcc 7,027 0 0 0 104 0 47 4 osh-native
37 no-host configure.tcc 6,977 0 0 0 104 0 47 3 osh-native-souffle
3 no-host configure.util-linux 2,104,117 1 0 0 42,400 0 18,708 992 bash
11 no-host configure.util-linux 2,268,061 0 0 0 42,592 0 17,398 1,242 dash
19 no-host configure.util-linux 5,367,896 12 0 0 39,136 0 17,001 1,311 osh-cpython
27 no-host configure.util-linux 1,935,284 0 0 0 39,120 0 17,032 1,135 osh-native
35 no-host configure.util-linux 1,920,436 1 0 0 39,136 0 16,925 1,015 osh-native-souffle
6 no-host configure.yash 255,877 0 0 0 4,120 0 867 86 bash
14 no-host configure.yash 244,996 0 0 0 3,720 0 873 78 dash
22 no-host configure.yash 386,726 0 0 0 3,728 0 850 83 osh-cpython
30 no-host configure.yash 256,539 0 0 0 3,736 0 851 82 osh-native
38 no-host configure.yash 255,717 0 0 0 3,736 0 849 86 osh-native-souffle
0 no-host hello-world 152 0 0 0 8 0 0 1 bash
8 no-host hello-world 79 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,749 0 0 0 8 0 7 5 osh-cpython
24 no-host hello-world 354 0 0 0 16 0 1 1 osh-native
32 no-host hello-world 350 0 0 0 16 0 1 0 osh-native-souffle

Shell and Host

shell label shell id
bash bash-5a426404
dash dash-0ebcf347
osh-cpython osh-1c9869bb
osh-native osh-1431cdb6
osh-native-souffle osh-d37c6d1e
host label host id
no-host no-host-556df3ee