OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 19 9 453 22 21 23.65 1.17
bin-true no-host 678 545 2,255 821 817 3.33 1.21
configure.cpython no-host 15,113 15,149 28,877 15,337 15,367 1.91 1.01
configure.ocaml no-host 3,808 3,740 4,661 3,865 3,895 1.22 1.01
configure.tcc no-host 48 42 196 49 50 4.08 1.01
configure.util-linux no-host 14,063 14,981 31,342 14,397 14,351 2.23 1.02
configure.yash no-host 1,829 1,777 2,465 1,872 1,861 1.35 1.02
hello-world no-host 1 1 49 3 3 33.52 1.95

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,542 1,132 23,284 3,174 3,090 9.16 1.25
bin-true no-host 106,386 66,491 765,152 128,256 126,797 7.19 1.21
configure.cpython no-host 2,222,378 2,237,366 6,252,552 2,224,613 2,219,926 2.81 1.00
configure.ocaml no-host 410,593 391,145 648,622 414,107 413,184 1.58 1.01
configure.tcc no-host 6,548 5,486 23,643 6,995 6,936 3.61 1.07
configure.util-linux no-host 2,102,847 2,267,500 5,355,771 1,999,854 1,992,309 2.55 0.95
configure.yash no-host 256,492 244,582 388,249 258,987 257,887 1.51 1.01
hello-world no-host 154 83 3,744 349 343 24.31 2.27

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.5 21.9 8.9 8.7 4.91 2.00
bin-true no-host 2.9 1.6 17.7 8.3 7.9 6.14 2.86
configure.cpython no-host 27.4 27.4 27.4 27.4 27.4 1.00 1.00
configure.ocaml no-host 20.6 20.7 20.6 20.6 20.4 1.00 1.00
configure.tcc no-host 19.3 19.3 19.4 19.3 19.3 1.01 1.00
configure.util-linux no-host 30.1 30.1 30.3 30.1 30.1 1.00 1.00
configure.yash no-host 24.2 24.2 24.4 24.5 24.4 1.01 1.01
hello-world no-host 2.9 1.4 17.6 6.4 5.9 6.09 2.23

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.2 6.4 3,747 3,747 0 694 1,841 1,212 2 0 50,000 0
bin-true no-host 821 1 2 2.6 8.3 154,712 13,192 2,849 9,744 78,310 66,658 3,008 3 50,000 0
configure.cpython no-host 15,337 1 63 15.0 27.4 1,772,694 49,826 28,559 86,397 908,062 778,235 22,694 60 50,000 0
configure.util-linux no-host 14,397 3 61 21.3 30.1 2,703,462 29,118 65,862 109,675 1,320,568 1,273,219 27,391 41 131,724 1
configure.ocaml no-host 3,865 1 2 2.6 20.6 150,068 11,023 5,193 7,531 71,595 70,942 1,448 3 50,000 0
configure.tcc no-host 49 0 0 1.3 19.3 30,553 30,553 0 1,520 14,765 14,268 226 0 50,000 0
configure.yash no-host 1,872 1 2 2.6 24.5 139,721 7,992 6,651 6,336 70,270 63,115 1,803 3 50,000 0
abuild-print-help no-host 22 1 2 2.8 8.9 116,296 64,502 42,712 4,571 54,510 57,215 270 3 85,424 1
hello-world no-host 3 0 0 0.2 5.9 3,746 3,746 0 695 1,841 1,210 2 0 50,000 0
bin-true no-host 817 1 2 2.6 7.9 154,711 12,980 2,787 9,745 78,310 66,656 3,008 3 50,000 0
configure.cpython no-host 15,367 1 61 15.0 27.4 1,772,693 49,077 28,497 86,398 908,062 778,233 22,694 60 50,000 0
configure.util-linux no-host 14,351 3 60 21.3 30.1 2,703,462 42,713 65,800 109,677 1,320,568 1,273,217 27,391 41 131,600 1
configure.ocaml no-host 3,895 1 2 2.6 20.4 150,068 10,961 5,131 7,533 71,595 70,940 1,448 3 50,000 0
configure.tcc no-host 50 0 0 1.3 19.3 30,553 30,553 0 1,522 14,765 14,266 226 0 50,000 0
configure.yash no-host 1,861 1 2 2.6 24.4 139,721 7,930 6,589 6,338 70,270 63,113 1,803 3 50,000 0
abuild-print-help no-host 21 1 2 2.8 8.7 116,295 64,440 42,650 4,572 54,510 57,213 270 3 85,300 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 19 20 0 4.5 bash
15 no-host abuild-print-help 9 8 1 3.5 dash
23 no-host abuild-print-help 453 406 50 21.9 osh-cpython
31 no-host abuild-print-help 22 14 9 8.9 osh-native
39 no-host abuild-print-help 21 16 6 8.7 osh-native-souffle
1 no-host bin-true 678 515 193 2.9 bash
9 no-host bin-true 545 424 132 1.6 dash
17 no-host bin-true 2,255 1,523 994 17.7 osh-cpython
25 no-host bin-true 821 578 260 8.3 osh-native
33 no-host bin-true 817 579 253 7.9 osh-native-souffle
2 no-host configure.cpython 15,113 8,970 6,543 27.4 bash
10 no-host configure.cpython 15,149 9,026 6,544 27.4 dash
18 no-host configure.cpython 28,877 17,879 12,881 27.4 osh-cpython
26 no-host configure.cpython 15,337 9,037 6,569 27.4 osh-native
34 no-host configure.cpython 15,367 8,952 6,549 27.4 osh-native-souffle
4 no-host configure.ocaml 3,808 1,646 1,201 20.6 bash
12 no-host configure.ocaml 3,740 1,596 1,162 20.7 dash
20 no-host configure.ocaml 4,661 2,299 1,574 20.6 osh-cpython
28 no-host configure.ocaml 3,865 1,680 1,225 20.6 osh-native
36 no-host configure.ocaml 3,895 1,711 1,221 20.4 osh-native-souffle
5 no-host configure.tcc 48 37 11 19.3 bash
13 no-host configure.tcc 42 28 14 19.3 dash
21 no-host configure.tcc 196 133 66 19.4 osh-cpython
29 no-host configure.tcc 49 27 22 19.3 osh-native
37 no-host configure.tcc 50 27 24 19.3 osh-native-souffle
3 no-host configure.util-linux 14,063 8,806 5,735 30.1 bash
11 no-host configure.util-linux 14,981 9,213 6,258 30.1 dash
19 no-host configure.util-linux 31,342 21,051 11,387 30.3 osh-cpython
27 no-host configure.util-linux 14,397 8,766 5,834 30.1 osh-native
35 no-host configure.util-linux 14,351 8,751 5,795 30.1 osh-native-souffle
6 no-host configure.yash 1,829 1,065 772 24.2 bash
14 no-host configure.yash 1,777 1,047 734 24.2 dash
22 no-host configure.yash 2,465 1,551 957 24.4 osh-cpython
30 no-host configure.yash 1,872 1,080 795 24.5 osh-native
38 no-host configure.yash 1,861 1,077 787 24.4 osh-native-souffle
0 no-host hello-world 1 1 0 2.9 bash
8 no-host hello-world 1 1 0 1.4 dash
16 no-host hello-world 49 33 16 17.6 osh-cpython
24 no-host hello-world 3 3 0 6.4 osh-native
32 no-host hello-world 3 0 3 5.9 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,542 0 0 0 16 0 41 0 bash
15 no-host abuild-print-help 1,132 0 0 0 8 0 39 1 dash
23 no-host abuild-print-help 23,284 0 0 0 8 0 45 8 osh-cpython
31 no-host abuild-print-help 3,174 0 0 0 16 0 39 2 osh-native
39 no-host abuild-print-help 3,090 0 0 0 16 0 39 1 osh-native-souffle
1 no-host bin-true 106,386 0 0 0 8 0 1,989 50 bash
9 no-host bin-true 66,491 0 0 0 8 0 1,993 43 dash
17 no-host bin-true 765,152 0 0 0 8 0 2,002 104 osh-cpython
25 no-host bin-true 128,256 0 0 0 16 0 1,990 180 osh-native
33 no-host bin-true 126,797 0 0 0 16 0 1,994 160 osh-native-souffle
2 no-host configure.cpython 2,222,378 0 0 0 32,928 0 14,430 1,264 bash
10 no-host configure.cpython 2,237,366 0 0 0 32,992 0 14,632 1,217 dash
18 no-host configure.cpython 6,252,552 1 0 0 27,512 0 13,213 1,359 osh-cpython
26 no-host configure.cpython 2,224,613 0 0 0 27,520 0 13,211 1,256 osh-native
34 no-host configure.cpython 2,219,926 0 0 0 27,520 0 13,279 1,011 osh-native-souffle
4 no-host configure.ocaml 410,593 0 0 0 5,120 0 1,946 173 bash
12 no-host configure.ocaml 391,145 0 0 0 5,112 0 1,880 151 dash
20 no-host configure.ocaml 648,622 0 0 0 5,112 0 1,854 188 osh-cpython
28 no-host configure.ocaml 414,107 0 0 0 5,120 0 1,935 151 osh-native
36 no-host configure.ocaml 413,184 0 0 0 5,120 0 1,934 177 osh-native-souffle
5 no-host configure.tcc 6,548 0 0 0 104 0 49 3 bash
13 no-host configure.tcc 5,486 0 0 0 96 0 47 1 dash
21 no-host configure.tcc 23,643 0 0 0 96 0 53 7 osh-cpython
29 no-host configure.tcc 6,995 0 0 0 104 0 47 2 osh-native
37 no-host configure.tcc 6,936 0 0 0 104 0 47 1 osh-native-souffle
3 no-host configure.util-linux 2,102,847 0 0 0 42,400 0 18,590 1,087 bash
11 no-host configure.util-linux 2,267,500 0 0 0 42,584 0 17,451 1,253 dash
19 no-host configure.util-linux 5,355,771 7 0 0 39,128 0 16,950 1,162 osh-cpython
27 no-host configure.util-linux 1,999,854 0 0 0 39,120 0 16,946 989 osh-native
35 no-host configure.util-linux 1,992,309 0 0 0 39,128 0 16,998 1,008 osh-native-souffle
6 no-host configure.yash 256,492 0 0 0 4,120 0 865 70 bash
14 no-host configure.yash 244,582 0 0 0 3,720 0 868 79 dash
22 no-host configure.yash 388,249 0 0 0 3,728 0 854 112 osh-cpython
30 no-host configure.yash 258,987 0 0 0 3,736 0 847 91 osh-native
38 no-host configure.yash 257,887 0 0 0 3,736 0 851 79 osh-native-souffle
0 no-host hello-world 154 0 0 0 8 0 1 0 bash
8 no-host hello-world 83 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,744 0 0 0 8 0 7 3 osh-cpython
24 no-host hello-world 349 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 343 0 0 0 16 0 1 0 osh-native-souffle

Shell and Host

shell label shell id
bash bash-c18025ff
dash dash-0ebcf347
osh-cpython osh-ad6e968a
osh-native osh-ac42b549
osh-native-souffle osh-c263532a
host label host id
no-host no-host-556df3ee