OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 21 10 495 23 22 23.97 1.11
bin-true no-host 689 546 2,456 827 845 3.56 1.20
configure.cpython no-host 15,485 15,495 30,156 15,767 15,629 1.95 1.02
configure.ocaml no-host 3,904 3,849 4,809 3,922 3,920 1.23 1.00
configure.tcc no-host 50 43 213 50 50 4.27 1.01
configure.util-linux no-host 14,375 15,213 32,843 14,748 14,693 2.28 1.03
configure.yash no-host 1,892 1,876 2,621 1,894 1,891 1.39 1.00
hello-world no-host 1 1 61 3 3 44.66 2.08

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,556 1,114 25,051 3,203 3,071 9.80 1.25
bin-true no-host 109,294 67,458 798,985 129,236 126,216 7.31 1.18
configure.cpython no-host 1,998,763 2,023,381 6,109,710 2,010,062 1,998,408 3.06 1.01
configure.ocaml no-host 373,676 353,911 615,996 376,820 376,565 1.65 1.01
configure.tcc no-host 6,102 5,013 24,003 6,512 6,456 3.93 1.07
configure.util-linux no-host 1,915,605 2,073,559 5,276,783 1,823,052 1,812,230 2.75 0.95
configure.yash no-host 233,357 222,203 368,856 235,206 235,486 1.58 1.01
hello-world no-host 146 76 3,968 344 340 27.18 2.36

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.5 22.9 9.2 8.5 5.13 2.06
bin-true no-host 3.0 1.4 18.7 8.3 7.5 6.19 2.74
configure.cpython no-host 29.3 29.1 29.1 29.2 29.2 0.99 1.00
configure.ocaml no-host 22.5 22.6 22.6 22.6 22.6 1.01 1.01
configure.tcc no-host 19.6 19.7 19.7 19.6 19.6 1.01 1.00
configure.util-linux no-host 30.2 30.3 30.2 30.3 30.3 1.00 1.00
configure.yash no-host 26.3 26.4 26.4 26.5 26.5 1.00 1.01
hello-world no-host 2.9 1.4 18.4 6.3 5.8 6.38 2.18

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.3 6.3 3,810 3,810 0 702 1,872 1,236 2 0 50,000 0
bin-true no-host 827 1 2 2.6 8.3 156,777 15,416 2,903 8,754 77,339 70,684 3,008 3 50,000 0
configure.cpython no-host 15,767 2 76 15.5 29.2 1,785,018 30,561 28,528 91,507 886,576 806,935 25,736 61 50,000 0
configure.util-linux no-host 14,748 3 70 21.6 30.3 2,709,219 53,763 65,707 114,374 1,295,039 1,299,806 32,241 41 131,414 1
configure.ocaml no-host 3,922 1 2 2.6 22.6 149,864 11,180 5,223 7,735 70,876 71,253 1,695 3 50,000 0
configure.tcc no-host 50 0 0 1.3 19.6 30,580 30,580 0 1,544 14,638 14,398 271 0 50,000 0
configure.yash no-host 1,894 1 2 2.6 26.5 139,886 8,047 6,702 6,399 68,341 65,146 2,095 3 50,000 0
abuild-print-help no-host 23 1 2 2.8 9.2 116,342 64,421 42,629 4,546 54,464 57,332 277 3 85,258 1
hello-world no-host 3 0 0 0.3 5.8 3,809 3,809 0 703 1,872 1,234 2 0 50,000 0
bin-true no-host 845 1 2 2.6 7.5 156,776 15,411 2,898 8,755 77,339 70,682 3,008 3 50,000 0
configure.cpython no-host 15,629 2 75 15.5 29.2 1,785,017 30,548 28,523 91,508 886,576 806,933 25,736 61 50,000 0
configure.util-linux no-host 14,693 3 69 21.6 30.3 2,709,219 53,740 65,702 114,376 1,295,039 1,299,804 32,241 41 131,404 1
configure.ocaml no-host 3,920 1 2 2.6 22.6 149,864 11,175 5,218 7,737 70,876 71,251 1,695 3 50,000 0
configure.tcc no-host 50 0 0 1.3 19.6 30,580 30,580 0 1,546 14,638 14,396 271 0 50,000 0
configure.yash no-host 1,891 1 2 2.6 26.5 139,886 8,036 6,691 6,401 68,341 65,144 2,095 3 50,000 0
abuild-print-help no-host 22 1 2 2.8 8.5 116,341 64,416 42,624 4,547 54,464 57,330 277 3 85,248 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 21 12 9 4.5 bash
15 no-host abuild-print-help 10 6 4 3.5 dash
23 no-host abuild-print-help 495 436 63 22.9 osh-cpython
31 no-host abuild-print-help 23 9 15 9.2 osh-native
39 no-host abuild-print-help 22 14 9 8.5 osh-native-souffle
1 no-host bin-true 689 286 437 3.0 bash
9 no-host bin-true 546 278 282 1.4 dash
17 no-host bin-true 2,456 842 1,879 18.7 osh-cpython
25 no-host bin-true 827 290 556 8.3 osh-native
33 no-host bin-true 845 296 565 7.5 osh-native-souffle
2 no-host configure.cpython 15,485 7,124 8,806 29.3 bash
10 no-host configure.cpython 15,495 7,132 8,810 29.1 dash
18 no-host configure.cpython 30,156 15,155 16,956 29.1 osh-cpython
26 no-host configure.cpython 15,767 6,952 9,138 29.2 osh-native
34 no-host configure.cpython 15,629 6,792 9,142 29.2 osh-native-souffle
4 no-host configure.ocaml 3,904 1,397 1,548 22.5 bash
12 no-host configure.ocaml 3,849 1,407 1,466 22.6 dash
20 no-host configure.ocaml 4,809 2,025 2,016 22.6 osh-cpython
28 no-host configure.ocaml 3,922 1,408 1,559 22.6 osh-native
36 no-host configure.ocaml 3,920 1,436 1,529 22.6 osh-native-souffle
5 no-host configure.tcc 50 25 25 19.6 bash
13 no-host configure.tcc 43 20 23 19.7 dash
21 no-host configure.tcc 213 150 66 19.7 osh-cpython
29 no-host configure.tcc 50 23 28 19.6 osh-native
37 no-host configure.tcc 50 24 26 19.6 osh-native-souffle
3 no-host configure.util-linux 14,375 6,209 8,697 30.2 bash
11 no-host configure.util-linux 15,213 6,271 9,510 30.3 dash
19 no-host configure.util-linux 32,843 18,746 15,229 30.2 osh-cpython
27 no-host configure.util-linux 14,748 5,909 9,075 30.3 osh-native
35 no-host configure.util-linux 14,693 5,878 9,050 30.3 osh-native-souffle
6 no-host configure.yash 1,892 980 923 26.3 bash
14 no-host configure.yash 1,876 947 934 26.4 dash
22 no-host configure.yash 2,621 1,450 1,218 26.4 osh-cpython
30 no-host configure.yash 1,894 957 942 26.5 osh-native
38 no-host configure.yash 1,891 944 952 26.5 osh-native-souffle
0 no-host hello-world 1 0 1 2.9 bash
8 no-host hello-world 1 1 0 1.4 dash
16 no-host hello-world 61 38 23 18.4 osh-cpython
24 no-host hello-world 3 2 1 6.3 osh-native
32 no-host hello-world 3 1 2 5.8 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,556 0 0 0 16 0 41 1 bash
15 no-host abuild-print-help 1,114 0 0 0 8 0 39 4 dash
23 no-host abuild-print-help 25,051 0 0 0 8 0 44 9 osh-cpython
31 no-host abuild-print-help 3,203 0 0 0 16 0 39 1 osh-native
39 no-host abuild-print-help 3,071 0 0 0 16 0 39 0 osh-native-souffle
1 no-host bin-true 109,294 0 0 0 8 0 1,981 262 bash
9 no-host bin-true 67,458 0 0 0 8 0 1,980 87 dash
17 no-host bin-true 798,985 0 0 0 8 0 1,985 143 osh-cpython
25 no-host bin-true 129,236 0 0 0 16 0 1,988 134 osh-native
33 no-host bin-true 126,216 0 0 0 16 0 1,974 270 osh-native-souffle
2 no-host configure.cpython 1,998,763 0 0 0 32,928 0 14,283 1,309 bash
10 no-host configure.cpython 2,023,381 0 0 0 32,992 0 14,322 1,385 dash
18 no-host configure.cpython 6,109,710 1 0 0 27,512 0 13,039 1,287 osh-cpython
26 no-host configure.cpython 2,010,062 0 0 0 27,520 0 13,177 1,149 osh-native
34 no-host configure.cpython 1,998,408 0 0 0 27,520 0 13,132 1,066 osh-native-souffle
4 no-host configure.ocaml 373,676 0 0 0 5,120 0 1,955 198 bash
12 no-host configure.ocaml 353,911 0 0 0 5,112 0 1,945 178 dash
20 no-host configure.ocaml 615,996 1 0 0 5,112 0 1,879 216 osh-cpython
28 no-host configure.ocaml 376,820 0 0 0 5,120 0 1,937 210 osh-native
36 no-host configure.ocaml 376,565 0 0 0 5,120 0 1,925 243 osh-native-souffle
5 no-host configure.tcc 6,102 0 0 0 104 0 46 1 bash
13 no-host configure.tcc 5,013 0 0 0 96 0 47 2 dash
21 no-host configure.tcc 24,003 0 0 0 96 0 54 6 osh-cpython
29 no-host configure.tcc 6,512 0 0 0 104 0 47 1 osh-native
37 no-host configure.tcc 6,456 0 0 0 104 0 47 4 osh-native-souffle
3 no-host configure.util-linux 1,915,605 0 0 0 42,400 0 18,414 1,333 bash
11 no-host configure.util-linux 2,073,559 1 0 0 42,584 0 17,288 1,510 dash
19 no-host configure.util-linux 5,276,783 10 0 0 39,112 0 17,128 1,289 osh-cpython
27 no-host configure.util-linux 1,823,052 0 0 0 39,128 0 17,020 1,282 osh-native
35 no-host configure.util-linux 1,812,230 1 0 0 39,136 0 16,970 1,199 osh-native-souffle
6 no-host configure.yash 233,357 0 0 0 4,120 0 870 88 bash
14 no-host configure.yash 222,203 0 0 0 3,720 0 868 85 dash
22 no-host configure.yash 368,856 0 0 0 3,728 0 853 96 osh-cpython
30 no-host configure.yash 235,206 0 0 0 3,736 0 847 95 osh-native
38 no-host configure.yash 235,486 0 0 0 3,736 0 850 94 osh-native-souffle
0 no-host hello-world 146 0 0 0 8 0 1 0 bash
8 no-host hello-world 76 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,968 0 0 0 8 0 7 2 osh-cpython
24 no-host hello-world 344 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 340 0 0 0 16 0 1 1 osh-native-souffle

Shell and Host

shell label shell id
bash bash-4aa64418
dash dash-0ebcf347
osh-cpython osh-d592bc06
osh-native osh-a874fa4e
osh-native-souffle osh-161321ea
host label host id
no-host no-host-e4839999