OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 19 9 473 22 20 25.27 1.17
bin-true no-host 653 527 2,433 791 788 3.73 1.21
configure.cpython no-host 14,755 15,051 29,452 14,926 15,093 2.00 1.01
configure.ocaml no-host 3,820 3,722 4,652 3,806 3,781 1.22 1.00
configure.tcc no-host 47 41 207 48 47 4.41 1.02
configure.util-linux no-host 13,724 14,617 31,800 14,177 13,989 2.32 1.03
configure.yash no-host 1,829 1,788 2,474 1,826 1,796 1.35 1.00
hello-world no-host 1 1 58 3 3 43.61 2.09

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,548 1,105 24,338 3,226 3,099 9.55 1.27
bin-true no-host 106,396 66,423 812,653 129,165 126,129 7.64 1.21
configure.cpython no-host 1,993,162 2,024,554 6,092,958 2,001,398 1,997,184 3.06 1.00
configure.ocaml no-host 373,638 353,824 617,793 376,600 375,869 1.65 1.01
configure.tcc no-host 6,061 5,015 24,065 6,534 6,480 3.97 1.08
configure.util-linux no-host 1,911,353 2,076,043 5,311,045 1,827,284 1,803,714 2.78 0.96
configure.yash no-host 233,234 222,312 370,064 235,035 235,543 1.59 1.01
hello-world no-host 146 75 3,982 345 340 27.27 2.36

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.5 22.7 9.2 8.7 5.10 2.06
bin-true no-host 3.0 1.4 18.5 8.1 7.7 6.15 2.70
configure.cpython no-host 29.3 29.2 29.2 29.3 29.1 1.00 1.00
configure.ocaml no-host 22.6 22.5 22.6 22.6 22.6 1.00 1.00
configure.tcc no-host 19.8 19.6 19.8 19.8 19.6 1.00 1.00
configure.util-linux no-host 30.2 30.2 30.3 30.3 30.3 1.00 1.00
configure.yash no-host 26.1 26.3 26.3 26.4 26.4 1.01 1.01
hello-world no-host 2.9 1.3 18.3 6.4 5.8 6.34 2.23

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.3 6.4 3,805 3,805 0 700 1,870 1,235 2 0 50,000 0
bin-true no-host 791 1 2 2.6 8.1 153,772 12,413 2,901 8,752 76,337 68,683 3,008 3 50,000 0
configure.cpython no-host 14,926 1 63 15.3 29.3 1,773,763 49,828 28,526 91,505 881,465 800,793 25,736 60 50,000 0
configure.util-linux no-host 14,177 3 63 21.6 30.3 2,704,674 53,757 65,725 114,372 1,293,957 1,296,345 32,241 41 131,450 1
configure.ocaml no-host 3,806 1 2 2.6 22.6 149,788 11,154 5,221 7,733 70,844 71,211 1,695 3 50,000 0
configure.tcc no-host 48 0 0 1.3 19.8 30,573 30,573 0 1,542 14,635 14,396 271 0 50,000 0
configure.yash no-host 1,826 1 2 2.6 26.4 139,684 8,041 6,700 6,395 68,253 65,036 2,095 3 50,000 0
abuild-print-help no-host 22 1 2 2.8 9.2 116,321 64,419 42,627 4,544 54,454 57,323 277 3 85,254 1
hello-world no-host 3 0 0 0.3 5.8 3,804 3,804 0 701 1,870 1,233 2 0 50,000 0
bin-true no-host 788 1 2 2.6 7.7 153,771 12,332 2,896 8,753 76,337 68,681 3,008 3 50,000 0
configure.cpython no-host 15,093 1 64 15.3 29.1 1,773,762 49,823 28,521 91,506 881,465 800,791 25,736 60 50,000 0
configure.util-linux no-host 13,989 3 64 21.6 30.3 2,704,674 53,734 65,720 114,374 1,293,957 1,296,343 32,241 41 131,440 1
configure.ocaml no-host 3,781 1 2 2.6 22.6 149,788 11,149 5,216 7,735 70,844 71,209 1,695 3 50,000 0
configure.tcc no-host 47 0 0 1.3 19.6 30,573 30,573 0 1,544 14,635 14,394 271 0 50,000 0
configure.yash no-host 1,796 1 2 2.6 26.4 139,684 8,030 6,689 6,397 68,253 65,034 2,095 3 50,000 0
abuild-print-help no-host 20 1 2 2.8 8.7 116,320 64,414 42,622 4,545 54,454 57,321 277 3 85,244 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 19 9 10 4.5 bash
15 no-host abuild-print-help 9 3 7 3.5 dash
23 no-host abuild-print-help 473 410 66 22.7 osh-cpython
31 no-host abuild-print-help 22 10 12 9.2 osh-native
39 no-host abuild-print-help 20 12 9 8.7 osh-native-souffle
1 no-host bin-true 653 307 380 3.0 bash
9 no-host bin-true 527 272 266 1.4 dash
17 no-host bin-true 2,433 846 1,855 18.5 osh-cpython
25 no-host bin-true 791 289 519 8.1 osh-native
33 no-host bin-true 788 285 520 7.7 osh-native-souffle
2 no-host configure.cpython 14,755 6,632 8,527 29.3 bash
10 no-host configure.cpython 15,051 6,865 8,633 29.2 dash
18 no-host configure.cpython 29,452 14,668 16,734 29.2 osh-cpython
26 no-host configure.cpython 14,926 6,513 8,686 29.3 osh-native
34 no-host configure.cpython 15,093 6,444 8,747 29.1 osh-native-souffle
4 no-host configure.ocaml 3,820 1,327 1,523 22.6 bash
12 no-host configure.ocaml 3,722 1,320 1,424 22.5 dash
20 no-host configure.ocaml 4,652 1,887 1,989 22.6 osh-cpython
28 no-host configure.ocaml 3,806 1,349 1,498 22.6 osh-native
36 no-host configure.ocaml 3,781 1,313 1,507 22.6 osh-native-souffle
5 no-host configure.tcc 47 20 27 19.8 bash
13 no-host configure.tcc 41 19 22 19.6 dash
21 no-host configure.tcc 207 142 68 19.8 osh-cpython
29 no-host configure.tcc 48 21 27 19.8 osh-native
37 no-host configure.tcc 47 24 24 19.6 osh-native-souffle
3 no-host configure.util-linux 13,724 5,869 8,349 30.2 bash
11 no-host configure.util-linux 14,617 5,859 9,270 30.2 dash
19 no-host configure.util-linux 31,800 17,965 14,951 30.3 osh-cpython
27 no-host configure.util-linux 14,177 5,661 8,743 30.3 osh-native
35 no-host configure.util-linux 13,989 5,593 8,618 30.3 osh-native-souffle
6 no-host configure.yash 1,829 928 910 26.1 bash
14 no-host configure.yash 1,788 901 891 26.3 dash
22 no-host configure.yash 2,474 1,398 1,121 26.3 osh-cpython
30 no-host configure.yash 1,826 879 952 26.4 osh-native
38 no-host configure.yash 1,796 896 904 26.4 osh-native-souffle
0 no-host hello-world 1 0 1 2.9 bash
8 no-host hello-world 1 0 1 1.3 dash
16 no-host hello-world 58 33 25 18.3 osh-cpython
24 no-host hello-world 3 1 1 6.4 osh-native
32 no-host hello-world 3 1 1 5.8 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,548 0 0 0 16 0 41 1 bash
15 no-host abuild-print-help 1,105 0 0 0 8 0 39 1 dash
23 no-host abuild-print-help 24,338 0 0 0 8 0 45 8 osh-cpython
31 no-host abuild-print-help 3,226 0 0 0 16 0 38 6 osh-native
39 no-host abuild-print-help 3,099 0 0 0 16 0 39 0 osh-native-souffle
1 no-host bin-true 106,396 0 0 0 8 0 1,984 265 bash
9 no-host bin-true 66,423 0 0 0 8 0 1,969 79 dash
17 no-host bin-true 812,653 0 0 0 8 0 1,997 425 osh-cpython
25 no-host bin-true 129,165 0 0 0 16 0 1,968 118 osh-native
33 no-host bin-true 126,129 0 0 0 16 0 1,982 87 osh-native-souffle
2 no-host configure.cpython 1,993,162 0 0 0 32,928 0 14,315 1,315 bash
10 no-host configure.cpython 2,024,554 0 0 0 32,992 0 14,330 1,030 dash
18 no-host configure.cpython 6,092,958 0 0 0 27,512 0 13,080 1,150 osh-cpython
26 no-host configure.cpython 2,001,398 0 0 0 27,520 0 13,116 1,167 osh-native
34 no-host configure.cpython 1,997,184 0 0 0 27,520 0 13,149 1,124 osh-native-souffle
4 no-host configure.ocaml 373,638 0 0 0 5,120 0 1,954 171 bash
12 no-host configure.ocaml 353,824 0 0 0 5,112 0 1,939 176 dash
20 no-host configure.ocaml 617,793 0 0 0 5,112 0 1,850 157 osh-cpython
28 no-host configure.ocaml 376,600 0 0 0 5,120 0 1,929 219 osh-native
36 no-host configure.ocaml 375,869 0 0 0 5,120 0 1,930 202 osh-native-souffle
5 no-host configure.tcc 6,061 0 0 0 104 0 48 1 bash
13 no-host configure.tcc 5,015 0 0 0 96 0 47 6 dash
21 no-host configure.tcc 24,065 0 0 0 96 0 53 3 osh-cpython
29 no-host configure.tcc 6,534 0 0 0 104 0 47 2 osh-native
37 no-host configure.tcc 6,480 0 0 0 104 0 47 2 osh-native-souffle
3 no-host configure.util-linux 1,911,353 1 0 0 42,400 0 18,454 1,374 bash
11 no-host configure.util-linux 2,076,043 0 0 0 42,584 0 17,308 1,384 dash
19 no-host configure.util-linux 5,311,045 8 0 0 39,144 0 17,012 1,367 osh-cpython
27 no-host configure.util-linux 1,827,284 0 0 0 39,120 0 17,019 1,260 osh-native
35 no-host configure.util-linux 1,803,714 0 0 0 39,128 0 16,996 1,168 osh-native-souffle
6 no-host configure.yash 233,234 0 0 0 4,120 0 871 82 bash
14 no-host configure.yash 222,312 0 0 0 3,728 0 866 97 dash
22 no-host configure.yash 370,064 0 0 0 3,728 0 854 75 osh-cpython
30 no-host configure.yash 235,035 0 0 0 3,736 0 853 99 osh-native
38 no-host configure.yash 235,543 0 0 0 3,736 0 850 86 osh-native-souffle
0 no-host hello-world 146 0 0 0 8 0 1 0 bash
8 no-host hello-world 75 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,982 0 0 0 8 0 7 4 osh-cpython
24 no-host hello-world 345 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 340 0 0 0 16 0 1 1 osh-native-souffle

Shell and Host

shell label shell id
bash bash-8ed6ce4e
dash dash-0ebcf347
osh-cpython osh-9f1604cf
osh-native osh-d3745d1f
osh-native-souffle osh-384e53f2
host label host id
no-host no-host-e4839999