OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 19 9 454 21 20 24.38 1.15
bin-true no-host 655 520 2,236 795 781 3.41 1.21
configure.cpython no-host 14,271 14,367 27,510 14,526 14,668 1.93 1.02
configure.ocaml no-host 3,694 3,627 4,486 3,722 3,684 1.21 1.01
configure.tcc no-host 46 40 191 47 46 4.12 1.02
configure.util-linux no-host 13,327 14,209 30,194 13,675 13,645 2.27 1.03
configure.yash no-host 1,743 1,696 2,355 1,763 1,754 1.35 1.01
hello-world no-host 1 1 48 3 3 38.63 2.26

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,522 1,093 23,118 3,171 3,112 9.17 1.26
bin-true no-host 105,399 66,409 759,173 125,191 126,109 7.20 1.19
configure.cpython no-host 1,995,856 2,029,431 5,969,946 2,004,899 1,989,937 2.99 1.00
configure.ocaml no-host 373,211 354,129 604,749 376,293 375,773 1.62 1.01
configure.tcc no-host 6,082 5,001 22,981 6,508 6,481 3.78 1.07
configure.util-linux no-host 1,905,735 2,067,824 5,122,717 1,804,863 1,811,480 2.69 0.95
configure.yash no-host 232,744 220,556 362,137 235,536 234,030 1.56 1.01
hello-world no-host 147 76 3,702 347 343 25.18 2.36

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.5 21.9 9.0 8.5 4.91 2.03
bin-true no-host 2.9 1.4 17.6 8.1 7.6 6.09 2.82
configure.cpython no-host 29.1 29.3 29.2 29.1 29.2 1.00 1.00
configure.ocaml no-host 22.6 22.6 22.6 22.6 22.6 1.00 1.00
configure.tcc no-host 19.7 19.7 19.8 19.6 19.7 1.01 0.99
configure.util-linux no-host 30.3 30.3 30.2 30.3 30.3 1.00 1.00
configure.yash no-host 26.4 26.2 26.3 26.4 26.4 1.00 1.00
hello-world no-host 2.8 1.4 17.6 6.2 5.9 6.38 2.24

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.2 6.2 3,693 3,693 0 690 1,822 1,181 2 0 50,000 0
bin-true no-host 795 1 2 2.6 8.1 154,658 12,892 2,794 9,740 78,291 66,627 3,008 3 50,000 0
configure.cpython no-host 14,526 1 60 15.0 29.1 1,772,640 49,084 28,504 86,393 908,043 778,204 22,694 60 50,000 0
configure.util-linux no-host 13,675 3 59 21.3 30.3 2,703,376 42,721 65,808 109,655 1,320,541 1,273,180 27,391 41 131,616 1
configure.ocaml no-host 3,722 1 2 2.6 22.6 150,010 10,969 5,139 7,525 71,579 70,906 1,448 3 50,000 0
configure.tcc no-host 47 0 0 1.3 19.6 30,499 30,499 0 1,516 14,746 14,237 226 0 50,000 0
configure.yash no-host 1,763 1 2 2.6 26.4 139,667 7,937 6,596 6,332 70,251 63,084 1,803 3 50,000 0
abuild-print-help no-host 21 1 2 2.8 9.0 116,242 64,447 42,657 4,567 54,491 57,184 270 3 85,314 1
hello-world no-host 3 0 0 0.2 5.9 3,692 3,692 0 691 1,822 1,179 2 0 50,000 0
bin-true no-host 781 1 2 2.6 7.6 154,657 12,680 2,732 9,741 78,291 66,625 3,008 3 50,000 0
configure.cpython no-host 14,668 1 60 15.0 29.2 1,772,639 44,180 28,442 86,394 908,043 778,202 22,694 60 50,000 0
configure.util-linux no-host 13,645 3 58 21.3 30.3 2,703,376 44,622 65,745 109,657 1,320,541 1,273,178 27,391 41 131,490 1
configure.ocaml no-host 3,684 1 2 2.6 22.6 150,010 10,906 5,076 7,527 71,579 70,904 1,448 3 50,000 0
configure.tcc no-host 46 0 0 1.3 19.7 30,499 30,499 0 1,518 14,746 14,235 226 0 50,000 0
configure.yash no-host 1,754 1 2 2.6 26.4 139,667 7,875 6,534 6,334 70,251 63,082 1,803 3 50,000 0
abuild-print-help no-host 20 1 2 2.8 8.5 116,241 64,385 42,595 4,568 54,491 57,182 270 3 85,190 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 19 9 10 4.5 bash
15 no-host abuild-print-help 9 3 7 3.5 dash
23 no-host abuild-print-help 454 401 57 21.9 osh-cpython
31 no-host abuild-print-help 21 13 9 9.0 osh-native
39 no-host abuild-print-help 20 9 12 8.5 osh-native-souffle
1 no-host bin-true 655 319 368 2.9 bash
9 no-host bin-true 520 282 249 1.4 dash
17 no-host bin-true 2,236 766 1,714 17.6 osh-cpython
25 no-host bin-true 795 298 513 8.1 osh-native
33 no-host bin-true 781 295 502 7.6 osh-native-souffle
2 no-host configure.cpython 14,271 6,525 8,153 29.1 bash
10 no-host configure.cpython 14,367 6,564 8,248 29.3 dash
18 no-host configure.cpython 27,510 13,662 15,653 29.2 osh-cpython
26 no-host configure.cpython 14,526 6,346 8,469 29.1 osh-native
34 no-host configure.cpython 14,668 6,325 8,451 29.2 osh-native-souffle
4 no-host configure.ocaml 3,694 1,307 1,422 22.6 bash
12 no-host configure.ocaml 3,627 1,258 1,391 22.6 dash
20 no-host configure.ocaml 4,486 1,782 1,909 22.6 osh-cpython
28 no-host configure.ocaml 3,722 1,243 1,520 22.6 osh-native
36 no-host configure.ocaml 3,684 1,282 1,441 22.6 osh-native-souffle
5 no-host configure.tcc 46 23 24 19.7 bash
13 no-host configure.tcc 40 20 20 19.7 dash
21 no-host configure.tcc 191 130 65 19.8 osh-cpython
29 no-host configure.tcc 47 16 32 19.6 osh-native
37 no-host configure.tcc 46 25 22 19.7 osh-native-souffle
3 no-host configure.util-linux 13,327 5,761 8,057 30.3 bash
11 no-host configure.util-linux 14,209 5,780 8,952 30.3 dash
19 no-host configure.util-linux 30,194 17,303 13,951 30.2 osh-cpython
27 no-host configure.util-linux 13,675 5,462 8,438 30.3 osh-native
35 no-host configure.util-linux 13,645 5,434 8,433 30.3 osh-native-souffle
6 no-host configure.yash 1,743 874 879 26.4 bash
14 no-host configure.yash 1,696 886 814 26.2 dash
22 no-host configure.yash 2,355 1,301 1,093 26.3 osh-cpython
30 no-host configure.yash 1,763 887 881 26.4 osh-native
38 no-host configure.yash 1,754 934 825 26.4 osh-native-souffle
0 no-host hello-world 1 0 1 2.8 bash
8 no-host hello-world 1 0 1 1.4 dash
16 no-host hello-world 48 27 21 17.6 osh-cpython
24 no-host hello-world 3 1 2 6.2 osh-native
32 no-host hello-world 3 2 1 5.9 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,522 0 0 0 16 0 41 1 bash
15 no-host abuild-print-help 1,093 0 0 0 8 0 39 0 dash
23 no-host abuild-print-help 23,118 0 0 0 8 0 45 3 osh-cpython
31 no-host abuild-print-help 3,171 0 0 0 16 0 39 5 osh-native
39 no-host abuild-print-help 3,112 0 0 0 16 0 39 2 osh-native-souffle
1 no-host bin-true 105,399 0 0 0 8 0 1,984 252 bash
9 no-host bin-true 66,409 0 0 0 8 0 1,980 100 dash
17 no-host bin-true 759,173 0 0 0 8 0 1,992 615 osh-cpython
25 no-host bin-true 125,191 0 0 0 16 0 1,983 362 osh-native
33 no-host bin-true 126,109 0 0 0 16 0 1,988 157 osh-native-souffle
2 no-host configure.cpython 1,995,856 0 0 0 32,928 0 14,237 1,181 bash
10 no-host configure.cpython 2,029,431 0 0 0 32,992 0 14,327 1,234 dash
18 no-host configure.cpython 5,969,946 0 0 0 27,512 0 13,031 1,285 osh-cpython
26 no-host configure.cpython 2,004,899 0 0 0 27,520 0 13,080 1,323 osh-native
34 no-host configure.cpython 1,989,937 1 0 0 27,520 0 13,008 1,287 osh-native-souffle
4 no-host configure.ocaml 373,211 0 0 0 5,120 0 1,947 170 bash
12 no-host configure.ocaml 354,129 0 0 0 5,112 0 1,943 206 dash
20 no-host configure.ocaml 604,749 0 0 0 5,112 0 1,851 182 osh-cpython
28 no-host configure.ocaml 376,293 0 0 0 5,120 0 1,928 217 osh-native
36 no-host configure.ocaml 375,773 0 0 0 5,120 0 1,928 201 osh-native-souffle
5 no-host configure.tcc 6,082 0 0 0 104 0 49 8 bash
13 no-host configure.tcc 5,001 0 0 0 96 0 47 5 dash
21 no-host configure.tcc 22,981 0 0 0 96 0 53 5 osh-cpython
29 no-host configure.tcc 6,508 0 0 0 104 0 47 1 osh-native
37 no-host configure.tcc 6,481 0 0 0 104 0 47 2 osh-native-souffle
3 no-host configure.util-linux 1,905,735 1 0 0 42,400 0 18,509 1,163 bash
11 no-host configure.util-linux 2,067,824 0 0 0 42,584 0 17,275 1,522 dash
19 no-host configure.util-linux 5,122,717 10 0 0 39,144 0 17,048 1,317 osh-cpython
27 no-host configure.util-linux 1,804,863 0 0 0 39,120 0 17,024 1,360 osh-native
35 no-host configure.util-linux 1,811,480 0 0 0 39,128 0 17,064 1,283 osh-native-souffle
6 no-host configure.yash 232,744 0 0 0 4,120 0 870 92 bash
14 no-host configure.yash 220,556 0 0 0 3,728 0 870 109 dash
22 no-host configure.yash 362,137 0 0 0 3,728 0 855 98 osh-cpython
30 no-host configure.yash 235,536 0 0 0 3,736 0 852 107 osh-native
38 no-host configure.yash 234,030 0 0 0 3,736 0 851 173 osh-native-souffle
0 no-host hello-world 147 0 0 0 8 0 1 0 bash
8 no-host hello-world 76 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,702 0 0 0 8 0 7 3 osh-cpython
24 no-host hello-world 347 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 343 0 0 0 16 0 1 1 osh-native-souffle

Shell and Host

shell label shell id
bash bash-0adb4f50
dash dash-0ebcf347
osh-cpython osh-1781ed25
osh-native osh-3ecb6953
osh-native-souffle osh-9d038577
host label host id
no-host no-host-e5b8d3ee