OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 18 9 470 21 20 25.98 1.19
bin-true no-host 632 514 2,269 758 764 3.59 1.20
configure.cpython no-host 14,288 14,364 28,003 14,453 14,591 1.96 1.01
configure.ocaml no-host 3,718 3,616 4,527 3,701 3,697 1.22 1.00
configure.tcc no-host 45 40 201 47 47 4.46 1.05
configure.util-linux no-host 13,286 14,255 30,591 13,642 13,577 2.30 1.03
configure.yash no-host 1,746 1,696 2,391 1,753 1,756 1.37 1.00
hello-world no-host 1 1 57 3 3 45.08 2.20

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,552 1,111 24,505 3,162 3,071 9.60 1.24
bin-true no-host 105,382 65,463 799,444 126,189 124,115 7.59 1.20
configure.cpython no-host 1,993,697 2,026,658 6,124,711 2,016,435 2,008,795 3.07 1.01
configure.ocaml no-host 373,554 353,594 617,891 376,865 376,449 1.65 1.01
configure.tcc no-host 6,075 5,004 23,934 6,487 6,462 3.94 1.07
configure.util-linux no-host 1,918,209 2,068,221 5,351,258 1,826,666 1,817,340 2.79 0.95
configure.yash no-host 234,092 222,191 368,854 235,810 235,027 1.58 1.01
hello-world no-host 151 76 3,980 345 342 26.36 2.28

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.3 23.0 9.2 8.4 5.16 2.06
bin-true no-host 3.0 1.4 18.5 8.3 7.9 6.15 2.74
configure.cpython no-host 29.2 29.2 29.2 29.1 29.0 1.00 1.00
configure.ocaml no-host 22.6 22.5 22.6 22.5 22.5 1.00 0.99
configure.tcc no-host 19.6 19.7 19.8 19.7 19.6 1.01 1.01
configure.util-linux no-host 30.3 30.3 30.2 30.3 30.3 1.00 1.00
configure.yash no-host 26.2 26.3 26.2 26.3 26.3 1.00 1.00
hello-world no-host 2.9 1.4 18.4 6.3 6.0 6.39 2.18

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.3 6.3 3,810 3,810 0 702 1,870 1,238 2 0 50,000 0
bin-true no-host 758 1 2 2.6 8.3 156,777 15,416 2,903 8,754 77,337 70,686 3,008 3 50,000 0
configure.cpython no-host 14,453 1 69 15.2 29.1 1,785,027 49,179 25,898 91,515 886,574 806,938 25,736 62 50,000 0
configure.util-linux no-host 13,642 3 65 21.6 30.3 2,709,227 56,216 66,561 114,377 1,295,038 1,299,812 32,241 42 133,122 1
configure.ocaml no-host 3,701 1 2 2.6 22.5 149,879 12,467 6,510 7,749 70,874 71,256 1,695 3 50,000 0
configure.tcc no-host 47 0 0 1.3 19.7 30,580 30,580 0 1,544 14,636 14,400 271 0 50,000 0
configure.yash no-host 1,753 1 2 2.6 26.3 139,886 8,047 6,702 6,399 68,339 65,148 2,095 3 50,000 0
abuild-print-help no-host 21 1 2 2.8 9.2 116,340 64,422 42,627 4,546 54,461 57,333 277 3 85,254 1
hello-world no-host 3 0 0 0.3 6.0 3,809 3,809 0 703 1,870 1,236 2 0 50,000 0
bin-true no-host 764 1 2 2.6 7.9 156,776 15,411 2,898 8,755 77,337 70,684 3,008 3 50,000 0
configure.cpython no-host 14,591 1 67 15.2 29.0 1,785,026 49,166 25,885 91,516 886,574 806,936 25,736 62 50,000 0
configure.util-linux no-host 13,577 3 66 21.6 30.3 2,709,227 56,191 66,554 114,379 1,295,038 1,299,810 32,241 42 133,108 1
configure.ocaml no-host 3,697 1 2 2.6 22.5 149,879 12,461 6,504 7,751 70,874 71,254 1,695 3 50,000 0
configure.tcc no-host 47 0 0 1.3 19.6 30,580 30,580 0 1,546 14,636 14,398 271 0 50,000 0
configure.yash no-host 1,756 1 2 2.6 26.3 139,886 8,036 6,691 6,401 68,339 65,146 2,095 3 50,000 0
abuild-print-help no-host 20 1 2 2.8 8.4 116,339 64,417 42,622 4,547 54,461 57,331 277 3 85,244 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 18 9 9 4.5 bash
15 no-host abuild-print-help 9 5 4 3.3 dash
23 no-host abuild-print-help 470 413 60 23.0 osh-cpython
31 no-host abuild-print-help 21 11 11 9.2 osh-native
39 no-host abuild-print-help 20 12 8 8.4 osh-native-souffle
1 no-host bin-true 632 299 367 3.0 bash
9 no-host bin-true 514 280 244 1.4 dash
17 no-host bin-true 2,269 799 1,726 18.5 osh-cpython
25 no-host bin-true 758 305 469 8.3 osh-native
33 no-host bin-true 764 311 471 7.9 osh-native-souffle
2 no-host configure.cpython 14,288 6,563 8,123 29.2 bash
10 no-host configure.cpython 14,364 6,551 8,237 29.2 dash
18 no-host configure.cpython 28,003 14,174 15,684 29.2 osh-cpython
26 no-host configure.cpython 14,453 6,388 8,349 29.1 osh-native
34 no-host configure.cpython 14,591 6,431 8,282 29.0 osh-native-souffle
4 no-host configure.ocaml 3,718 1,298 1,433 22.6 bash
12 no-host configure.ocaml 3,616 1,262 1,376 22.5 dash
20 no-host configure.ocaml 4,527 1,811 1,925 22.6 osh-cpython
28 no-host configure.ocaml 3,701 1,293 1,448 22.5 osh-native
36 no-host configure.ocaml 3,697 1,297 1,439 22.5 osh-native-souffle
5 no-host configure.tcc 45 23 23 19.6 bash
13 no-host configure.tcc 40 22 18 19.7 dash
21 no-host configure.tcc 201 140 64 19.8 osh-cpython
29 no-host configure.tcc 47 26 22 19.7 osh-native
37 no-host configure.tcc 47 24 23 19.6 osh-native-souffle
3 no-host configure.util-linux 13,286 5,833 7,938 30.3 bash
11 no-host configure.util-linux 14,255 5,885 8,854 30.3 dash
19 no-host configure.util-linux 30,591 17,645 14,012 30.2 osh-cpython
27 no-host configure.util-linux 13,642 5,633 8,239 30.3 osh-native
35 no-host configure.util-linux 13,577 5,505 8,299 30.3 osh-native-souffle
6 no-host configure.yash 1,746 902 853 26.2 bash
14 no-host configure.yash 1,696 905 795 26.3 dash
22 no-host configure.yash 2,391 1,329 1,106 26.2 osh-cpython
30 no-host configure.yash 1,753 864 895 26.3 osh-native
38 no-host configure.yash 1,756 868 894 26.3 osh-native-souffle
0 no-host hello-world 1 1 1 2.9 bash
8 no-host hello-world 1 0 1 1.4 dash
16 no-host hello-world 57 42 15 18.4 osh-cpython
24 no-host hello-world 3 0 3 6.3 osh-native
32 no-host hello-world 3 1 2 6.0 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,552 0 0 0 16 0 41 1 bash
15 no-host abuild-print-help 1,111 0 0 0 8 0 39 0 dash
23 no-host abuild-print-help 24,505 0 0 0 8 0 45 8 osh-cpython
31 no-host abuild-print-help 3,162 0 0 0 16 0 39 0 osh-native
39 no-host abuild-print-help 3,071 0 0 0 16 0 39 2 osh-native-souffle
1 no-host bin-true 105,382 0 0 0 8 0 1,984 195 bash
9 no-host bin-true 65,463 0 0 0 8 0 1,980 191 dash
17 no-host bin-true 799,444 0 0 0 8 0 1,979 283 osh-cpython
25 no-host bin-true 126,189 0 0 0 16 0 1,988 85 osh-native
33 no-host bin-true 124,115 0 0 0 16 0 1,994 89 osh-native-souffle
2 no-host configure.cpython 1,993,697 0 0 0 32,928 0 14,251 1,427 bash
10 no-host configure.cpython 2,026,658 0 0 0 32,992 0 14,383 1,294 dash
18 no-host configure.cpython 6,124,711 1 0 0 27,512 0 13,051 1,195 osh-cpython
26 no-host configure.cpython 2,016,435 1 0 0 27,520 0 13,095 1,201 osh-native
34 no-host configure.cpython 2,008,795 0 0 0 27,520 0 13,034 1,118 osh-native-souffle
4 no-host configure.ocaml 373,554 0 0 0 5,120 0 1,956 198 bash
12 no-host configure.ocaml 353,594 0 0 0 5,112 0 1,942 172 dash
20 no-host configure.ocaml 617,891 0 0 0 5,112 0 1,854 199 osh-cpython
28 no-host configure.ocaml 376,865 0 0 0 5,120 0 1,924 217 osh-native
36 no-host configure.ocaml 376,449 0 0 0 5,120 0 1,925 229 osh-native-souffle
5 no-host configure.tcc 6,075 0 0 0 104 0 49 2 bash
13 no-host configure.tcc 5,004 0 0 0 96 0 47 1 dash
21 no-host configure.tcc 23,934 0 0 0 96 0 53 4 osh-cpython
29 no-host configure.tcc 6,487 0 0 0 104 0 47 2 osh-native
37 no-host configure.tcc 6,462 0 0 0 104 0 47 15 osh-native-souffle
3 no-host configure.util-linux 1,918,209 0 0 0 42,400 0 18,496 1,334 bash
11 no-host configure.util-linux 2,068,221 0 0 0 42,584 0 17,341 1,595 dash
19 no-host configure.util-linux 5,351,258 8 0 0 39,112 0 17,037 1,375 osh-cpython
27 no-host configure.util-linux 1,826,666 0 0 0 39,128 0 17,009 1,244 osh-native
35 no-host configure.util-linux 1,817,340 0 0 0 39,136 0 16,943 1,124 osh-native-souffle
6 no-host configure.yash 234,092 0 0 0 4,120 0 871 96 bash
14 no-host configure.yash 222,191 0 0 0 3,720 0 872 85 dash
22 no-host configure.yash 368,854 0 0 0 3,728 0 854 98 osh-cpython
30 no-host configure.yash 235,810 0 0 0 3,736 0 849 98 osh-native
38 no-host configure.yash 235,027 0 0 0 3,736 0 849 96 osh-native-souffle
0 no-host hello-world 151 0 0 0 8 0 1 0 bash
8 no-host hello-world 76 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,980 0 0 0 8 0 7 1 osh-cpython
24 no-host hello-world 345 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 342 0 0 0 16 0 1 0 osh-native-souffle

Shell and Host

shell label shell id
bash bash-a010df25
dash dash-0ebcf347
osh-cpython osh-9b360f29
osh-native osh-b6cb58e8
osh-native-souffle osh-6cb07ffe
host label host id
no-host no-host-20fc69f9