OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 16 9 548 22 21 34.92 1.39
bin-true no-host 610 435 2,426 796 798 3.97 1.30
configure.cpython no-host 13,720 13,794 29,253 14,749 14,609 2.13 1.07
configure.ocaml no-host 3,650 3,595 4,612 3,705 3,704 1.26 1.01
configure.tcc no-host 46 41 223 49 49 4.90 1.08
configure.util-linux no-host 12,574 13,211 32,083 13,528 13,634 2.55 1.08
configure.yash no-host 1,694 1,631 2,414 1,709 1,705 1.42 1.01
hello-world no-host 1 1 60 3 3 50.70 2.44

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,423 1,160 24,735 3,435 3,382 10.21 1.42
bin-true no-host 99,945 49,415 837,752 136,249 135,180 8.38 1.36
configure.cpython no-host 1,630,427 1,646,117 5,976,777 1,812,341 1,793,766 3.67 1.11
configure.ocaml no-host 298,621 280,589 558,615 316,706 316,451 1.87 1.06
configure.tcc no-host 5,191 4,222 23,873 5,937 5,936 4.60 1.14
configure.util-linux no-host 1,540,414 1,700,479 4,987,875 1,720,000 1,715,832 3.24 1.12
configure.yash no-host 173,891 159,560 315,967 179,643 179,226 1.82 1.03
hello-world no-host 152 86 3,868 386 376 25.45 2.54

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.6 3.9 22.7 9.8 9.3 4.94 2.14
bin-true no-host 3.3 1.4 18.2 8.8 8.3 5.55 2.68
configure.cpython no-host 34.5 34.7 34.7 34.5 34.5 1.00 1.00
configure.ocaml no-host 28.7 28.7 28.7 28.7 28.6 1.00 1.00
configure.tcc no-host 26.1 26.3 26.2 26.2 26.2 1.00 1.00
configure.util-linux no-host 39.5 39.7 39.5 39.6 39.5 1.00 1.00
configure.yash no-host 33.5 33.3 33.3 33.4 33.5 0.99 1.00
hello-world no-host 3.1 1.6 18.1 6.9 6.4 5.74 2.21

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.3 6.9 3,831 3,831 0 704 1,881 1,246 2 0 50,000 0
bin-true no-host 796 1 2 2.6 8.8 156,803 15,433 2,920 8,761 77,348 70,694 3,008 3 50,000 0
configure.cpython no-host 14,749 1 65 15.2 34.5 1,782,994 46,106 25,369 91,622 885,112 806,260 25,789 62 50,000 0
configure.util-linux no-host 13,528 3 60 21.4 39.6 2,707,127 126,359 66,591 113,463 1,294,568 1,299,096 32,539 41 133,182 1
configure.ocaml no-host 3,705 1 2 2.7 28.7 149,878 12,481 6,524 7,819 70,840 71,219 1,693 3 50,000 0
configure.tcc no-host 49 0 0 1.3 26.2 30,633 30,633 0 1,582 14,646 14,405 271 0 50,000 0
configure.yash no-host 1,709 1 2 2.6 33.4 140,022 8,085 6,719 6,526 68,348 65,148 2,095 3 50,000 0
abuild-print-help no-host 22 1 2 2.8 9.8 116,373 64,443 42,644 4,562 54,472 57,339 277 3 85,288 1
hello-world no-host 3 0 0 0.3 6.4 3,830 3,830 0 705 1,881 1,244 2 0 50,000 0
bin-true no-host 798 1 2 2.6 8.3 156,802 15,428 2,915 8,762 77,348 70,692 3,008 3 50,000 0
configure.cpython no-host 14,609 2 67 15.2 34.5 1,782,993 46,092 25,355 91,623 885,112 806,258 25,789 62 50,000 0
configure.util-linux no-host 13,634 3 61 21.4 39.5 2,707,127 126,352 66,584 113,465 1,294,568 1,299,094 32,539 41 133,168 1
configure.ocaml no-host 3,704 1 2 2.7 28.6 149,878 12,475 6,518 7,821 70,840 71,217 1,693 3 50,000 0
configure.tcc no-host 49 0 0 1.3 26.2 30,633 30,633 0 1,584 14,646 14,403 271 0 50,000 0
configure.yash no-host 1,705 1 2 2.6 33.5 140,022 8,074 6,708 6,528 68,348 65,146 2,095 3 50,000 0
abuild-print-help no-host 21 1 2 2.8 9.3 116,372 64,438 42,639 4,563 54,472 57,337 277 3 85,278 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 16 10 7 4.6 bash
15 no-host abuild-print-help 9 5 5 3.9 dash
23 no-host abuild-print-help 548 493 59 22.7 osh-cpython
31 no-host abuild-print-help 22 9 13 9.8 osh-native
39 no-host abuild-print-help 21 10 12 9.3 osh-native-souffle
1 no-host bin-true 610 312 334 3.3 bash
9 no-host bin-true 435 274 165 1.4 dash
17 no-host bin-true 2,426 889 1,800 18.2 osh-cpython
25 no-host bin-true 796 318 494 8.8 osh-native
33 no-host bin-true 798 302 511 8.3 osh-native-souffle
2 no-host configure.cpython 13,720 7,506 6,576 34.5 bash
10 no-host configure.cpython 13,794 7,535 6,642 34.7 dash
18 no-host configure.cpython 29,253 16,076 15,118 34.7 osh-cpython
26 no-host configure.cpython 14,749 7,355 7,678 34.5 osh-native
34 no-host configure.cpython 14,609 7,326 7,432 34.5 osh-native-souffle
4 no-host configure.ocaml 3,650 1,458 1,226 28.7 bash
12 no-host configure.ocaml 3,595 1,482 1,136 28.7 dash
20 no-host configure.ocaml 4,612 2,065 1,769 28.7 osh-cpython
28 no-host configure.ocaml 3,705 1,470 1,276 28.7 osh-native
36 no-host configure.ocaml 3,704 1,478 1,263 28.6 osh-native-souffle
5 no-host configure.tcc 46 29 18 26.1 bash
13 no-host configure.tcc 41 21 20 26.3 dash
21 no-host configure.tcc 223 170 57 26.2 osh-cpython
29 no-host configure.tcc 49 20 30 26.2 osh-native
37 no-host configure.tcc 49 26 23 26.2 osh-native-souffle
3 no-host configure.util-linux 12,574 6,509 6,517 39.5 bash
11 no-host configure.util-linux 13,211 6,344 7,342 39.7 dash
19 no-host configure.util-linux 32,083 20,175 12,967 39.5 osh-cpython
27 no-host configure.util-linux 13,528 6,296 7,456 39.6 osh-native
35 no-host configure.util-linux 13,634 6,247 7,467 39.5 osh-native-souffle
6 no-host configure.yash 1,694 936 766 33.5 bash
14 no-host configure.yash 1,631 936 697 33.3 dash
22 no-host configure.yash 2,414 1,436 1,017 33.3 osh-cpython
30 no-host configure.yash 1,709 919 793 33.4 osh-native
38 no-host configure.yash 1,705 917 792 33.5 osh-native-souffle
0 no-host hello-world 1 0 1 3.1 bash
8 no-host hello-world 1 0 1 1.6 dash
16 no-host hello-world 60 37 23 18.1 osh-cpython
24 no-host hello-world 3 2 1 6.9 osh-native
32 no-host hello-world 3 1 2 6.4 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,423 0 0 0 8 0 41 0 bash
15 no-host abuild-print-help 1,160 0 0 0 8 0 40 0 dash
23 no-host abuild-print-help 24,735 0 0 0 8 0 45 5 osh-cpython
31 no-host abuild-print-help 3,435 0 0 0 16 0 38 2 osh-native
39 no-host abuild-print-help 3,382 0 0 0 16 0 39 0 osh-native-souffle
1 no-host bin-true 99,945 0 0 0 8 0 1,990 65 bash
9 no-host bin-true 49,415 0 0 0 8 0 2,966 68 dash
17 no-host bin-true 837,752 0 0 0 8 0 1,997 142 osh-cpython
25 no-host bin-true 136,249 0 0 0 16 0 1,979 95 osh-native
33 no-host bin-true 135,180 0 0 0 16 0 1,964 127 osh-native-souffle
2 no-host configure.cpython 1,630,427 0 0 0 25,272 0 14,049 1,035 bash
10 no-host configure.cpython 1,646,117 1 0 0 25,272 0 14,092 1,067 dash
18 no-host configure.cpython 5,976,777 0 0 0 25,272 0 12,870 964 osh-cpython
26 no-host configure.cpython 1,812,341 0 0 0 25,280 0 12,859 1,169 osh-native
34 no-host configure.cpython 1,793,766 0 0 0 25,280 0 12,862 1,075 osh-native-souffle
4 no-host configure.ocaml 298,621 0 0 0 4,432 0 2,014 225 bash
12 no-host configure.ocaml 280,589 0 0 0 4,432 0 2,106 184 dash
20 no-host configure.ocaml 558,615 0 0 0 4,432 0 1,914 206 osh-cpython
28 no-host configure.ocaml 316,706 0 0 0 4,440 0 1,992 202 osh-native
36 no-host configure.ocaml 316,451 0 0 0 4,440 0 1,991 206 osh-native-souffle
5 no-host configure.tcc 5,191 0 0 0 80 0 49 2 bash
13 no-host configure.tcc 4,222 0 0 0 88 0 52 1 dash
21 no-host configure.tcc 23,873 0 0 0 88 0 53 6 osh-cpython
29 no-host configure.tcc 5,937 0 0 0 96 0 47 4 osh-native
37 no-host configure.tcc 5,936 0 0 0 96 0 47 2 osh-native-souffle
3 no-host configure.util-linux 1,540,414 0 0 0 35,944 0 18,973 1,085 bash
11 no-host configure.util-linux 1,700,479 1 0 0 35,936 0 17,037 1,192 dash
19 no-host configure.util-linux 4,987,875 1 0 0 35,960 0 17,523 845 osh-cpython
27 no-host configure.util-linux 1,720,000 0 0 0 35,944 0 17,514 1,153 osh-native
35 no-host configure.util-linux 1,715,832 0 0 0 35,944 0 17,419 1,053 osh-native-souffle
6 no-host configure.yash 173,891 0 0 0 3,616 0 877 109 bash
14 no-host configure.yash 159,560 0 0 0 3,624 0 999 110 dash
22 no-host configure.yash 315,967 0 0 0 3,616 0 859 100 osh-cpython
30 no-host configure.yash 179,643 0 0 0 3,624 0 859 101 osh-native
38 no-host configure.yash 179,226 0 0 0 3,624 0 856 109 osh-native-souffle
0 no-host hello-world 152 0 0 0 8 0 1 0 bash
8 no-host hello-world 86 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,868 0 0 0 8 0 7 2 osh-cpython
24 no-host hello-world 386 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 376 0 0 0 16 0 1 0 osh-native-souffle

Shell and Host

shell label shell id
bash bash-21f91a38
dash dash-cea41b3e
osh-cpython osh-700705a5
osh-native osh-dffcbb86
osh-native-souffle osh-adfa5fb7
host label host id
no-host no-host-31404de8