OSH Runtime Performance

Source code: benchmarks/osh-runtime.sh

Raw files

Elapsed Time by Shell (milliseconds)

Some benchmarks call many external tools, while some exercise the shell interpreter itself.

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 27 14 529 34 32 19.73 1.26
bin-true no-host 1,070 895 3,798 1,366 1,175 3.55 1.28
configure.cpython no-host 19,680 20,170 39,113 20,397 20,631 1.99 1.04
configure.ocaml no-host 4,619 4,557 5,738 4,580 4,580 1.24 0.99
configure.tcc no-host 63 56 237 52 69 3.77 0.83
configure.util-linux no-host 19,682 20,266 41,868 20,243 19,490 2.13 1.03
configure.yash no-host 2,344 2,264 2,984 2,285 2,192 1.27 0.97
hello-world no-host 3 1 62 3 3 20.07 0.99

Minor Page Faults

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 2,546 1,133 23,467 3,202 3,122 9.22 1.26
bin-true no-host 106,395 66,479 759,924 128,140 123,178 7.14 1.20
configure.cpython no-host 2,222,843 2,239,045 6,197,004 2,230,831 2,206,263 2.79 1.00
configure.ocaml no-host 410,949 391,167 643,799 413,135 412,778 1.57 1.01
configure.tcc no-host 6,540 5,469 23,877 6,948 6,886 3.65 1.06
configure.util-linux no-host 2,094,056 2,264,548 5,351,538 2,000,176 1,993,103 2.56 0.96
configure.yash no-host 256,551 245,029 386,424 257,092 258,141 1.51 1.00
hello-world no-host 152 82 3,953 342 334 26.01 2.25

Memory Usage (Max Resident Set Size in MB)

Memory usage is measured in MB (powers of 10), not MiB (powers of 2).

workload host name bash dash osh-cpython osh-native osh-native-souffle py bash ratio native bash ratio
abuild-print-help no-host 4.5 3.4 22.7 9.2 8.8 5.10 2.06
bin-true no-host 2.9 1.3 18.3 8.1 7.6 6.34 2.82
configure.cpython no-host 27.3 27.5 27.4 27.4 27.5 1.00 1.00
configure.ocaml no-host 20.6 20.6 20.6 20.6 20.6 1.00 1.00
configure.tcc no-host 19.3 19.4 19.3 19.4 19.1 1.00 1.01
configure.util-linux no-host 30.3 30.1 30.3 30.3 30.0 1.00 1.00
configure.yash no-host 24.5 24.2 24.6 24.4 24.2 1.00 0.99
hello-world no-host 2.9 1.4 18.3 6.3 5.8 6.33 2.18

GC Stats

workload host name elapsed ms max gc millis total gc millis allocated MB max rss MB num allocated num live max survived num in heap num in pool 1 num in pool 2 num gc points num gc done gc threshold num growths
hello-world no-host 3 0 0 0.2 6.3 3,791 3,791 0 694 1,863 1,234 2 0 50,000 0
bin-true no-host 1,366 1 2 2.6 8.1 153,755 12,331 2,896 8,743 76,327 68,685 3,008 3 50,000 0
configure.cpython no-host 20,397 6 96 14.9 27.4 1,771,756 49,614 28,521 85,415 878,120 808,221 25,733 60 50,000 0
configure.util-linux no-host 20,243 8 81 21.2 30.3 2,702,522 53,775 65,700 108,691 1,290,645 1,303,186 32,239 41 131,400 1
configure.ocaml no-host 4,580 1 2 2.6 20.6 150,089 11,121 5,216 7,508 70,974 71,607 1,695 3 50,000 0
configure.tcc no-host 52 0 0 1.3 19.4 30,595 30,595 0 1,518 14,646 14,431 271 0 50,000 0
configure.yash no-host 2,285 1 2 2.5 24.4 139,421 8,033 6,695 5,989 67,947 65,485 2,095 3 50,000 0
abuild-print-help no-host 34 1 2 2.8 9.2 116,301 64,412 42,622 4,532 54,441 57,328 277 3 85,244 1
hello-world no-host 3 0 0 0.2 5.8 3,790 3,790 0 695 1,863 1,232 2 0 50,000 0
bin-true no-host 1,175 1 2 2.6 7.6 153,754 12,120 2,834 8,744 76,327 68,683 3,008 3 50,000 0
configure.cpython no-host 20,631 4 81 14.9 27.5 1,771,755 49,072 28,459 85,416 878,120 808,219 25,733 60 50,000 0
configure.util-linux no-host 19,490 4 76 21.2 30.0 2,702,522 53,702 65,638 108,693 1,290,645 1,303,184 32,239 41 131,276 1
configure.ocaml no-host 4,580 1 2 2.6 20.6 150,089 11,059 5,154 7,510 70,974 71,605 1,695 3 50,000 0
configure.tcc no-host 69 0 0 1.3 19.1 30,595 30,595 0 1,520 14,646 14,429 271 0 50,000 0
configure.yash no-host 2,192 1 2 2.5 24.2 139,421 7,971 6,633 5,991 67,947 65,483 2,095 3 50,000 0
abuild-print-help no-host 32 1 2 2.8 8.8 116,300 64,350 42,560 4,533 54,441 57,326 277 3 85,120 1

rusage Details

task id host name workload elapsed ms user ms sys ms max rss MB shell label
7 no-host abuild-print-help 27 18 10 4.5 bash
15 no-host abuild-print-help 14 6 7 3.4 dash
23 no-host abuild-print-help 529 461 70 22.7 osh-cpython
31 no-host abuild-print-help 34 31 4 9.2 osh-native
39 no-host abuild-print-help 32 21 12 8.8 osh-native-souffle
1 no-host bin-true 1,070 658 406 2.9 bash
9 no-host bin-true 895 511 335 1.3 dash
17 no-host bin-true 3,798 2,128 1,858 18.3 osh-cpython
25 no-host bin-true 1,366 680 632 8.1 osh-native
33 no-host bin-true 1,175 675 448 7.6 osh-native-souffle
2 no-host configure.cpython 19,680 10,697 9,083 27.3 bash
10 no-host configure.cpython 20,170 10,836 9,506 27.5 dash
18 no-host configure.cpython 39,113 21,793 19,437 27.4 osh-cpython
26 no-host configure.cpython 20,397 10,707 9,656 27.4 osh-native
34 no-host configure.cpython 20,631 10,680 9,734 27.5 osh-native-souffle
4 no-host configure.ocaml 4,619 1,847 1,775 20.6 bash
12 no-host configure.ocaml 4,557 1,812 1,733 20.6 dash
20 no-host configure.ocaml 5,738 2,724 2,248 20.6 osh-cpython
28 no-host configure.ocaml 4,580 1,891 1,716 20.6 osh-native
36 no-host configure.ocaml 4,580 1,836 1,743 20.6 osh-native-souffle
5 no-host configure.tcc 63 32 31 19.3 bash
13 no-host configure.tcc 56 28 28 19.4 dash
21 no-host configure.tcc 237 152 89 19.3 osh-cpython
29 no-host configure.tcc 52 36 17 19.4 osh-native
37 no-host configure.tcc 69 34 36 19.1 osh-native-souffle
3 no-host configure.util-linux 19,682 10,579 9,208 30.3 bash
11 no-host configure.util-linux 20,266 10,584 9,840 30.1 dash
19 no-host configure.util-linux 41,868 25,429 17,307 30.3 osh-cpython
27 no-host configure.util-linux 20,243 10,285 9,591 30.3 osh-native
35 no-host configure.util-linux 19,490 10,120 9,117 30.0 osh-native-souffle
6 no-host configure.yash 2,344 1,211 1,118 24.5 bash
14 no-host configure.yash 2,264 1,157 1,063 24.2 dash
22 no-host configure.yash 2,984 1,703 1,308 24.6 osh-cpython
30 no-host configure.yash 2,285 1,128 1,129 24.4 osh-native
38 no-host configure.yash 2,192 1,185 995 24.2 osh-native-souffle
0 no-host hello-world 3 3 0 2.9 bash
8 no-host hello-world 1 1 0 1.4 dash
16 no-host hello-world 62 33 28 18.3 osh-cpython
24 no-host hello-world 3 0 3 6.3 osh-native
32 no-host hello-world 3 3 0 5.8 osh-native-souffle

More Details

task id host name workload minor faults major faults swaps in block out block signals voluntary ctx involuntary ctx shell label
7 no-host abuild-print-help 2,546 0 0 0 16 0 39 1 bash
15 no-host abuild-print-help 1,133 0 0 0 8 0 37 0 dash
23 no-host abuild-print-help 23,467 0 0 0 8 0 44 11 osh-cpython
31 no-host abuild-print-help 3,202 0 0 0 16 0 37 4 osh-native
39 no-host abuild-print-help 3,122 0 0 0 16 0 39 1 osh-native-souffle
1 no-host bin-true 106,395 0 0 0 8 0 1,989 174 bash
9 no-host bin-true 66,479 0 0 0 8 0 1,994 65 dash
17 no-host bin-true 759,924 0 0 0 8 0 1,990 207 osh-cpython
25 no-host bin-true 128,140 0 0 0 16 0 1,989 101 osh-native
33 no-host bin-true 123,178 0 0 0 16 0 1,992 115 osh-native-souffle
2 no-host configure.cpython 2,222,843 1 0 0 32,928 0 14,477 1,141 bash
10 no-host configure.cpython 2,239,045 2 0 0 32,992 0 14,442 1,312 dash
18 no-host configure.cpython 6,197,004 1 0 0 27,536 0 13,358 1,432 osh-cpython
26 no-host configure.cpython 2,230,831 0 0 0 27,520 0 13,343 1,097 osh-native
34 no-host configure.cpython 2,206,263 1 0 0 27,520 0 13,340 1,119 osh-native-souffle
4 no-host configure.ocaml 410,949 0 0 0 5,120 0 1,946 165 bash
12 no-host configure.ocaml 391,167 0 0 0 5,112 0 1,941 157 dash
20 no-host configure.ocaml 643,799 0 0 0 5,112 0 1,894 193 osh-cpython
28 no-host configure.ocaml 413,135 0 0 0 5,120 0 1,936 180 osh-native
36 no-host configure.ocaml 412,778 0 0 0 5,120 0 1,939 172 osh-native-souffle
5 no-host configure.tcc 6,540 0 0 0 104 0 49 4 bash
13 no-host configure.tcc 5,469 0 0 0 96 0 47 4 dash
21 no-host configure.tcc 23,877 0 0 0 96 0 52 8 osh-cpython
29 no-host configure.tcc 6,948 0 0 0 104 0 47 3 osh-native
37 no-host configure.tcc 6,886 0 0 0 104 0 46 2 osh-native-souffle
3 no-host configure.util-linux 2,094,056 1 0 0 42,408 0 18,323 1,237 bash
11 no-host configure.util-linux 2,264,548 4 0 0 42,584 0 17,006 1,249 dash
19 no-host configure.util-linux 5,351,538 9 0 0 39,160 0 16,942 1,465 osh-cpython
27 no-host configure.util-linux 2,000,176 8 0 0 39,128 0 16,908 1,375 osh-native
35 no-host configure.util-linux 1,993,103 11 0 0 39,128 0 16,830 1,245 osh-native-souffle
6 no-host configure.yash 256,551 0 0 0 4,120 0 869 87 bash
14 no-host configure.yash 245,029 0 0 0 3,720 0 867 91 dash
22 no-host configure.yash 386,424 1 0 0 3,728 0 852 113 osh-cpython
30 no-host configure.yash 257,092 0 0 0 3,736 0 850 81 osh-native
38 no-host configure.yash 258,141 0 0 0 3,736 0 844 95 osh-native-souffle
0 no-host hello-world 152 0 0 0 8 0 1 0 bash
8 no-host hello-world 82 0 0 0 8 0 1 0 dash
16 no-host hello-world 3,953 0 0 0 8 0 7 5 osh-cpython
24 no-host hello-world 342 0 0 0 16 0 1 0 osh-native
32 no-host hello-world 334 0 0 0 16 0 1 0 osh-native-souffle

Shell and Host

shell label shell id
bash bash-9b81c677
dash dash-0ebcf347
osh-cpython osh-9572fdfb
osh-native osh-8ef7aad1
osh-native-souffle osh-6997f550
host label host id
no-host no-host-556df3ee