Overview of Oils Code

category total lines num files
OSH (and common libraries) 48,627 86
YSH 11,524 27
Data Languages 3,317 12
Tools 1,525 5
OSH stdlib 560 9
YSH stdlib 2,101 15
Code Borrowed from Python's stdlib 470 6
Spec Tests 71,348 222
Gold Tests 2,782 27
Python Unit Tests 14,192 83
Other Shell Tests 19,619 73
Test Automation 1,658 11
mycpp Python Runtime 989 4
mycpp Translator 6,461 12
mycpp Test Data 4,893 57
Zephyr ASDL 3,193 11
pgen2 Parser Generator 1,854 8
Other Code Generators 3,112 11
Yaks 507 4
Hand-written C++ Code 3,765 30
Garbage-Collected Runtime 5,485 26
Unit tests in C++ 7,652 26
Incremental C++ Build 4,047 20
Build Automation 7,430 31
Build of oils-ref Tarball 1,252 11
Release Automation 2,803 8
Soil: Multi-cloud CI with containers 4,070 23
Benchmarks 10,026 49
Metrics 1,669 8
Generated Python Code 21,365 26
Doc Tools 5,583 16
Web 1,155 6

OSH (and common libraries)

This is the input to the translators, written in statically-typed Python. Note that bash is at least 140K lines of code, and OSH implements a large part of bash and more.

bin/oils_for_unix.py                            263
builtin/assign_osh.py                           643
builtin/bracket_osh.py                          291
builtin/completion_osh.py                       513
builtin/dirs_osh.py                             350
builtin/io_osh.py                               142
builtin/misc_osh.py                             127
builtin/printf_osh.py                           600
builtin/process_osh.py                          819
builtin/pure_osh.py                             597
builtin/read_osh.py                             530
builtin/readline_osh.py                         471
builtin/trap_osh.py                             411
core/alloc.py                                   211
core/bash_impl.py                               629
core/comp_ui.py                                 586
core/completion.py                            1,529
core/dev.py                                     764
core/error.py                                   382
core/executor.py                              1,063
core/main_loop.py                               487
core/num.py                                      24
core/optview.py                                  57
core/process.py                               2,242
core/pyos.py                                    326
core/pyutil.py                                  222
core/runtime.asdl                               196
core/sh_init.py                                 324
core/shell.py                                 1,308
core/state.py                                 3,054
core/util.py                                    170
core/value.asdl                                 236
core/vm.py                                      542
display/ansi.py                                  14
display/pp_hnode.py                             309
display/pp_value.py                             350
display/pretty.asdl                              44
display/pretty.py                               332
display/ui.py                                   615
frontend/args.py                                708
frontend/builtin_def.py                         171
frontend/consts.py                              409
frontend/flag_def.py                            602
frontend/flag_spec.py                           356
frontend/flag_util.py                            69
frontend/id_kind_def.py                         817
frontend/lexer.py                               515
frontend/lexer_def.py                         1,132
frontend/location.py                            616
frontend/match.py                               331
frontend/option_def.py                          441
frontend/parse_lib.py                           409
frontend/py_readline.py                         178
frontend/reader.py                              280
frontend/signal_def.py                           92
frontend/syntax.asdl                            711
frontend/syntax_abbrev.py                       147
frontend/typed_args.py                          675
frontend/types.asdl                              61
osh/arith_parse.py                              197
osh/bool_parse.py                               302
osh/bool_stat.py                                135
osh/braces.py                                   539
osh/cmd_eval.py                               2,630
osh/cmd_parse.py                              2,889
osh/glob_.py                                    565
osh/history.py                                  193
osh/prompt.py                                   368
osh/sh_expr_eval.py                           1,309
osh/split.py                                    449
osh/string_ops.py                               554
osh/tdop.py                                     342
osh/word_.py                                    865
osh/word_compile.py                             310
osh/word_eval.py                              2,667
osh/word_parse.py                             2,374
pyext/fanos.c                                   119
pyext/fastfunc.c                                133
pyext/fastfunc.pyi                               11
pyext/fastlex.c                                 422
pyext/fastlex.pyi                                23
pyext/libc.c                                    441
pyext/libc.pyi                                   26
pyext/line_input.pyi                             58
pyext/posix_.pyi                                213

48,627 lines in 86 files  

YSH

Expression grammar, parser, evaluator, etc.

builtin/completion_ysh.py                        69
builtin/error_ysh.py                            307
builtin/func_eggex.py                           186
builtin/func_hay.py                             157
builtin/func_misc.py                            611
builtin/func_reflect.py                         332
builtin/hay_ysh.py                              432
builtin/io_ysh.py                               291
builtin/json_ysh.py                             140
builtin/meta_oils.py                          1,020
builtin/method_dict.py                           95
builtin/method_io.py                            318
builtin/method_list.py                          205
builtin/method_other.py                         120
builtin/method_str.py                           622
builtin/method_type.py                          141
builtin/module_ysh.py                           141
builtin/private_ysh.py                          220
builtin/pure_ysh.py                             232
ysh/expr_eval.py                              1,627
ysh/expr_parse.py                               389
ysh/expr_to_ast.py                            1,740
ysh/func_proc.py                                606
ysh/grammar.pgen2                               553
ysh/regex_translate.py                          410
ysh/val_ops.py                                  560

11,524 lines in 27 files  

Data Languages

JSON, J8 Notation, ...

data_lang/htm8.asdl                              86
data_lang/nil8.asdl                              31
data_lang/htm8.py                               754
data_lang/htm8_util.py                          294
data_lang/j8.py                               1,335
data_lang/j8_lite.py                             57
data_lang/pyj8.py                                27
data_lang/j8.h                                  314
data_lang/j8_libc.c                             221
data_lang/j8_libc.h                              28
data_lang/utf8.h                                170

3,317 lines in 12 files  

Tools

tools/deps.py                                   148
tools/fmt.py                                     25
tools/readlink.py                                31
tools/ysh_ify.py                              1,321

1,525 lines in 5 files  

OSH stdlib

stdlib/osh/bash-strict.sh                         8
stdlib/osh/byo-server-test.sh                    73
stdlib/osh/byo-server.sh                        138
stdlib/osh/no-quotes-test.sh                     82
stdlib/osh/no-quotes.sh                         137
stdlib/osh/task-five.sh                          62
stdlib/osh/two-test.sh                           33
stdlib/osh/two.sh                                27

560 lines in 9 files  

YSH stdlib

stdlib/ysh/args-test.ysh                        390
stdlib/ysh/args.ysh                             291
stdlib/ysh/def-test.ysh                          43
stdlib/ysh/def.ysh                                7
stdlib/ysh/list-test.ysh                         77
stdlib/ysh/list.ysh                              53
stdlib/ysh/math-test.ysh                         74
stdlib/ysh/math.ysh                              98
stdlib/ysh/quote-test.ysh                       280
stdlib/ysh/quote.ysh                            203
stdlib/ysh/stream.ysh                           310
stdlib/ysh/table.ysh                            150
stdlib/ysh/yblocks-test.ysh                      72
stdlib/ysh/yblocks.ysh                           53

2,101 lines in 15 files  

Code Borrowed from Python's stdlib

pylib/cgi.py                                     19
pylib/collections_.py                           214
pylib/os_path.py                                169
pylib/path_stat.py                               37
pylib/pylocale.py                                31

470 lines in 6 files  

Spec Tests

A comprehensive test suite that compares OSH against other shells. If OSH passes these tests in BOTH Python and C++, it means that the translation works.

spec/alias.test.sh                              573
spec/append.test.sh                             324
spec/arg-parse.test.sh                           50
spec/arith-context.test.sh                      245
spec/arith-dynamic.test.sh                       95
spec/arith.test.sh                            1,044
spec/array-assign.test.sh                       376
spec/array-assoc.test.sh                        790
spec/array-basic.test.sh                         45
spec/array-compat.test.sh                       188
spec/array-literal.test.sh                      336
spec/array-sparse.test.sh                     1,228
spec/array.test.sh                            1,027
spec/assign-deferred.test.sh                    115
spec/assign-dialects.test.sh                    149
spec/assign-extended.test.sh                    986
spec/assign.test.sh                             768
spec/background.test.sh                         398
spec/ble-features.test.sh                       387
spec/ble-idioms.test.sh                         574
spec/ble-unset.test.sh                          243
spec/blog-other1.test.sh                         66
spec/blog1.test.sh                               96
spec/blog2.test.sh                               60
spec/bool-parse.test.sh                         176
spec/brace-expansion.test.sh                    501
spec/bugs.test.sh                               451
spec/builtin-bash.test.sh                       215
spec/builtin-bind.test.sh                       158
spec/builtin-bracket.test.sh                    770
spec/builtin-cd.test.sh                         518
spec/builtin-completion.test.sh                 641
spec/builtin-dirs.test.sh                       261
spec/builtin-echo.test.sh                       320
spec/builtin-eval-source.test.sh                379
spec/builtin-fc.test.sh                         255
spec/builtin-getopts.test.sh                    435
spec/builtin-history.test.sh                    403
spec/builtin-kill.test.sh                       327
spec/builtin-meta-assign.test.sh                376
spec/builtin-meta.test.sh                       395
spec/builtin-misc.test.sh                       164
spec/builtin-printf.test.sh                   1,597
spec/builtin-process.test.sh                    622
spec/builtin-read.test.sh                     1,185
spec/builtin-set.test.sh                        424
spec/builtin-special.test.sh                    310
spec/builtin-times.test.sh                       16
spec/builtin-trap-bash.test.sh                  657
spec/builtin-trap-err.test.sh                   647
spec/builtin-trap.test.sh                       563
spec/builtin-type-bash.test.sh                  369
spec/builtin-type.test.sh                       166
spec/builtin-vars.test.sh                       707
spec/case_.test.sh                              244
spec/command-parsing.test.sh                     65
spec/command-sub-ksh.test.sh                    109
spec/command-sub.test.sh                        299
spec/command_.test.sh                           246
spec/comments.test.sh                            12
spec/dbracket.test.sh                           486
spec/divergence.test.sh                         117
spec/dparen.test.sh                             208
spec/empty-bodies.test.sh                        25
spec/errexit-osh.test.sh                        813
spec/errexit.test.sh                            532
spec/exit-status.test.sh                        323
spec/explore-parsing.test.sh                     43
spec/extglob-files.test.sh                      384
spec/extglob-match.test.sh                      375
spec/fatal-errors.test.sh                       187
spec/for-expr.test.sh                           173
spec/func-parsing.test.sh                       106
spec/glob-bash.test.sh                          144
spec/glob.test.sh                               386
spec/globignore.test.sh                         164
spec/globstar.test.sh                            97
spec/hay-isolation.test.sh                      169
spec/hay-meta.test.sh                           307
spec/hay.test.sh                                888
spec/here-doc.test.sh                           429
spec/if_.test.sh                                 55
spec/interactive-parse.test.sh                   37
spec/interactive.test.sh                        401
spec/introspect.test.sh                         287
spec/known-differences.test.sh                   28
spec/let.test.sh                                 23
spec/loop.test.sh                               615
spec/nameref.test.sh                            637
spec/nix-idioms.test.sh                         196
spec/nocasematch-match.test.sh                   71
spec/nul-bytes.test.sh                          519
spec/osh-bugs.test.sh                            64
spec/paren-ambiguity.test.sh                    149
spec/parse-errors.test.sh                       215
spec/pipeline.test.sh                           277
spec/posix.test.sh                              161
spec/print-source-code.test.sh                   63
spec/process-sub.test.sh                        204
spec/prompt.test.sh                             349
spec/quote.test.sh                              308
spec/redir-order.test.sh                         71
spec/redirect-command.test.sh                   344
spec/redirect-multi.test.sh                     308
spec/redirect.test.sh                           595
spec/regex.test.sh                              635
spec/serialize.test.sh                          232
spec/sh-func.test.sh                            186
spec/sh-options-bash.test.sh                    153
spec/sh-options.test.sh                         765
spec/sh-usage.test.sh                           243
spec/shell-bugs.test.sh                          36
spec/shell-grammar.test.sh                      208
spec/smoke.test.sh                              126
spec/spec-harness-bug.test.sh                     9
spec/strict-options.test.sh                     351
spec/subshell.test.sh                            23
spec/temp-binding.test.sh                       161
spec/tilde.test.sh                              184
spec/toysh-posix.test.sh                        395
spec/toysh.test.sh                              140
spec/type-compat.test.sh                        159
spec/unicode.test.sh                            197
spec/var-num.test.sh                             43
spec/var-op-bash.test.sh                        536
spec/var-op-len.test.sh                         269
spec/var-op-patsub.test.sh                      413
spec/var-op-slice.test.sh                       410
spec/var-op-strip.test.sh                       398
spec/var-op-test.test.sh                        829
spec/var-ref.test.sh                            769
spec/var-sub-quote.test.sh                      362
spec/var-sub.test.sh                             64
spec/vars-bash.test.sh                           31
spec/vars-special.test.sh                       797
spec/whitespace.test.sh                         114
spec/word-eval.test.sh                           65
spec/word-split.test.sh                         912
spec/xtrace.test.sh                             428
spec/ysh-TODO-deprecate.test.sh                 184
spec/ysh-assign.test.sh                         424
spec/ysh-augmented.test.sh                      249
spec/ysh-bin.test.sh                             20
spec/ysh-blocks.test.sh                         348
spec/ysh-bugs.test.sh                           400
spec/ysh-builtin-ctx.test.sh                    168
spec/ysh-builtin-error.test.sh                  436
spec/ysh-builtin-eval.test.sh                   860
spec/ysh-builtin-help.test.sh                   124
spec/ysh-builtin-meta.test.sh                   353
spec/ysh-builtin-module.test.sh                 500
spec/ysh-builtin-private.test.sh                596
spec/ysh-builtin-process.test.sh                 85
spec/ysh-builtin-shopt.test.sh                  183
spec/ysh-builtin-trap.test.sh                    46
spec/ysh-builtins.test.sh                       702
spec/ysh-bytecode.test.sh                        18
spec/ysh-case.test.sh                           245
spec/ysh-closures.test.sh                       180
spec/ysh-cmd-lang.test.sh                        54
spec/ysh-command-sub.test.sh                    147
spec/ysh-completion.test.sh                     204
spec/ysh-control-flow.test.sh                   182
spec/ysh-convert.test.sh                        202
spec/ysh-demo.test.sh                            61
spec/ysh-dev.test.sh                            163
spec/ysh-dict.test.sh                           126
spec/ysh-env.test.sh                            309
spec/ysh-expr-arith.test.sh                     663
spec/ysh-expr-bool.test.sh                      186
spec/ysh-expr-compare.test.sh                   429
spec/ysh-expr-sub.test.sh                       121
spec/ysh-expr.test.sh                           460
spec/ysh-for.test.sh                            266
spec/ysh-func-builtin.test.sh                   184
spec/ysh-func.test.sh                           514
spec/ysh-funcs-external.test.sh                  86
spec/ysh-glob.test.sh                            36
spec/ysh-int-float.test.sh                      224
spec/ysh-interactive.test.sh                     41
spec/ysh-introspect.test.sh                     394
spec/ysh-json.test.sh                         1,288
spec/ysh-keywords.test.sh                        37
spec/ysh-list.test.sh                           262
spec/ysh-method-io.test.sh                      129
spec/ysh-method-other.test.sh                   155
spec/ysh-methods.test.sh                        690
spec/ysh-multiline.test.sh                      180
spec/ysh-namespaces.test.sh                     107
spec/ysh-nul-bytes.test.sh                       35
spec/ysh-object.test.sh                         337
spec/ysh-old.test.sh                             15
spec/ysh-options-assign.test.sh                 106
spec/ysh-options.test.sh                        760
spec/ysh-place.test.sh                          136
spec/ysh-printing.test.sh                       396
spec/ysh-proc-meta.test.sh                      267
spec/ysh-proc.test.sh                           865
spec/ysh-prompt.test.sh                         147
spec/ysh-purity.test.sh                         407
spec/ysh-regex-api.test.sh                    1,062
spec/ysh-regex-bytes-chars.test.sh              281
spec/ysh-regex.test.sh                          642
spec/ysh-reserved.test.sh                        48
spec/ysh-scope.test.sh                          908
spec/ysh-slice-range.test.sh                    278
spec/ysh-source.test.sh                          29
spec/ysh-special-vars.test.sh                    50
spec/ysh-stdlib-args.test.sh                     37
spec/ysh-stdlib.test.sh                          25
spec/ysh-string.test.sh                         542
spec/ysh-tuple.test.sh                           28
spec/ysh-unicode.test.sh                        185
spec/ysh-usage.test.sh                          287
spec/ysh-user-feedback.test.sh                  213
spec/ysh-var-sub.test.sh                         45
spec/ysh-with-sh.test.sh                        221
spec/ysh-word-eval.test.sh                      144
spec/ysh-xtrace.test.sh                         589
spec/zsh-assoc.test.sh                           63
spec/zsh-idioms.test.sh                          57

71,348 lines in 222 files  

Gold Tests

Another suite that tests shells "from the outside". Instead of making explicit assertions, we verify that OSH behaves like bash.

test/gold/abuild.sh                               9
test/gold/and-or.sh                              27
test/gold/autoconf-backtick.sh                   50
test/gold/big-here-doc.sh                     1,531
test/gold/case-in-subshell.sh                    37
test/gold/char-class.sh                          33
test/gold/command-sub-2.sh                       12
test/gold/command-sub.sh                         39
test/gold/comments.sh                            22
test/gold/complex-here-docs.sh                  192
test/gold/configure-bug.sh                       13
test/gold/de1.sh                                 24
test/gold/declare.sh                             28
test/gold/dollar-sq.sh                           42
test/gold/echo-e.sh                              36
test/gold/errexit-confusion.sh                  139
test/gold/errexit.sh                            121
test/gold/export.sh                              65
test/gold/glob.sh                                26
test/gold/lineno.sh                              23
test/gold/nix.sh                                 32
test/gold/readlink.sh                            61
test/gold/readonly.sh                            24
test/gold/scope.sh                              126
test/gold/strip-op-char-class.sh                  6
test/gold/word-eval.sh                           64

2,782 lines in 27 files  

Python Unit Tests

asdl/examples/typed_arith_parse_test.py         230
asdl/format_test.py                              39
asdl/front_end_test.py                          217
asdl/gen_python_test.py                         180
asdl/target_lang_test.py                        128
benchmarks/uftrace_allocs_test.py                20
build/dynamic_deps_test.py                       23
build/ninja_lib_test.py                         260
builtin/bracket_osh_test.py                      26
builtin/completion_osh_test.py                   16
builtin/func_misc_test.py                        27
builtin/misc_osh_test.py                         28
builtin/read_osh_test.py                         35
builtin/readline_osh_test.py                    100
builtin/trap_osh_test.py                         42
client/py_fanos_test.py                         108
core/alloc_test.py                               52
core/comp_ui_test.py                            177
core/completion_test.py                         826
core/process_test.py                            719
core/pyutil_test.py                              16
core/state_test.py                              353
core/util_test.py                                63
data_lang/htm8_test.py                          579
data_lang/htm8_util_test.py                     234
data_lang/j8_test.py                            229
data_lang/pyj8_test.py                           49
devtools/services/zulip_test.py                  64
display/pretty_test.py                          130
display/ui_test.py                               29
doctools/cmark_test.py                          300
doctools/help_gen_test.py                       119
doctools/html_lib_test.py                        50
doctools/html_old_test.py                       150
doctools/oils_doc_test.py                        86
doctools/spelling_test.py                        50
doctools/split_doc_test.py                      108
doctools/ul_table_test.py                       432
frontend/args_test.py                           237
frontend/id_kind_def_test.py                     97
frontend/lexer_def_test.py                      462
frontend/lexer_gen_test.py                       53
frontend/lexer_test.py                          241
frontend/match_test.py                          131
frontend/reader_test.py                          74
frontend/typed_args_test.py                     158
mycpp/format_strings_test.py                     47
mycpp/mops_test.py                               32
mycpp/mylib_test.py                              71
mycpp/pass_state_test.py                        596
osh/arith_parse_test.py                         185
osh/bool_parse_test.py                          176
osh/braces_test.py                              185
osh/cmd_eval_test.py                             81
osh/cmd_parse_test.py                         1,487
osh/glob_test.py                                169
osh/history_test.py                             126
osh/prompt_test.py                               57
osh/sh_expr_eval_test.py                        103
osh/split_test.py                               181
osh/string_ops_test.py                          203
osh/word_compile_test.py                         16
osh/word_eval_test.py                           157
osh/word_parse_test.py                          607
osh/word_test.py                                176
pyext/fanos_test.py                             274
pyext/fastfunc_test.py                           83
pyext/fastlex_test.py                            85
pyext/libc_test.py                              376
pyext/line_input_test.py                         27
pyext/posix_test.py                             188
pylib/os_path_test.py                            59
pylib/path_stat_test.py                          34
soil/web_test.py                                 54
spec/stateful/harness_test.py                    46
test/sh_spec_test.py                            186
test/wild_report_test.py                         28
tools/fmt_test.py                                18
tools/ysh_ify_test.py                            78
ysh/expr_parse_test.py                          163
ysh/regex_translate_test.py                      51
ysh/val_ops_test.py                              40

14,192 lines in 83 files  

Other Shell Tests

test/all.sh                                      92
test/alpine.sh                                  347
test/asan.sh                                     41
test/bug-1986.sh                                122
test/bug-2068.sh                                 51
test/bug-2123.sh                                 22
test/bugs.sh                                    231
test/child-state.sh                              86
test/common-test.sh                              22
test/common.sh                                  271
test/configure-effects.sh                        82
test/coverage.sh                                180
test/cpp-unit.sh                                 45
test/fedora.sh                                   22
test/gold.sh                                    169
test/lint.sh                                    316
test/lossless.sh                                148
test/ltrace.sh                                   82
test/manual.sh                                  121
test/nix.sh                                      24
test/nohup.sh                                    89
test/opyc.sh                                    118
test/osh-usage.sh                               273
test/other.sh                                    36
test/parse-err-compare.sh                       138
test/parse-errors.sh                            853
test/process-table-portable.sh                  164
test/process-table.sh                           172
test/process_table.py                           224
test/py2_lint.py                                105
test/py3_lint.py                                 18
test/py_stdlib.sh                                32
test/report.R                                    54
test/runtime-errors.sh                        1,341
test/sh-assert.sh                               164
test/sh_spec.py                               1,668
test/shebang.sh                                  26
test/shell-vs-shell.sh                          313
test/signal-report.sh                            62
test/signal-state.sh                             78
test/smoosh.sh                                  220
test/smoosh_import.py                            30
test/souffle-smoke.sh                            19
test/spec-alpine.sh                             154
test/spec-bin.sh                                 45
test/spec-common.sh                             113
test/spec-compat-html.sh                        418
test/spec-compat.sh                             351
test/spec-cpp.sh                                370
test/spec-export.sh                              51
test/spec-py.sh                                 297
test/spec-runner.sh                             454
test/spec-util.sh                                39
test/spec-version.sh                            127
test/spec.sh                                    901
test/spec_lib.py                                308
test/stateful.sh                                238
test/syscall.py                                 280
test/syscall.sh                                 520
test/tools-deps.sh                               76
test/torture.sh                                  37
test/tsv-lib-test.sh                             71
test/tsv-lib.sh                                 111
test/unit.sh                                    292
test/vagrant.sh                                  54
test/wild-runner.sh                             204
test/wild.sh                                    730
test/wild_report.py                             832
test/ysh-every-string.sh                        183
test/ysh-ify.sh                                 657
test/ysh-parse-errors.sh                      1,811
test/ysh-runtime-errors.sh                    1,224

19,619 lines in 73 files  

Test Automation

asdl/TEST.sh                                    128
core/TEST.sh                                     21
cpp/TEST.sh                                     117
data_lang/TEST.sh                                34
frontend/TEST.sh                                 22
mycpp/TEST.sh                                   457
opy/TEST.sh                                     270
pea/TEST.sh                                     411
stdlib/TEST.sh                                   75
yaks/TEST.sh                                    123

1,658 lines in 11 files  

mycpp Python Runtime

Stubs that are re-implemented in C++

mycpp/iolib.py                                  165
mycpp/mops.py                                   293
mycpp/mylib.py                                  531

989 lines in 4 files  

mycpp Translator

This prototype uses the MyPy frontend to translate statically-typed Python to C++. The generated code calls a small runtime which implements things like List[T], Dict[K, V], and Python's len().

mycpp/const_pass.py                             298
mycpp/control_flow_pass.py                      586
mycpp/conversion_pass.py                        417
mycpp/cppgen_pass.py                          2,893
mycpp/crash.py                                   38
mycpp/format_strings.py                         124
mycpp/mycpp_main.py                             291
mycpp/pass_state.py                             610
mycpp/translate.py                              234
mycpp/util.py                                   200
mycpp/visitor.py                                770

6,461 lines in 12 files  

mycpp Test Data

Small Python examples that translate to C++, compile, and run.

mycpp/examples/__init__.py                        0
mycpp/examples/asdl_generated.py                 65
mycpp/examples/cartesian.py                      78
mycpp/examples/cgi.py                            62
mycpp/examples/classes.py                       296
mycpp/examples/containers.py                    289
mycpp/examples/control_flow.py                  113
mycpp/examples/escape.py                         70
mycpp/examples/fib_iter.py                       53
mycpp/examples/fib_recursive.py                  49
mycpp/examples/files.py                          64
mycpp/examples/gc_stack_roots.py                207
mycpp/examples/invalid_assignment.py             24
mycpp/examples/invalid_condition.py              31
mycpp/examples/invalid_ctx_name.py               12
mycpp/examples/invalid_ctx_raise.py              44
mycpp/examples/invalid_default_args.py           44
mycpp/examples/invalid_except.py                 38
mycpp/examples/invalid_format_args.py            18
mycpp/examples/invalid_format_strings.py         22
mycpp/examples/invalid_global.py                 18
mycpp/examples/invalid_other.py                  63
mycpp/examples/invalid_partialtype.py            27
mycpp/examples/invalid_print.py                  13
mycpp/examples/invalid_python.py                 61
mycpp/examples/invalid_switch.py                 84
mycpp/examples/invalid_switch_break.py           58
mycpp/examples/invalid_try_else.py               38
mycpp/examples/invalid_types.py                  31
mycpp/examples/invalid_types_2.py                31
mycpp/examples/invalid_unreachable_code.py         15
mycpp/examples/length.py                        109
mycpp/examples/lexer_main.py                     47
mycpp/examples/loops.py                         229
mycpp/examples/modules.py                        81
mycpp/examples/parse.py                         344
mycpp/examples/pea_hello.py                      10
mycpp/examples/pgen2_demo.py                    106
mycpp/examples/scoped_resource.py               259
mycpp/examples/test_arith_ops.py                 45
mycpp/examples/test_cast.py                     211
mycpp/examples/test_classes_gc.py               212
mycpp/examples/test_conditional.py               66
mycpp/examples/test_ctx_pattern.py               65
mycpp/examples/test_default_args.py              75
mycpp/examples/test_func_method_name_conflict.py         69
mycpp/examples/test_globals.py                   62
mycpp/examples/test_hoist.py                     61
mycpp/examples/test_integers.py                  75
mycpp/examples/test_io_os_error.py              131
mycpp/examples/test_iterators.py                101
mycpp/examples/test_small_str.py                 33
mycpp/examples/test_strings.py                  277
mycpp/examples/test_switch.py                    80
mycpp/examples/tuple_return_value.py             83
mycpp/examples/varargs.py                        44

4,893 lines in 57 files  

Zephyr ASDL

A DSL for algebraic data types, borrowed from Python. Oils is the most strongly typed Bourne shell implementation!

asdl/asdl_main.py                               344
asdl/ast.py                                     270
asdl/format.py                                  157
asdl/front_end.py                               590
asdl/gen_cpp.py                                 892
asdl/gen_python.py                              649
asdl/pybase.py                                   39
asdl/runtime.py                                  68
asdl/util.py                                     11
asdl/visitor.py                                 173

3,193 lines in 11 files  

pgen2 Parser Generator

An LL(1) parser generator used to parse YSH expressions. Also borrowed from CPython.

pgen2/driver.py                                 101
pgen2/grammar.py                                303
pgen2/parse.py                                  225
pgen2/pgen.py                                   430
pgen2/pnode.py                                   67
pgen2/token.py                                  149
pgen2/tokenize.py                               579

1,854 lines in 8 files  

Other Code Generators

In order to make Oils statically typed, we had to abandon Python reflection and use C++ source code generation instead. The lexer, flag definitions, and constants can be easily compiled to C++.

core/optview_gen.py                              93
cpp/embedded_file_gen.py                         66
doctools/help_gen.py                            694
frontend/consts_gen.py                          653
frontend/flag_gen.py                            546
frontend/lexer_gen.py                           503
frontend/option_gen.py                           94
frontend/signal_gen.py                          119
osh/arith_parse_gen.py                           99
ysh/grammar_gen.py                              245

3,112 lines in 11 files  

Yaks

Experimental replacement for mycpp

yaks/gen_cpp.py                                 104
yaks/transform.py                               250
yaks/yaks_main.py                               153

507 lines in 4 files  

Hand-written C++ Code

Includes OS bindings. Small C++ files like cpp/osh_arith_parse.{cc,h} correspond to larger Python files like osh/arith_parse.py.

cpp/core.cc                                     442
cpp/core.h                                      133
cpp/data_lang.cc                                273
cpp/data_lang.h                                  34
cpp/embedded_file.h                              18
cpp/fanos.cc                                     57
cpp/fanos.h                                      18
cpp/fanos_shared.h                               49
cpp/frontend_flag_spec.cc                       234
cpp/frontend_flag_spec.h                        153
cpp/frontend_match.cc                           164
cpp/frontend_match.h                             94
cpp/frontend_pyreadline.cc                      399
cpp/frontend_pyreadline.h                        94
cpp/libc.cc                                     266
cpp/libc.h                                       68
cpp/libc_demo.cc                                217
cpp/osh.cc                                      164
cpp/osh.h                                        50
cpp/osh_tdop.cc                                  16
cpp/osh_tdop.h                                   52
cpp/pgen2.cc                                     44
cpp/pgen2.h                                      86
cpp/pylib.cc                                     66
cpp/pylib.h                                      24
cpp/stdlib.cc                                   264
cpp/stdlib.h                                    170
cpp/translation_stubs.h                          11
cpp/unicode_demo.cc                             105

3,765 lines in 30 files  

Garbage-Collected Runtime

Uses a fork-friendly Mark-Sweep collector.

mycpp/common.h                                   92
mycpp/comparators.h                              90
mycpp/gc_alloc.h                                228
mycpp/gc_builtins.cc                            469
mycpp/gc_builtins.h                             175
mycpp/gc_dict.h                                 471
mycpp/gc_iolib.cc                                46
mycpp/gc_iolib.h                                175
mycpp/gc_list.h                                 546
mycpp/gc_mops.cc                                 79
mycpp/gc_mops.h                                 120
mycpp/gc_mylib.cc                               335
mycpp/gc_mylib.h                                398
mycpp/gc_obj.h                                  185
mycpp/gc_slab.h                                  58
mycpp/gc_str.cc                                 674
mycpp/gc_str.h                                  229
mycpp/gc_tuple.h                                151
mycpp/hash.cc                                    45
mycpp/hash.h                                     22
mycpp/mark_sweep_heap.cc                        457
mycpp/mark_sweep_heap.h                         352
mycpp/probes.h                                   39
mycpp/runtime.h                                  25
mycpp/test_common.h                              24

5,485 lines in 26 files  

Unit tests in C++

The goal is to make the spec tests pass, but unit tests are helpful too.

cpp/core_test.cc                                361
cpp/data_lang_test.cc                           178
cpp/data_race_test.cc                           206
cpp/frontend_flag_spec_test.cc                  154
cpp/frontend_match_test.cc                       87
cpp/libc_test.cc                                196
cpp/obj_layout_test.cc                          215
cpp/osh_test.cc                                  50
cpp/pgen2_test.cc                                32
cpp/pylib_test.cc                                67
cpp/stdlib_test.cc                              158
mycpp/bump_leak_heap_test.cc                     62
mycpp/float_test.cc                             150
mycpp/gc_builtins_test.cc                       460
mycpp/gc_dict_test.cc                           745
mycpp/gc_heap_test.cc                           479
mycpp/gc_iolib_test.cc                          109
mycpp/gc_list_test.cc                           593
mycpp/gc_mops_test.cc                           182
mycpp/gc_mylib_test.cc                          353
mycpp/gc_str_test.cc                          1,582
mycpp/gc_stress_test.cc                         222
mycpp/gc_tuple_test.cc                          139
mycpp/mark_sweep_heap_test.cc                   370
mycpp/small_str_test.cc                         502

7,652 lines in 26 files  

Incremental C++ Build

NINJA-config.sh                                 108
asdl/NINJA_subgraph.py                           56
bin/NINJA_subgraph.py                           214
build/ninja-rules-cpp.sh                        431
build/ninja-rules-py.sh                         368
build/ninja_lib.py                              807
build/ninja_lib_test.py                         260
build/ninja_main.py                             541
core/NINJA_subgraph.py                           54
cpp/NINJA_subgraph.py                           231
data_lang/NINJA_subgraph.py                      66
display/NINJA_subgraph.py                        16
frontend/NINJA_subgraph.py                      177
mycpp/NINJA_subgraph.py                         467
osh/NINJA_subgraph.py                            32
pea/NINJA_subgraph.py                            19
prebuilt/NINJA_subgraph.py                       79
yaks/NINJA_subgraph.py                           82
ysh/NINJA_subgraph.py                            39

4,047 lines in 20 files  

Build Automation

build/clean.sh                                   65
build/common.sh                                  86
build/cpp.sh                                     40
build/deps.sh                                 1,424
build/detect-cc.c                                 4
build/detect-fnm-extmatch.c                       6
build/detect-glob-period.c                        6
build/detect-pwent.c                              9
build/detect-readline.c                          19
build/detect-systemtap-sdt.c                      6
build/dev-setup-test.sh                          30
build/dev-shell-test.sh                          95
build/dev-shell.sh                               86
build/dev.sh                                      5
build/doc.sh                                  1,015
build/dynamic-deps.sh                           318
build/dynamic_deps.py                           166
build/native.sh                                 208
build/ninja-rules-cpp.sh                        431
build/ninja-rules-py.sh                         368
build/ninja_lib.py                              807
build/ninja_main.py                             541
build/oils-preamble.sh                          169
build/old-wedges.sh                             121
build/py.sh                                     497
build/py2.sh                                     20
build/stamp.sh                                   76
build/static-oils.sh                             50
configure                                       587
install                                         175

7,430 lines in 31 files  

Build of oils-ref Tarball

Makefile                                        196
build/ref/c_module_srcs.py                       84
build/ref/c_module_toc.py                        49
build/ref/hello.mk                               51
build/ref/make_zip.py                            53
build/ref/oils-ref.mk                            61
build/ref/old-ovm-test.sh                        42
build/ref/ovm-actions.sh                        190
build/ref/ovm-compile.sh                        430
build/ref/portable-rules.mk                      96

1,252 lines in 11 files  

Release Automation

devtools/release-history.sh                     207
devtools/release-native.sh                      245
devtools/release-note.sh                        285
devtools/release-patch.sh                        26
devtools/release-test.sh                         36
devtools/release-version.sh                   1,167
devtools/release.sh                             837

2,803 lines in 8 files  

Soil: Multi-cloud CI with containers

soil/__init__.py                                  0
soil/admin.sh                                    28
soil/collect_json.py                             35
soil/common.sh                                  114
soil/cpp-tarball.sh                             131
soil/diagnose.sh                                142
soil/github-actions.sh                          149
soil/github-tool.sh                              43
soil/history.sh                                  78
soil/host-shim.sh                               434
soil/maybe-merge.sh                             153
soil/sourcehut.sh                                81
soil/tests.sh                                    20
soil/timeout-test.sh                             50
soil/wait.sh                                    142
soil/web-init.sh                                190
soil/web-remote-test.sh                          46
soil/web-worker.sh                              429
soil/web.py                                     798
soil/web.sh                                     243
soil/web_test.py                                 54
soil/worker.sh                                  710

4,070 lines in 23 files  

Benchmarks

benchmarks/__init__.py                            0
benchmarks/auto.sh                              107
benchmarks/autoconf.sh                          652
benchmarks/awk-python.sh                         97
benchmarks/builtin-io.sh                         60
benchmarks/cachegrind.sh                         30
benchmarks/cachegrind_to_tsv.py                  54
benchmarks/callgrind.sh                          65
benchmarks/common.R                              98
benchmarks/common.sh                            115
benchmarks/compute.sh                           735
benchmarks/fake_libc.py                          14
benchmarks/gc-cachegrind.sh                      92
benchmarks/gc.sh                                738
benchmarks/gc_stats_to_tsv.py                    51
benchmarks/gperftools.sh                         60
benchmarks/id-test.sh                            42
benchmarks/id.sh                                483
benchmarks/micro.sh                              45
benchmarks/mimalloc.sh                          107
benchmarks/mycpp.sh                             117
benchmarks/osh-parser.sh                        528
benchmarks/osh-runtime.sh                       585
benchmarks/osh_runtime.py                        64
benchmarks/ovm-build.sh                         491
benchmarks/parse-help.sh                         65
benchmarks/perf.sh                              482
benchmarks/private-builtin.sh                   151
benchmarks/pypy.sh                               42
benchmarks/pytrace.py                            92
benchmarks/pytrace.sh                           184
benchmarks/report.R                           1,390
benchmarks/report.sh                            111
benchmarks/report_test.R                         36
benchmarks/startup.sh                           165
benchmarks/systemtap.sh                          75
benchmarks/time-test.sh                         324
benchmarks/time_.py                             270
benchmarks/tokens.sh                             24
benchmarks/uftrace.sh                           349
benchmarks/uftrace_allocs.py                    180
benchmarks/uftrace_allocs_test.py                20
benchmarks/uftrace_plugin.py                     99
benchmarks/virtual_memory.py                     59
benchmarks/vm-baseline.sh                       180
benchmarks/wedge.R                               97
benchmarks/wedge.sh                              41
benchmarks/ysh-json.sh                          160

10,026 lines in 49 files  

Metrics

metrics/bytecode.R                              388
metrics/bytecode.sh                             143
metrics/cpython-defs.R                           72
metrics/native-code.R                            88
metrics/native-code.sh                          197
metrics/source-code.sh                          627
metrics/tarball.sh                              154

1,669 lines in 8 files  

Generated Python Code

For the Python App Bundle.

_devbuild/gen/__init__.py                         0
_devbuild/gen/arg_types.py                      444
_devbuild/gen/arith_nt.py                         4
_devbuild/gen/demo_lib_asdl.py                  217
_devbuild/gen/expr_asdl.py                      254
_devbuild/gen/find_asdl.py                      837
_devbuild/gen/find_nt.py                         10
_devbuild/gen/grammar_nt.py                      74
_devbuild/gen/help_meta.py                      524
_devbuild/gen/hnode_asdl.py                     126
_devbuild/gen/htm8_asdl.py                      166
_devbuild/gen/id_kind.py                        529
_devbuild/gen/id_kind_asdl.py                   946
_devbuild/gen/mycpp_asdl.py                   1,200
_devbuild/gen/nil8_asdl.py                      294
_devbuild/gen/option_asdl.py                    458
_devbuild/gen/pretty_asdl.py                    336
_devbuild/gen/runtime_asdl.py                 1,539
_devbuild/gen/shared_variant_asdl.py            442
_devbuild/gen/syntax_asdl.py                  7,944
_devbuild/gen/typed_arith_asdl.py               441
_devbuild/gen/typed_demo_asdl.py              1,177
_devbuild/gen/types_asdl.py                     184
_devbuild/gen/value_asdl.py                   1,750
_devbuild/gen/yaks_asdl.py                    1,469

21,365 lines in 26 files  

Doc Tools

doctools/cmark.py                               538
doctools/doc_html.py                            116
doctools/fmt_check.py                            91
doctools/html_head.py                            99
doctools/html_lib.py                             51
doctools/html_old.py                            377
doctools/micro_syntax.cc                      1,157
doctools/micro_syntax.re2c.h                    907
doctools/oils_doc.py                            707
doctools/ref_check.py                           182
doctools/spelling.py                            143
doctools/split_doc.py                           160
doctools/src_tree.py                            474
doctools/ul_table.py                            568
doctools/util.py                                 13

5,583 lines in 16 files  

Web

web/ajax.js                                     149
web/osh-to-oil.js                                79
web/table/csv2html.py                           449
web/table/schema2sqlite.py                       45
web/table/table-sort.js                         433

1,155 lines in 6 files