Overview of Oils Code

category total lines num files
OSH (and common libraries) 45,437 86
YSH 10,850 26
Data Languages 2,171 9
Tools 1,330 5
OSH stdlib 560 9
YSH stdlib 1,570 13
Code Borrowed from Python's stdlib 429 5
Spec Tests 58,999 191
Gold Tests 2,775 27
Python Unit Tests 12,481 79
Other Shell Tests 17,953 66
Test Automation 1,617 11
mycpp Python Runtime 948 4
mycpp Translator 6,156 12
mycpp Test Data 4,582 50
Zephyr ASDL 3,196 11
pgen2 Parser Generator 1,854 8
Other Code Generators 3,053 11
Yaks 507 4
Hand-written C++ Code 3,353 29
Garbage-Collected Runtime 5,368 26
Unit tests in C++ 7,828 27
Incremental C++ Build 3,708 20
Build Automation 8,268 39
Release Automation 2,570 8
Soil: Multi-cloud CI with containers 3,861 23
Benchmarks 9,763 49
Metrics 1,642 8
Generated Python Code 20,730 25
Doc Tools 5,314 16
Web 1,086 5

Related Documents

The README for oilshell/oil has another overview of the repository.

OSH (and common libraries)

This is the input to the translators, written in statically-typed Python. Note that bash is at least 140K lines of code, and OSH implements a large part of bash and more.

bin/oils_for_unix.py                            202
builtin/assign_osh.py                           558
builtin/bracket_osh.py                          292
builtin/completion_osh.py                       507
builtin/dirs_osh.py                             348
builtin/io_osh.py                               182
builtin/misc_osh.py                             127
builtin/printf_osh.py                           552
builtin/process_osh.py                          617
builtin/pure_osh.py                             588
builtin/read_osh.py                             525
builtin/readline_osh.py                         280
builtin/trap_osh.py                             322
core/alloc.py                                   320
core/bash_impl.py                               486
core/comp_ui.py                                 576
core/completion.py                            1,517
core/dev.py                                     762
core/error.py                                   345
core/executor.py                                858
core/main_loop.py                               413
core/num.py                                      24
core/optview.py                                  58
core/process.py                               1,972
core/pyos.py                                    290
core/pyutil.py                                  225
core/runtime.asdl                               191
core/sh_init.py                                 293
core/shell.py                                 1,225
core/state.py                                 2,805
core/util.py                                    168
core/value.asdl                                 226
core/vm.py                                      342
display/ansi.py                                  15
display/pp_hnode.py                             311
display/pp_value.py                             349
display/pretty.asdl                              44
display/pretty.py                               333
display/ui.py                                   602
frontend/args.py                                681
frontend/builtin_def.py                         166
frontend/consts.py                              399
frontend/flag_def.py                            564
frontend/flag_spec.py                           351
frontend/flag_util.py                            69
frontend/id_kind_def.py                         810
frontend/lexer.py                               455
frontend/lexer_def.py                         1,138
frontend/location.py                            577
frontend/match.py                               310
frontend/option_def.py                          419
frontend/parse_lib.py                           405
frontend/py_readline.py                         161
frontend/reader.py                              280
frontend/signal_def.py                           94
frontend/syntax.asdl                            655
frontend/syntax_abbrev.py                       146
frontend/typed_args.py                          684
frontend/types.asdl                              61
osh/arith_parse.py                              197
osh/bool_parse.py                               272
osh/bool_stat.py                                135
osh/braces.py                                   539
osh/cmd_eval.py                               2,330
osh/cmd_parse.py                              2,865
osh/glob_.py                                    508
osh/history.py                                  193
osh/prompt.py                                   357
osh/sh_expr_eval.py                           1,298
osh/split.py                                    320
osh/string_ops.py                               545
osh/tdop.py                                     342
osh/word_.py                                    805
osh/word_compile.py                             309
osh/word_eval.py                              2,548
osh/word_parse.py                             2,218
pyext/fanos.c                                   119
pyext/fastfunc.c                                133
pyext/fastfunc.pyi                               11
pyext/fastlex.c                                 389
pyext/fastlex.pyi                                21
pyext/libc.c                                    422
pyext/libc.pyi                                   19
pyext/line_input.pyi                             57
pyext/posix_.pyi                                210

45,437 lines in 86 files  

YSH

Expression grammar, parser, evaluator, etc.

builtin/completion_ysh.py                        69
builtin/error_ysh.py                            307
builtin/func_eggex.py                           186
builtin/func_hay.py                             157
builtin/func_misc.py                            794
builtin/func_reflect.py                         216
builtin/hay_ysh.py                              429
builtin/io_ysh.py                               298
builtin/json_ysh.py                             139
builtin/meta_oils.py                            855
builtin/method_dict.py                           95
builtin/method_io.py                            253
builtin/method_list.py                          138
builtin/method_other.py                          32
builtin/method_str.py                           587
builtin/method_type.py                          141
builtin/module_ysh.py                           141
builtin/pure_ysh.py                             229
ysh/expr_eval.py                              1,591
ysh/expr_parse.py                               389
ysh/expr_to_ast.py                            1,721
ysh/func_proc.py                                597
ysh/grammar.pgen2                               541
ysh/regex_translate.py                          393
ysh/val_ops.py                                  552

10,850 lines in 26 files  

Data Languages

JSON, J8 Notation, ...

data_lang/nil8.asdl                              31
data_lang/j8.py                               1,326
data_lang/j8_lite.py                             54
data_lang/pyj8.py                                27
data_lang/j8.h                                  314
data_lang/j8_libc.c                             221
data_lang/j8_libc.h                              28
data_lang/utf8.h                                170

2,171 lines in 9 files  

Tools

tools/deps.py                                   149
tools/fmt.py                                     25
tools/readlink.py                                31
tools/ysh_ify.py                              1,125

1,330 lines in 5 files  

OSH stdlib

stdlib/osh/bash-strict.sh                         8
stdlib/osh/byo-server-test.sh                    73
stdlib/osh/byo-server.sh                        138
stdlib/osh/no-quotes-test.sh                     82
stdlib/osh/no-quotes.sh                         137
stdlib/osh/task-five.sh                          62
stdlib/osh/two-test.sh                           33
stdlib/osh/two.sh                                27

560 lines in 9 files  

YSH stdlib

stdlib/ysh/args-test.ysh                        352
stdlib/ysh/args.ysh                             281
stdlib/ysh/def-test.ysh                          43
stdlib/ysh/def.ysh                                7
stdlib/ysh/list-test.ysh                         87
stdlib/ysh/list.ysh                              64
stdlib/ysh/math-test.ysh                         64
stdlib/ysh/math.ysh                              87
stdlib/ysh/stream.ysh                           310
stdlib/ysh/table.ysh                            150
stdlib/ysh/yblocks-test.ysh                      72
stdlib/ysh/yblocks.ysh                           53

1,570 lines in 13 files  

Code Borrowed from Python's stdlib

pylib/cgi.py                                     19
pylib/collections_.py                           214
pylib/os_path.py                                169
pylib/path_stat.py                               27

429 lines in 5 files  

Spec Tests

A comprehensive test suite that compares OSH against other shells. If OSH passes these tests in BOTH Python and C++, it means that the translation works.

spec/alias.test.sh                              571
spec/append.test.sh                             303
spec/arith-context.test.sh                      216
spec/arith-dynamic.test.sh                       95
spec/arith.test.sh                              998
spec/array-basic.test.sh                         45
spec/array-compat.test.sh                       189
spec/array.test.sh                            1,010
spec/assign-deferred.test.sh                    107
spec/assign-dialects.test.sh                     61
spec/assign-extended.test.sh                    881
spec/assign.test.sh                             674
spec/assoc-zsh.test.sh                           59
spec/assoc.test.sh                              835
spec/background.test.sh                         184
spec/ble-features.test.sh                       645
spec/ble-idioms.test.sh                       1,064
spec/blog-other1.test.sh                         57
spec/blog1.test.sh                               95
spec/blog2.test.sh                               60
spec/brace-expansion.test.sh                    502
spec/bugs.test.sh                               418
spec/builtin-bash.test.sh                       192
spec/builtin-bind.test.sh                       101
spec/builtin-bracket.test.sh                    767
spec/builtin-cd.test.sh                         333
spec/builtin-completion.test.sh                 596
spec/builtin-dirs.test.sh                       241
spec/builtin-echo.test.sh                       248
spec/builtin-eval-source.test.sh                345
spec/builtin-getopts.test.sh                    347
spec/builtin-history.test.sh                    198
spec/builtin-meta.test.sh                       620
spec/builtin-misc.test.sh                       161
spec/builtin-printf.test.sh                   1,284
spec/builtin-process.test.sh                    588
spec/builtin-read.test.sh                       769
spec/builtin-special.test.sh                    141
spec/builtin-times.test.sh                       16
spec/builtin-trap-bash.test.sh                  657
spec/builtin-trap-err.test.sh                   647
spec/builtin-trap.test.sh                       366
spec/builtin-type-bash.test.sh                  326
spec/builtin-type.test.sh                       117
spec/builtin-vars.test.sh                       707
spec/case_.test.sh                              244
spec/command-parsing.test.sh                     64
spec/command-sub.test.sh                        285
spec/command_.test.sh                           194
spec/comments.test.sh                            11
spec/dbracket.test.sh                           467
spec/dparen.test.sh                             205
spec/empty-bodies.test.sh                        24
spec/errexit-osh.test.sh                        814
spec/errexit.test.sh                            532
spec/exit-status.test.sh                        280
spec/explore-parsing.test.sh                     42
spec/extglob-files.test.sh                      384
spec/extglob-match.test.sh                      400
spec/fatal-errors.test.sh                       186
spec/for-expr.test.sh                           173
spec/func-parsing.test.sh                       106
spec/glob.test.sh                               388
spec/globignore.test.sh                         162
spec/globstar.test.sh                            97
spec/hay-isolation.test.sh                      169
spec/hay-meta.test.sh                           257
spec/hay.test.sh                                655
spec/here-doc.test.sh                           370
spec/if_.test.sh                                 55
spec/interactive-parse.test.sh                   37
spec/interactive.test.sh                        343
spec/introspect.test.sh                         287
spec/let.test.sh                                 23
spec/loop.test.sh                               474
spec/nameref.test.sh                            637
spec/nix-idioms.test.sh                         196
spec/nocasematch-match.test.sh                   71
spec/nul-bytes.test.sh                          159
spec/osh-bugs.test.sh                            31
spec/parse-errors.test.sh                       226
spec/pipeline.test.sh                           234
spec/posix.test.sh                              153
spec/process-sub.test.sh                        204
spec/prompt.test.sh                             253
spec/quote.test.sh                              231
spec/redirect-command.test.sh                   306
spec/redirect-multi.test.sh                     308
spec/redirect.test.sh                           485
spec/regex.test.sh                              631
spec/serialize.test.sh                          230
spec/sh-func.test.sh                            152
spec/sh-options.test.sh                         879
spec/sh-usage.test.sh                            94
spec/shell-grammar.test.sh                      206
spec/smoke.test.sh                              126
spec/strict-options.test.sh                     277
spec/subshell.test.sh                            23
spec/tilde.test.sh                              184
spec/toysh-posix.test.sh                        344
spec/toysh.test.sh                              143
spec/type-compat.test.sh                        159
spec/unicode.test.sh                            196
spec/var-num.test.sh                             43
spec/var-op-bash.test.sh                        430
spec/var-op-len.test.sh                         220
spec/var-op-patsub.test.sh                      413
spec/var-op-slice.test.sh                       400
spec/var-op-strip.test.sh                       357
spec/var-op-test.test.sh                        520
spec/var-ref.test.sh                            778
spec/var-sub-quote.test.sh                      312
spec/var-sub.test.sh                             60
spec/vars-bash.test.sh                           30
spec/vars-special.test.sh                       788
spec/whitespace.test.sh                         114
spec/word-eval.test.sh                           62
spec/word-split.test.sh                         458
spec/xtrace.test.sh                             394
spec/ysh-TODO-deprecate.test.sh                 144
spec/ysh-assign.test.sh                         423
spec/ysh-augmented.test.sh                      245
spec/ysh-bin.test.sh                             20
spec/ysh-blocks.test.sh                         339
spec/ysh-bugs.test.sh                           347
spec/ysh-builtin-ctx.test.sh                    168
spec/ysh-builtin-error.test.sh                  436
spec/ysh-builtin-eval.test.sh                   699
spec/ysh-builtin-help.test.sh                   124
spec/ysh-builtin-meta.test.sh                   351
spec/ysh-builtin-module.test.sh                 489
spec/ysh-builtin-process.test.sh                 85
spec/ysh-builtin-shopt.test.sh                  183
spec/ysh-builtins.test.sh                       676
spec/ysh-case.test.sh                           245
spec/ysh-closures.test.sh                       179
spec/ysh-cmd-lang.test.sh                        54
spec/ysh-command-sub.test.sh                    147
spec/ysh-completion.test.sh                     204
spec/ysh-control-flow.test.sh                   182
spec/ysh-convert.test.sh                        190
spec/ysh-demo.test.sh                            59
spec/ysh-dev.test.sh                            163
spec/ysh-dict.test.sh                           120
spec/ysh-env.test.sh                            309
spec/ysh-expr-arith.test.sh                     614
spec/ysh-expr-bool.test.sh                      185
spec/ysh-expr-compare.test.sh                   427
spec/ysh-expr-sub.test.sh                        42
spec/ysh-expr.test.sh                           442
spec/ysh-for.test.sh                            192
spec/ysh-func-builtin.test.sh                   250
spec/ysh-func.test.sh                           496
spec/ysh-funcs-external.test.sh                  86
spec/ysh-int-float.test.sh                      220
spec/ysh-interactive.test.sh                     41
spec/ysh-json.test.sh                         1,251
spec/ysh-keywords.test.sh                        37
spec/ysh-list.test.sh                           179
spec/ysh-method-io.test.sh                       86
spec/ysh-methods.test.sh                        624
spec/ysh-multiline.test.sh                      179
spec/ysh-namespaces.test.sh                     107
spec/ysh-object.test.sh                         296
spec/ysh-options-assign.test.sh                 106
spec/ysh-options.test.sh                        764
spec/ysh-place.test.sh                          136
spec/ysh-printing.test.sh                       391
spec/ysh-proc-meta.test.sh                      241
spec/ysh-proc.test.sh                           814
spec/ysh-prompt.test.sh                         147
spec/ysh-regex-api.test.sh                      939
spec/ysh-regex.test.sh                          711
spec/ysh-reserved.test.sh                        48
spec/ysh-scope.test.sh                          768
spec/ysh-slice-range.test.sh                    278
spec/ysh-source.test.sh                          29
spec/ysh-special-vars.test.sh                    50
spec/ysh-stdlib-args.test.sh                     37
spec/ysh-stdlib.test.sh                          25
spec/ysh-string.test.sh                         593
spec/ysh-tuple.test.sh                           28
spec/ysh-unicode.test.sh                        184
spec/ysh-usage.test.sh                          107
spec/ysh-user-feedback.test.sh                  141
spec/ysh-var-sub.test.sh                         45
spec/ysh-with-sh.test.sh                        221
spec/ysh-word-eval.test.sh                      144
spec/ysh-xtrace.test.sh                         591
spec/zsh-idioms.test.sh                          45

58,999 lines in 191 files  

Gold Tests

Another suite that tests shells "from the outside". Instead of making explicit assertions, we verify that OSH behaves like bash.

test/gold/abuild.sh                               9
test/gold/and-or.sh                              27
test/gold/autoconf-backtick.sh                   50
test/gold/big-here-doc.sh                     1,531
test/gold/case-in-subshell.sh                    37
test/gold/char-class.sh                          33
test/gold/command-sub-2.sh                       12
test/gold/command-sub.sh                         39
test/gold/comments.sh                            22
test/gold/complex-here-docs.sh                  192
test/gold/configure-bug.sh                       13
test/gold/de1.sh                                 24
test/gold/declare.sh                             28
test/gold/dollar-sq.sh                           42
test/gold/echo-e.sh                              36
test/gold/errexit-confusion.sh                  139
test/gold/errexit.sh                            121
test/gold/export.sh                              65
test/gold/glob.sh                                26
test/gold/lineno.sh                              23
test/gold/nix.sh                                 32
test/gold/readlink.sh                            61
test/gold/readonly.sh                            24
test/gold/scope.sh                              126
test/gold/strip-op-char-class.sh                  6
test/gold/word-eval.sh                           57

2,775 lines in 27 files  

Python Unit Tests

asdl/examples/typed_arith_parse_test.py         230
asdl/format_test.py                              39
asdl/front_end_test.py                          217
asdl/gen_python_test.py                         180
asdl/target_lang_test.py                        128
benchmarks/uftrace_allocs_test.py                20
build/dynamic_deps_test.py                       23
build/ninja_lib_test.py                         255
builtin/bracket_osh_test.py                      26
builtin/completion_osh_test.py                   16
builtin/func_misc_test.py                        27
builtin/misc_osh_test.py                         28
builtin/read_osh_test.py                         35
builtin/readline_osh_test.py                    100
client/py_fanos_test.py                         108
core/alloc_test.py                               54
core/comp_ui_test.py                            177
core/completion_test.py                         827
core/process_test.py                            277
core/pyutil_test.py                              17
core/state_test.py                              353
core/util_test.py                                64
data_lang/j8_test.py                            229
data_lang/pyj8_test.py                           52
display/pretty_test.py                          130
display/ui_test.py                               29
doctools/cmark_test.py                          203
doctools/help_gen_test.py                       116
doctools/html_lib_test.py                        49
doctools/oils_doc_test.py                        81
doctools/spelling_test.py                        49
doctools/split_doc_test.py                      104
doctools/ul_table_test.py                       378
frontend/args_test.py                           237
frontend/id_kind_def_test.py                     97
frontend/lexer_def_test.py                      454
frontend/lexer_gen_test.py                       53
frontend/lexer_test.py                          216
frontend/match_test.py                          131
frontend/reader_test.py                          74
frontend/typed_args_test.py                     158
lazylex/html_test.py                            152
mycpp/format_strings_test.py                     43
mycpp/mops_test.py                               32
mycpp/mylib_test.py                              56
mycpp/pass_state_test.py                        596
osh/arith_parse_test.py                         185
osh/bool_parse_test.py                          122
osh/braces_test.py                              185
osh/cmd_eval_test.py                             81
osh/cmd_parse_test.py                         1,493
osh/glob_test.py                                160
osh/history_test.py                             126
osh/prompt_test.py                               60
osh/sh_expr_eval_test.py                        102
osh/split_test.py                               182
osh/string_ops_test.py                          203
osh/word_compile_test.py                         16
osh/word_eval_test.py                           157
osh/word_parse_test.py                          607
osh/word_test.py                                176
pyext/fanos_test.py                             274
pyext/fastfunc_test.py                           83
pyext/fastlex_test.py                            85
pyext/libc_test.py                              310
pyext/line_input_test.py                         27
pyext/posix_test.py                             184
pylib/os_path_test.py                            59
pylib/path_stat_test.py                          24
soil/web_test.py                                 52
spec/stateful/harness_test.py                    46
test/sh_spec_test.py                            184
test/wild_report_test.py                         28
tools/fmt_test.py                                18
tools/ysh_ify_test.py                            78
ysh/expr_parse_test.py                          163
ysh/regex_translate_test.py                      51
ysh/val_ops_test.py                              40

12,481 lines in 79 files  

Other Shell Tests

test/all.sh                                      92
test/alpine.sh                                  307
test/asan.sh                                     41
test/ble.sh                                     139
test/bug-1986.sh                                122
test/bug-2123.sh                                 22
test/bugs.sh                                    223
test/common-test.sh                              22
test/common.sh                                  271
test/coverage.sh                                180
test/cpp-unit.sh                                 45
test/fedora.sh                                   22
test/gentoo.sh                                   83
test/gold.sh                                    168
test/lint.sh                                    314
test/lossless.sh                                148
test/ltrace.sh                                   49
test/nix.sh                                      24
test/nohup.sh                                    89
test/opyc.sh                                    118
test/osh-usage.sh                               220
test/other.sh                                    34
test/parse-err-compare.sh                       138
test/parse-errors.sh                            849
test/process-table-portable.sh                  164
test/process-table.sh                           171
test/process_table.py                           224
test/py2_lint.py                                105
test/py3_lint.py                                 18
test/py_stdlib.sh                                32
test/report.R                                    54
test/runtime-errors.sh                        1,297
test/sh-assert.sh                               164
test/sh_spec.py                               1,509
test/shebang.sh                                  26
test/shell-vs-shell.sh                          314
test/signal-state.sh                            102
test/smoosh.sh                                  220
test/smoosh_import.py                            30
test/souffle-smoke.sh                            19
test/spec-alpine.sh                             154
test/spec-bin.sh                                 41
test/spec-common.sh                              93
test/spec-cpp.sh                                367
test/spec-py.sh                                 167
test/spec-runner.sh                             375
test/spec-util.sh                                39
test/spec-version.sh                            127
test/spec.sh                                    998
test/spec_lib.py                                286
test/stateful.sh                                239
test/syscall.py                                 280
test/syscall.sh                                 508
test/tools-deps.sh                               76
test/tsv-lib-test.sh                             71
test/tsv-lib.sh                                 111
test/unit.sh                                    292
test/vagrant.sh                                  54
test/wild-runner.sh                             204
test/wild.sh                                    729
test/wild_report.py                             832
test/ysh-every-string.sh                        192
test/ysh-ify.sh                                 657
test/ysh-parse-errors.sh                      1,727
test/ysh-runtime-errors.sh                    1,165

17,953 lines in 66 files  

Test Automation

asdl/TEST.sh                                    129
core/TEST.sh                                     21
cpp/TEST.sh                                     112
data_lang/TEST.sh                                35
frontend/TEST.sh                                 22
mycpp/TEST.sh                                   430
opy/TEST.sh                                     270
pea/TEST.sh                                     411
stdlib/TEST.sh                                   64
yaks/TEST.sh                                    123

1,617 lines in 11 files  

mycpp Python Runtime

Stubs that are re-implemented in C++

mycpp/iolib.py                                  165
mycpp/mops.py                                   285
mycpp/mylib.py                                  498

948 lines in 4 files  

mycpp Translator

This prototype uses the MyPy frontend to translate statically-typed Python to C++. The generated code calls a small runtime which implements things like List[T], Dict[K, V], and Python's len().

mycpp/const_pass.py                             234
mycpp/control_flow_pass.py                      548
mycpp/conversion_pass.py                        367
mycpp/cppgen_pass.py                          2,823
mycpp/crash.py                                   38
mycpp/format_strings.py                         120
mycpp/mycpp_main.py                             261
mycpp/pass_state.py                             611
mycpp/translate.py                              218
mycpp/util.py                                   200
mycpp/visitor.py                                736

6,156 lines in 12 files  

mycpp Test Data

Small Python examples that translate to C++, compile, and run.

mycpp/examples/__init__.py                        0
mycpp/examples/asdl_generated.py                 65
mycpp/examples/cartesian.py                      78
mycpp/examples/cgi.py                            62
mycpp/examples/classes.py                       296
mycpp/examples/containers.py                    289
mycpp/examples/control_flow.py                  113
mycpp/examples/escape.py                         70
mycpp/examples/fib_iter.py                       53
mycpp/examples/fib_recursive.py                  49
mycpp/examples/files.py                          64
mycpp/examples/gc_stack_roots.py                188
mycpp/examples/invalid_condition.py              31
mycpp/examples/invalid_default_args.py           44
mycpp/examples/invalid_except.py                 38
mycpp/examples/invalid_format_args.py            15
mycpp/examples/invalid_format_strings.py         20
mycpp/examples/invalid_global.py                 18
mycpp/examples/invalid_other.py                  63
mycpp/examples/invalid_print.py                  13
mycpp/examples/invalid_python.py                 61
mycpp/examples/invalid_switch.py                 84
mycpp/examples/invalid_try_else.py               38
mycpp/examples/invalid_types.py                  31
mycpp/examples/invalid_types_2.py                31
mycpp/examples/length.py                        109
mycpp/examples/lexer_main.py                     47
mycpp/examples/loops.py                         229
mycpp/examples/modules.py                        81
mycpp/examples/parse.py                         344
mycpp/examples/pea_hello.py                      10
mycpp/examples/pgen2_demo.py                    106
mycpp/examples/scoped_resource.py               221
mycpp/examples/test_arith_ops.py                 45
mycpp/examples/test_cast.py                     211
mycpp/examples/test_classes_gc.py               212
mycpp/examples/test_conditional.py               66
mycpp/examples/test_ctx_pattern.py               65
mycpp/examples/test_default_args.py              75
mycpp/examples/test_globals.py                   62
mycpp/examples/test_hoist.py                     61
mycpp/examples/test_integers.py                  75
mycpp/examples/test_io_os_error.py              131
mycpp/examples/test_iterators.py                101
mycpp/examples/test_small_str.py                 33
mycpp/examples/test_strings.py                  277
mycpp/examples/test_switch.py                    80
mycpp/examples/tuple_return_value.py             83
mycpp/examples/varargs.py                        44

4,582 lines in 50 files  

Zephyr ASDL

A DSL for algebraic data types, borrowed from Python. Oils is the most strongly typed Bourne shell implementation!

asdl/asdl_main.py                               344
asdl/ast.py                                     270
asdl/format.py                                  157
asdl/front_end.py                               590
asdl/gen_cpp.py                                 892
asdl/gen_python.py                              649
asdl/pybase.py                                   42
asdl/runtime.py                                  68
asdl/util.py                                     11
asdl/visitor.py                                 173

3,196 lines in 11 files  

pgen2 Parser Generator

An LL(1) parser generator used to parse YSH expressions. Also borrowed from CPython.

pgen2/driver.py                                 101
pgen2/grammar.py                                303
pgen2/parse.py                                  225
pgen2/pgen.py                                   430
pgen2/pnode.py                                   67
pgen2/token.py                                  149
pgen2/tokenize.py                               579

1,854 lines in 8 files  

Other Code Generators

In order to make Oils statically typed, we had to abandon Python reflection and use C++ source code generation instead. The lexer, flag definitions, and constants can be easily compiled to C++.

core/optview_gen.py                              94
cpp/embedded_file_gen.py                         66
doctools/help_gen.py                            659
frontend/consts_gen.py                          651
frontend/flag_gen.py                            540
frontend/lexer_gen.py                           500
frontend/option_gen.py                           94
frontend/signal_gen.py                          105
osh/arith_parse_gen.py                           99
ysh/grammar_gen.py                              245

3,053 lines in 11 files  

Yaks

Experimental replacement for mycpp

yaks/gen_cpp.py                                 104
yaks/transform.py                               250
yaks/yaks_main.py                               153

507 lines in 4 files  

Hand-written C++ Code

Includes OS bindings. Small C++ files like cpp/osh_arith_parse.{cc,h} correspond to larger Python files like osh/arith_parse.py.

cpp/core.cc                                     398
cpp/core.h                                      126
cpp/data_lang.cc                                273
cpp/data_lang.h                                  34
cpp/embedded_file.h                              18
cpp/fanos.cc                                     57
cpp/fanos.h                                      18
cpp/fanos_shared.h                               49
cpp/frontend_flag_spec.cc                       231
cpp/frontend_flag_spec.h                        151
cpp/frontend_match.cc                           155
cpp/frontend_match.h                             92
cpp/frontend_pyreadline.cc                      386
cpp/frontend_pyreadline.h                        86
cpp/libc.cc                                     240
cpp/libc.h                                       38
cpp/osh.cc                                      161
cpp/osh.h                                        50
cpp/osh_tdop.cc                                  16
cpp/osh_tdop.h                                   52
cpp/pgen2.cc                                     44
cpp/pgen2.h                                      86
cpp/preamble.h                                   66
cpp/pylib.cc                                     58
cpp/pylib.h                                      22
cpp/stdlib.cc                                   258
cpp/stdlib.h                                    177
cpp/translation_stubs.h                          11

3,353 lines in 29 files  

Garbage-Collected Runtime

Uses a fork-friendly Mark-Sweep collector.

mycpp/common.h                                   92
mycpp/comparators.h                              90
mycpp/gc_alloc.h                                258
mycpp/gc_builtins.cc                            461
mycpp/gc_builtins.h                             175
mycpp/gc_dict.h                                 471
mycpp/gc_iolib.cc                                46
mycpp/gc_iolib.h                                175
mycpp/gc_list.h                                 540
mycpp/gc_mops.cc                                 79
mycpp/gc_mops.h                                 120
mycpp/gc_mylib.cc                               311
mycpp/gc_mylib.h                                385
mycpp/gc_obj.h                                  185
mycpp/gc_slab.h                                  58
mycpp/gc_str.cc                                 645
mycpp/gc_str.h                                  219
mycpp/gc_tuple.h                                151
mycpp/hash.cc                                    45
mycpp/hash.h                                     22
mycpp/mark_sweep_heap.cc                        450
mycpp/mark_sweep_heap.h                         302
mycpp/probes.h                                   39
mycpp/runtime.h                                  25
mycpp/test_common.h                              24

5,368 lines in 26 files  

Unit tests in C++

The goal is to make the spec tests pass, but unit tests are helpful too.

cpp/core_test.cc                                361
cpp/data_lang_test.cc                           178
cpp/data_race_test.cc                           206
cpp/frontend_flag_spec_test.cc                  154
cpp/frontend_match_test.cc                       87
cpp/libc_test.cc                                403
cpp/obj_layout_test.cc                          215
cpp/osh_test.cc                                  50
cpp/pgen2_test.cc                                32
cpp/pylib_test.cc                                52
cpp/stdlib_test.cc                              158
cpp/unicode_test.cc                              71
mycpp/bump_leak_heap_test.cc                     62
mycpp/float_test.cc                             150
mycpp/gc_builtins_test.cc                       447
mycpp/gc_dict_test.cc                           745
mycpp/gc_heap_test.cc                           479
mycpp/gc_iolib_test.cc                          109
mycpp/gc_list_test.cc                           569
mycpp/gc_mops_test.cc                           182
mycpp/gc_mylib_test.cc                          327
mycpp/gc_str_test.cc                          1,558
mycpp/gc_stress_test.cc                         222
mycpp/gc_tuple_test.cc                          139
mycpp/mark_sweep_heap_test.cc                   370
mycpp/small_str_test.cc                         502

7,828 lines in 27 files  

Incremental C++ Build

NINJA-config.sh                                 137
asdl/NINJA_subgraph.py                           56
bin/NINJA_subgraph.py                           146
build/ninja-rules-cpp.sh                        420
build/ninja-rules-py.sh                         390
build/ninja_lib.py                              568
build/ninja_lib_test.py                         255
build/ninja_main.py                             445
core/NINJA_subgraph.py                           58
cpp/NINJA_subgraph.py                           227
data_lang/NINJA_subgraph.py                      66
display/NINJA_subgraph.py                        16
frontend/NINJA_subgraph.py                      177
mycpp/NINJA_subgraph.py                         482
osh/NINJA_subgraph.py                            32
pea/NINJA_subgraph.py                            19
prebuilt/NINJA_subgraph.py                       79
yaks/NINJA_subgraph.py                           96
ysh/NINJA_subgraph.py                            39

3,708 lines in 20 files  

Build Automation

Makefile                                        196
build/c_module_srcs.py                           84
build/c_module_toc.py                            49
build/clean.sh                                   65
build/common.sh                                  81
build/cpp.sh                                     40
build/cpython-defs.sh                           354
build/cpython_defs.py                           520
build/deps.sh                                 1,223
build/detect-cc.c                                 4
build/detect-pwent.c                              9
build/detect-readline.c                          19
build/detect-systemtap-sdt.c                      6
build/dev-setup-test.sh                          30
build/dev-shell-test.sh                          95
build/dev-shell.sh                              155
build/dev.sh                                      5
build/doc.sh                                    749
build/dynamic-deps.sh                           234
build/dynamic_deps.py                           163
build/hello.mk                                   51
build/make_zip.py                                53
build/native.sh                                 125
build/ninja-rules-cpp.sh                        420
build/ninja-rules-py.sh                         390
build/ninja_lib.py                              568
build/ninja_main.py                             445
build/oil.mk                                     61
build/oils-preamble.sh                          120
build/old-ovm-test.sh                            42
build/ovm-actions.sh                            190
build/ovm-compile.sh                            431
build/portable-rules.mk                          96
build/py.sh                                     439
build/py2.sh                                     14
build/stamp.sh                                   76
configure                                       492
install                                         174

8,268 lines in 39 files  

Release Automation

devtools/release-history.sh                     207
devtools/release-native.sh                      142
devtools/release-note.sh                        282
devtools/release-patch.sh                        26
devtools/release-test.sh                         36
devtools/release-version.sh                   1,059
devtools/release.sh                             818

2,570 lines in 8 files  

Soil: Multi-cloud CI with containers

soil/__init__.py                                  0
soil/admin.sh                                    28
soil/collect_json.py                             35
soil/common.sh                                  107
soil/cpp-tarball.sh                              84
soil/diagnose.sh                                139
soil/github-actions.sh                          142
soil/github-tool.sh                              43
soil/history.sh                                  78
soil/host-shim.sh                               389
soil/maybe-merge.sh                             153
soil/sourcehut.sh                                81
soil/tests.sh                                    20
soil/timeout-test.sh                             50
soil/wait.sh                                    142
soil/web-init.sh                                190
soil/web-remote-test.sh                          46
soil/web-worker.sh                              416
soil/web.py                                     782
soil/web.sh                                     236
soil/web_test.py                                 52
soil/worker.sh                                  648

3,861 lines in 23 files  

Benchmarks

benchmarks/__init__.py                            0
benchmarks/auto.sh                              106
benchmarks/autoconf.sh                          603
benchmarks/awk-python.sh                         97
benchmarks/builtin-io.sh                         60
benchmarks/cachegrind.sh                         30
benchmarks/cachegrind_to_tsv.py                  54
benchmarks/callgrind.sh                          65
benchmarks/common.R                              98
benchmarks/common.sh                            108
benchmarks/compute.sh                           651
benchmarks/fake_libc.py                          14
benchmarks/gc-cachegrind.sh                      92
benchmarks/gc.sh                                760
benchmarks/gc_stats_to_tsv.py                    51
benchmarks/gperftools.sh                         60
benchmarks/id-test.sh                            41
benchmarks/id.sh                                468
benchmarks/micro.sh                              45
benchmarks/mimalloc.sh                          107
benchmarks/mycpp.sh                             117
benchmarks/osh-parser.sh                        524
benchmarks/osh-runtime.sh                       581
benchmarks/osh_runtime.py                        64
benchmarks/ovm-build.sh                         491
benchmarks/parse-help.sh                         65
benchmarks/perf.sh                              483
benchmarks/pypy.sh                               42
benchmarks/pytrace.py                            92
benchmarks/pytrace.sh                           184
benchmarks/report.R                           1,366
benchmarks/report.sh                            112
benchmarks/report_test.R                         36
benchmarks/startup.sh                           165
benchmarks/systemtap.sh                          75
benchmarks/time-test.sh                         324
benchmarks/time_.py                             266
benchmarks/tokens.sh                             24
benchmarks/uftrace.sh                           349
benchmarks/uftrace_allocs.py                    180
benchmarks/uftrace_allocs_test.py                20
benchmarks/uftrace_plugin.py                     99
benchmarks/virtual_memory.py                     59
benchmarks/vm-baseline.sh                       180
benchmarks/wedge.R                               97
benchmarks/wedge.sh                              41
benchmarks/ysh-for.sh                           110
benchmarks/ysh-json.sh                          107

9,763 lines in 49 files  

Metrics

metrics/bytecode.R                              388
metrics/bytecode.sh                             143
metrics/cpython-defs.R                           72
metrics/native-code.R                            88
metrics/native-code.sh                          194
metrics/source-code.sh                          603
metrics/tarball.sh                              154

1,642 lines in 8 files  

Generated Python Code

For the Python App Bundle.

_devbuild/gen/__init__.py                         0
_devbuild/gen/arg_types.py                      408
_devbuild/gen/arith_nt.py                         4
_devbuild/gen/demo_lib_asdl.py                  217
_devbuild/gen/expr_asdl.py                      254
_devbuild/gen/find_asdl.py                      837
_devbuild/gen/find_nt.py                         10
_devbuild/gen/grammar_nt.py                      73
_devbuild/gen/help_meta.py                      478
_devbuild/gen/hnode_asdl.py                     126
_devbuild/gen/id_kind.py                        524
_devbuild/gen/id_kind_asdl.py                   936
_devbuild/gen/mycpp_asdl.py                   1,200
_devbuild/gen/nil8_asdl.py                      294
_devbuild/gen/option_asdl.py                    416
_devbuild/gen/pretty_asdl.py                    336
_devbuild/gen/runtime_asdl.py                 1,526
_devbuild/gen/shared_variant_asdl.py            442
_devbuild/gen/syntax_asdl.py                  7,753
_devbuild/gen/typed_arith_asdl.py               441
_devbuild/gen/typed_demo_asdl.py              1,177
_devbuild/gen/types_asdl.py                     184
_devbuild/gen/value_asdl.py                   1,625
_devbuild/gen/yaks_asdl.py                    1,469

20,730 lines in 25 files  

Doc Tools

doctools/cmark.py                               519
doctools/doc_html.py                            106
doctools/fmt_check.py                            91
doctools/html_head.py                            98
doctools/html_lib.py                             48
doctools/micro_syntax.cc                      1,122
doctools/micro_syntax.re2c.h                    688
doctools/oils_doc.py                            675
doctools/ref_check.py                           182
doctools/spelling.py                            141
doctools/split_doc.py                           157
doctools/src_tree.py                            470
doctools/ul_table.py                            493
doctools/util.py                                 16
lazylex/html.py                                 508

5,314 lines in 16 files  

Web

web/ajax.js                                     149
web/osh-to-oil.js                                79
web/table/csv2html.py                           425
web/table/table-sort.js                         433

1,086 lines in 5 files