Overview of Oils Code

category total lines num files
OSH (and common libraries) 44,560 84
YSH 10,822 26
Data Languages 2,231 9
Tools 1,330 5
OSH stdlib 560 9
YSH stdlib 1,570 13
Code Borrowed from Python's stdlib 429 5
Spec Tests 57,377 190
Gold Tests 2,775 27
Python Unit Tests 11,853 77
Other Shell Tests 17,874 66
Test Automation 1,438 11
mycpp Translator 7,405 16
mycpp Test Data 4,356 48
Zephyr ASDL 3,289 11
pgen2 Parser Generator 1,854 8
Other Code Generators 3,053 11
Yaks 510 4
Hand-written C++ Code 3,336 29
Garbage-Collected Runtime 5,319 26
Unit tests in C++ 7,811 27
Incremental C++ Build 3,588 20
Build Automation 8,188 39
Release Automation 2,570 8
Soil: Multi-cloud CI with containers 3,858 23
Benchmarks 9,755 49
Metrics 1,632 8
Generated Python Code 27,802 24
Doc Tools 4,689 15
Web 1,086 5

Related Documents

The README for oilshell/oil has another overview of the repository.

OSH (and common libraries)

This is the input to the translators, written in statically-typed Python. Note that bash is at least 140K lines of code, and OSH implements a large part of bash and more.

bin/oils_for_unix.py                            201
builtin/assign_osh.py                           581
builtin/bracket_osh.py                          292
builtin/completion_osh.py                       508
builtin/dirs_osh.py                             346
builtin/io_osh.py                               182
builtin/misc_osh.py                             127
builtin/printf_osh.py                           552
builtin/process_osh.py                          617
builtin/pure_osh.py                             588
builtin/read_osh.py                             528
builtin/readline_osh.py                         269
builtin/trap_osh.py                             322
core/alloc.py                                   320
core/comp_ui.py                                 578
core/completion.py                            1,517
core/dev.py                                     767
core/error.py                                   345
core/executor.py                                859
core/main_loop.py                               413
core/num.py                                      24
core/optview.py                                  58
core/process.py                               1,973
core/pyos.py                                    290
core/pyutil.py                                  225
core/runtime.asdl                               188
core/sh_init.py                                 293
core/shell.py                                 1,223
core/state.py                                 2,805
core/util.py                                    168
core/value.asdl                                 220
core/vm.py                                      342
display/ansi.py                                  15
display/pp_value.py                             513
display/pretty.asdl                              30
display/pretty.py                               303
display/ui.py                                   602
frontend/args.py                                677
frontend/builtin_def.py                         166
frontend/consts.py                              399
frontend/flag_def.py                            566
frontend/flag_spec.py                           349
frontend/flag_util.py                            68
frontend/id_kind_def.py                         809
frontend/lexer.py                               455
frontend/lexer_def.py                         1,138
frontend/location.py                            576
frontend/match.py                               310
frontend/option_def.py                          419
frontend/parse_lib.py                           405
frontend/py_readline.py                         161
frontend/reader.py                              280
frontend/signal_def.py                           94
frontend/syntax.asdl                            651
frontend/syntax_abbrev.py                       141
frontend/typed_args.py                          684
frontend/types.asdl                              61
osh/arith_parse.py                              197
osh/bool_parse.py                               272
osh/bool_stat.py                                135
osh/braces.py                                   539
osh/cmd_eval.py                               2,306
osh/cmd_parse.py                              2,862
osh/glob_.py                                    494
osh/history.py                                  193
osh/prompt.py                                   357
osh/sh_expr_eval.py                           1,253
osh/split.py                                    311
osh/string_ops.py                               545
osh/tdop.py                                     342
osh/word_.py                                    805
osh/word_compile.py                             309
osh/word_eval.py                              2,449
osh/word_parse.py                             2,219
pyext/fanos.c                                   119
pyext/fastfunc.c                                133
pyext/fastfunc.pyi                               11
pyext/fastlex.c                                 389
pyext/fastlex.pyi                                21
pyext/libc.c                                    421
pyext/libc.pyi                                   18
pyext/line_input.pyi                             57
pyext/posix_.pyi                                210

44,560 lines in 84 files  

YSH

Expression grammar, parser, evaluator, etc.

builtin/completion_ysh.py                        69
builtin/error_ysh.py                            308
builtin/func_eggex.py                           186
builtin/func_hay.py                             157
builtin/func_misc.py                            794
builtin/func_reflect.py                         216
builtin/hay_ysh.py                              429
builtin/io_ysh.py                               301
builtin/json_ysh.py                             139
builtin/meta_oils.py                            855
builtin/method_dict.py                           95
builtin/method_io.py                            253
builtin/method_list.py                          100
builtin/method_other.py                          32
builtin/method_str.py                           587
builtin/method_type.py                          141
builtin/module_ysh.py                           141
builtin/pure_ysh.py                             228
ysh/expr_eval.py                              1,591
ysh/expr_parse.py                               387
ysh/expr_to_ast.py                            1,721
ysh/func_proc.py                                597
ysh/grammar.pgen2                               541
ysh/regex_translate.py                          393
ysh/val_ops.py                                  561

10,822 lines in 26 files  

Data Languages

JSON, J8 Notation, ...

data_lang/nil8.asdl                              31
data_lang/j8.py                               1,386
data_lang/j8_lite.py                             54
data_lang/pyj8.py                                27
data_lang/j8.h                                  314
data_lang/j8_libc.c                             221
data_lang/j8_libc.h                              28
data_lang/utf8.h                                170

2,231 lines in 9 files  

Tools

tools/deps.py                                   149
tools/fmt.py                                     25
tools/readlink.py                                31
tools/ysh_ify.py                              1,125

1,330 lines in 5 files  

OSH stdlib

stdlib/osh/bash-strict.sh                         8
stdlib/osh/byo-server-test.sh                    73
stdlib/osh/byo-server.sh                        138
stdlib/osh/no-quotes-test.sh                     82
stdlib/osh/no-quotes.sh                         137
stdlib/osh/task-five.sh                          62
stdlib/osh/two-test.sh                           33
stdlib/osh/two.sh                                27

560 lines in 9 files  

YSH stdlib

stdlib/ysh/args-test.ysh                        352
stdlib/ysh/args.ysh                             281
stdlib/ysh/def-test.ysh                          43
stdlib/ysh/def.ysh                                7
stdlib/ysh/list-test.ysh                         87
stdlib/ysh/list.ysh                              64
stdlib/ysh/math-test.ysh                         64
stdlib/ysh/math.ysh                              87
stdlib/ysh/stream.ysh                           310
stdlib/ysh/table.ysh                            150
stdlib/ysh/yblocks-test.ysh                      72
stdlib/ysh/yblocks.ysh                           53

1,570 lines in 13 files  

Code Borrowed from Python's stdlib

pylib/cgi.py                                     19
pylib/collections_.py                           214
pylib/os_path.py                                169
pylib/path_stat.py                               27

429 lines in 5 files  

Spec Tests

A comprehensive test suite that compares OSH against other shells. If OSH passes these tests in BOTH Python and C++, it means that the translation works.

spec/alias.test.sh                              571
spec/append.test.sh                             303
spec/arith-context.test.sh                      216
spec/arith-dynamic.test.sh                       95
spec/arith.test.sh                              998
spec/array-basic.test.sh                         45
spec/array-compat.test.sh                       189
spec/array.test.sh                              803
spec/assign-deferred.test.sh                    107
spec/assign-dialects.test.sh                     61
spec/assign-extended.test.sh                    881
spec/assign.test.sh                             674
spec/assoc-zsh.test.sh                           59
spec/assoc.test.sh                              797
spec/background.test.sh                         184
spec/ble-features.test.sh                       645
spec/ble-idioms.test.sh                         334
spec/blog-other1.test.sh                         57
spec/blog1.test.sh                               95
spec/blog2.test.sh                               60
spec/brace-expansion.test.sh                    502
spec/bugs.test.sh                               417
spec/builtin-bash.test.sh                       192
spec/builtin-bind.test.sh                        21
spec/builtin-bracket.test.sh                    767
spec/builtin-cd.test.sh                         333
spec/builtin-completion.test.sh                 596
spec/builtin-dirs.test.sh                       241
spec/builtin-echo.test.sh                       248
spec/builtin-eval-source.test.sh                345
spec/builtin-getopts.test.sh                    347
spec/builtin-history.test.sh                    198
spec/builtin-meta.test.sh                       620
spec/builtin-misc.test.sh                       161
spec/builtin-printf.test.sh                   1,284
spec/builtin-process.test.sh                    588
spec/builtin-read.test.sh                       770
spec/builtin-special.test.sh                    141
spec/builtin-times.test.sh                       16
spec/builtin-trap-bash.test.sh                  657
spec/builtin-trap-err.test.sh                   647
spec/builtin-trap.test.sh                       366
spec/builtin-type-bash.test.sh                  326
spec/builtin-type.test.sh                       117
spec/builtin-vars.test.sh                       707
spec/case_.test.sh                              244
spec/command-parsing.test.sh                     64
spec/command-sub.test.sh                        285
spec/command_.test.sh                           194
spec/comments.test.sh                            11
spec/dbracket.test.sh                           467
spec/dparen.test.sh                             205
spec/empty-bodies.test.sh                        24
spec/errexit-osh.test.sh                        814
spec/errexit.test.sh                            532
spec/exit-status.test.sh                        280
spec/explore-parsing.test.sh                     42
spec/extglob-files.test.sh                      384
spec/extglob-match.test.sh                      400
spec/fatal-errors.test.sh                       186
spec/for-expr.test.sh                           173
spec/func-parsing.test.sh                       106
spec/glob.test.sh                               388
spec/globignore.test.sh                         159
spec/hay-isolation.test.sh                      169
spec/hay-meta.test.sh                           257
spec/hay.test.sh                                655
spec/here-doc.test.sh                           370
spec/if_.test.sh                                 55
spec/interactive-parse.test.sh                   37
spec/interactive.test.sh                        343
spec/introspect.test.sh                         287
spec/let.test.sh                                 23
spec/loop.test.sh                               474
spec/nameref.test.sh                            637
spec/nix-idioms.test.sh                         196
spec/nocasematch-match.test.sh                   71
spec/nul-bytes.test.sh                          159
spec/osh-bugs.test.sh                            31
spec/parse-errors.test.sh                       226
spec/pipeline.test.sh                           234
spec/posix.test.sh                              153
spec/process-sub.test.sh                        204
spec/prompt.test.sh                             253
spec/quote.test.sh                              231
spec/redirect-command.test.sh                   306
spec/redirect-multi.test.sh                     308
spec/redirect.test.sh                           485
spec/regex.test.sh                              631
spec/serialize.test.sh                          230
spec/sh-func.test.sh                            152
spec/sh-options.test.sh                         879
spec/sh-usage.test.sh                            94
spec/shell-grammar.test.sh                      206
spec/smoke.test.sh                              126
spec/strict-options.test.sh                     277
spec/subshell.test.sh                            23
spec/tilde.test.sh                              184
spec/toysh-posix.test.sh                        344
spec/toysh.test.sh                              143
spec/type-compat.test.sh                        159
spec/unicode.test.sh                            196
spec/var-num.test.sh                             43
spec/var-op-bash.test.sh                        397
spec/var-op-len.test.sh                         220
spec/var-op-patsub.test.sh                      413
spec/var-op-slice.test.sh                       403
spec/var-op-strip.test.sh                       357
spec/var-op-test.test.sh                        520
spec/var-ref.test.sh                            509
spec/var-sub-quote.test.sh                      312
spec/var-sub.test.sh                             60
spec/vars-bash.test.sh                           30
spec/vars-special.test.sh                       790
spec/whitespace.test.sh                         114
spec/word-eval.test.sh                           62
spec/word-split.test.sh                         421
spec/xtrace.test.sh                             369
spec/ysh-TODO-deprecate.test.sh                 144
spec/ysh-assign.test.sh                         423
spec/ysh-augmented.test.sh                      245
spec/ysh-bin.test.sh                             20
spec/ysh-blocks.test.sh                         339
spec/ysh-bugs.test.sh                           324
spec/ysh-builtin-ctx.test.sh                    168
spec/ysh-builtin-error.test.sh                  436
spec/ysh-builtin-eval.test.sh                   699
spec/ysh-builtin-help.test.sh                   124
spec/ysh-builtin-meta.test.sh                   356
spec/ysh-builtin-module.test.sh                 489
spec/ysh-builtin-process.test.sh                 85
spec/ysh-builtin-shopt.test.sh                  183
spec/ysh-builtins.test.sh                       676
spec/ysh-case.test.sh                           245
spec/ysh-closures.test.sh                       179
spec/ysh-cmd-lang.test.sh                        54
spec/ysh-command-sub.test.sh                    147
spec/ysh-completion.test.sh                     204
spec/ysh-control-flow.test.sh                   182
spec/ysh-convert.test.sh                        190
spec/ysh-demo.test.sh                            59
spec/ysh-dev.test.sh                            163
spec/ysh-dict.test.sh                           120
spec/ysh-env.test.sh                            309
spec/ysh-expr-arith.test.sh                     614
spec/ysh-expr-bool.test.sh                      185
spec/ysh-expr-compare.test.sh                   427
spec/ysh-expr-sub.test.sh                        42
spec/ysh-expr.test.sh                           442
spec/ysh-for.test.sh                            192
spec/ysh-func-builtin.test.sh                   250
spec/ysh-func.test.sh                           496
spec/ysh-funcs-external.test.sh                  86
spec/ysh-int-float.test.sh                      220
spec/ysh-interactive.test.sh                     41
spec/ysh-json.test.sh                         1,251
spec/ysh-keywords.test.sh                        37
spec/ysh-list.test.sh                           179
spec/ysh-method-io.test.sh                       86
spec/ysh-methods.test.sh                        595
spec/ysh-multiline.test.sh                      179
spec/ysh-namespaces.test.sh                      75
spec/ysh-object.test.sh                         296
spec/ysh-options-assign.test.sh                 106
spec/ysh-options.test.sh                        764
spec/ysh-place.test.sh                          136
spec/ysh-printing.test.sh                       391
spec/ysh-proc-meta.test.sh                      241
spec/ysh-proc.test.sh                           814
spec/ysh-prompt.test.sh                         147
spec/ysh-regex-api.test.sh                      939
spec/ysh-regex.test.sh                          711
spec/ysh-reserved.test.sh                        48
spec/ysh-scope.test.sh                          768
spec/ysh-slice-range.test.sh                    278
spec/ysh-source.test.sh                          29
spec/ysh-special-vars.test.sh                    50
spec/ysh-stdlib-args.test.sh                     37
spec/ysh-stdlib.test.sh                          25
spec/ysh-string.test.sh                         593
spec/ysh-tuple.test.sh                           28
spec/ysh-unicode.test.sh                        184
spec/ysh-usage.test.sh                           78
spec/ysh-user-feedback.test.sh                  141
spec/ysh-var-sub.test.sh                         45
spec/ysh-with-sh.test.sh                        221
spec/ysh-word-eval.test.sh                      144
spec/ysh-xtrace.test.sh                         591
spec/zsh-idioms.test.sh                          45

57,377 lines in 190 files  

Gold Tests

Another suite that tests shells "from the outside". Instead of making explicit assertions, we verify that OSH behaves like bash.

test/gold/abuild.sh                               9
test/gold/and-or.sh                              27
test/gold/autoconf-backtick.sh                   50
test/gold/big-here-doc.sh                     1,531
test/gold/case-in-subshell.sh                    37
test/gold/char-class.sh                          33
test/gold/command-sub-2.sh                       12
test/gold/command-sub.sh                         39
test/gold/comments.sh                            22
test/gold/complex-here-docs.sh                  192
test/gold/configure-bug.sh                       13
test/gold/de1.sh                                 24
test/gold/declare.sh                             28
test/gold/dollar-sq.sh                           42
test/gold/echo-e.sh                              36
test/gold/errexit-confusion.sh                  139
test/gold/errexit.sh                            121
test/gold/export.sh                              65
test/gold/glob.sh                                26
test/gold/lineno.sh                              23
test/gold/nix.sh                                 32
test/gold/readlink.sh                            61
test/gold/readonly.sh                            24
test/gold/scope.sh                              126
test/gold/strip-op-char-class.sh                  6
test/gold/word-eval.sh                           57

2,775 lines in 27 files  

Python Unit Tests

asdl/examples/typed_arith_parse_test.py         227
asdl/format_test.py                              46
asdl/front_end_test.py                          184
asdl/gen_python_test.py                         180
benchmarks/uftrace_allocs_test.py                20
build/dynamic_deps_test.py                       23
build/ninja_lib_test.py                         255
builtin/bracket_osh_test.py                      26
builtin/completion_osh_test.py                   16
builtin/func_misc_test.py                        27
builtin/misc_osh_test.py                         28
builtin/read_osh_test.py                         35
builtin/readline_osh_test.py                    100
client/py_fanos_test.py                         108
core/alloc_test.py                               54
core/comp_ui_test.py                            177
core/completion_test.py                         827
core/process_test.py                            277
core/pyutil_test.py                              17
core/state_test.py                              353
core/util_test.py                                64
data_lang/j8_test.py                            229
data_lang/pyj8_test.py                           52
display/pretty_test.py                          130
display/ui_test.py                               29
doctools/cmark_test.py                          203
doctools/help_gen_test.py                       116
doctools/html_lib_test.py                        49
doctools/oils_doc_test.py                        79
doctools/spelling_test.py                        49
doctools/split_doc_test.py                      104
frontend/args_test.py                           237
frontend/id_kind_def_test.py                     97
frontend/lexer_def_test.py                      454
frontend/lexer_gen_test.py                       53
frontend/lexer_test.py                          216
frontend/match_test.py                          131
frontend/reader_test.py                          74
frontend/typed_args_test.py                     158
lazylex/html_test.py                             78
mycpp/format_strings_test.py                     43
mycpp/mops_test.py                               32
mycpp/mylib_test.py                              56
mycpp/pass_state_test.py                        596
osh/arith_parse_test.py                         185
osh/bool_parse_test.py                          122
osh/braces_test.py                              186
osh/cmd_eval_test.py                             81
osh/cmd_parse_test.py                         1,493
osh/glob_test.py                                160
osh/history_test.py                             126
osh/prompt_test.py                               60
osh/sh_expr_eval_test.py                        102
osh/split_test.py                               164
osh/string_ops_test.py                          203
osh/word_compile_test.py                         16
osh/word_eval_test.py                           157
osh/word_parse_test.py                          607
osh/word_test.py                                176
pyext/fanos_test.py                             274
pyext/fastfunc_test.py                           83
pyext/fastlex_test.py                            85
pyext/libc_test.py                              310
pyext/line_input_test.py                         27
pyext/posix_test.py                             184
pylib/os_path_test.py                            59
pylib/path_stat_test.py                          24
soil/web_test.py                                 52
spec/stateful/harness_test.py                    46
test/sh_spec_test.py                            184
test/wild_report_test.py                         28
tools/fmt_test.py                                18
tools/ysh_ify_test.py                            78
ysh/expr_parse_test.py                          163
ysh/regex_translate_test.py                      51
ysh/val_ops_test.py                              40

11,853 lines in 77 files  

Other Shell Tests

test/all.sh                                      92
test/alpine.sh                                  307
test/asan.sh                                     41
test/ble.sh                                     139
test/bug-1986.sh                                122
test/bug-2123.sh                                 22
test/bugs.sh                                    223
test/common-test.sh                              22
test/common.sh                                  271
test/coverage.sh                                180
test/cpp-unit.sh                                 45
test/fedora.sh                                   22
test/gentoo.sh                                   83
test/gold.sh                                    168
test/lint.sh                                    312
test/lossless.sh                                148
test/ltrace.sh                                   49
test/nix.sh                                      24
test/nohup.sh                                    89
test/opyc.sh                                    118
test/osh-usage.sh                               222
test/other.sh                                    34
test/parse-err-compare.sh                       138
test/parse-errors.sh                            833
test/process-table-portable.sh                  164
test/process-table.sh                           171
test/process_table.py                           224
test/py2_lint.py                                105
test/py3_lint.py                                 18
test/py_stdlib.sh                                32
test/report.R                                    54
test/runtime-errors.sh                        1,286
test/sh-assert.sh                               164
test/sh_spec.py                               1,482
test/shebang.sh                                  26
test/shell-vs-shell.sh                          314
test/signal-state.sh                            102
test/smoosh.sh                                  220
test/smoosh_import.py                            30
test/souffle-smoke.sh                            19
test/spec-alpine.sh                             154
test/spec-bin.sh                                 41
test/spec-common.sh                              93
test/spec-cpp.sh                                363
test/spec-py.sh                                 167
test/spec-runner.sh                             375
test/spec-util.sh                                39
test/spec-version.sh                            127
test/spec.sh                                    994
test/spec_lib.py                                286
test/stateful.sh                                239
test/syscall.py                                 280
test/syscall.sh                                 508
test/tools-deps.sh                               76
test/tsv-lib-test.sh                             71
test/tsv-lib.sh                                 111
test/unit.sh                                    292
test/vagrant.sh                                  54
test/wild-runner.sh                             204
test/wild.sh                                    729
test/wild_report.py                             832
test/ysh-every-string.sh                        192
test/ysh-ify.sh                                 657
test/ysh-parse-errors.sh                      1,727
test/ysh-runtime-errors.sh                    1,148

17,874 lines in 66 files  

Test Automation

asdl/TEST.sh                                     86
core/TEST.sh                                     21
cpp/TEST.sh                                     112
data_lang/TEST.sh                                35
frontend/TEST.sh                                 22
mycpp/TEST.sh                                   361
opy/TEST.sh                                     270
pea/TEST.sh                                     344
stdlib/TEST.sh                                   64
yaks/TEST.sh                                    123

1,438 lines in 11 files  

mycpp Translator

This prototype uses the MyPy frontend to translate statically-typed Python to C++. The generated code calls a small runtime which implements things like List[T], Dict[K, V], and Python's len().

mycpp/compare_pairs.py                           60
mycpp/const_pass.py                             532
mycpp/control_flow_pass.py                      559
mycpp/cppgen_pass.py                          3,107
mycpp/crash.py                                   38
mycpp/debug_pass.py                             465
mycpp/format_strings.py                         114
mycpp/iolib.py                                  165
mycpp/ir_pass.py                                 99
mycpp/mops.py                                   285
mycpp/mycpp_main.py                             423
mycpp/mylib.py                                  533
mycpp/pass_state.py                             610
mycpp/util.py                                   177
mycpp/visitor.py                                238

7,405 lines in 16 files  

mycpp Test Data

Small Python examples that translate to C++, compile, and run.

mycpp/examples/__init__.py                        0
mycpp/examples/asdl_generated.py                 65
mycpp/examples/cartesian.py                      78
mycpp/examples/cgi.py                            62
mycpp/examples/classes.py                       298
mycpp/examples/containers.py                    289
mycpp/examples/control_flow.py                  113
mycpp/examples/escape.py                         70
mycpp/examples/fib_iter.py                       53
mycpp/examples/fib_recursive.py                  49
mycpp/examples/files.py                          64
mycpp/examples/gc_stack_roots.py                187
mycpp/examples/invalid_condition.py              31
mycpp/examples/invalid_default_args.py           44
mycpp/examples/invalid_except.py                 38
mycpp/examples/invalid_format_args.py            15
mycpp/examples/invalid_format_strings.py         20
mycpp/examples/invalid_global.py                 18
mycpp/examples/invalid_print.py                  13
mycpp/examples/invalid_python.py                 61
mycpp/examples/invalid_switch.py                 84
mycpp/examples/invalid_try_else.py               38
mycpp/examples/invalid_types.py                  31
mycpp/examples/invalid_types_2.py                31
mycpp/examples/length.py                        109
mycpp/examples/lexer_main.py                     47
mycpp/examples/loops.py                         177
mycpp/examples/modules.py                        81
mycpp/examples/parse.py                         271
mycpp/examples/pgen2_demo.py                    106
mycpp/examples/scoped_resource.py               221
mycpp/examples/test_arith_ops.py                 45
mycpp/examples/test_cast.py                     211
mycpp/examples/test_classes_gc.py               212
mycpp/examples/test_conditional.py               66
mycpp/examples/test_ctx_pattern.py               65
mycpp/examples/test_default_args.py              75
mycpp/examples/test_globals.py                   62
mycpp/examples/test_hoist.py                     61
mycpp/examples/test_integers.py                  75
mycpp/examples/test_io_os_error.py              131
mycpp/examples/test_iterators.py                101
mycpp/examples/test_small_str.py                 33
mycpp/examples/test_strings.py                  272
mycpp/examples/test_switch.py                    80
mycpp/examples/tuple_return_value.py             54
mycpp/examples/varargs.py                        49

4,356 lines in 48 files  

Zephyr ASDL

A DSL for algebraic data types, borrowed from Python. Oils is the most strongly typed Bourne shell implementation!

asdl/asdl_main.py                               299
asdl/ast.py                                     238
asdl/format.py                                  535
asdl/front_end.py                               515
asdl/gen_cpp.py                                 826
asdl/gen_python.py                              611
asdl/pybase.py                                   51
asdl/runtime.py                                  67
asdl/util.py                                     11
asdl/visitor.py                                 136

3,289 lines in 11 files  

pgen2 Parser Generator

An LL(1) parser generator used to parse YSH expressions. Also borrowed from CPython.

pgen2/driver.py                                 101
pgen2/grammar.py                                303
pgen2/parse.py                                  225
pgen2/pgen.py                                   430
pgen2/pnode.py                                   67
pgen2/token.py                                  149
pgen2/tokenize.py                               579

1,854 lines in 8 files  

Other Code Generators

In order to make Oils statically typed, we had to abandon Python reflection and use C++ source code generation instead. The lexer, flag definitions, and constants can be easily compiled to C++.

core/optview_gen.py                              94
cpp/embedded_file_gen.py                         66
doctools/help_gen.py                            659
frontend/consts_gen.py                          651
frontend/flag_gen.py                            540
frontend/lexer_gen.py                           500
frontend/option_gen.py                           94
frontend/signal_gen.py                          105
osh/arith_parse_gen.py                           99
ysh/grammar_gen.py                              245

3,053 lines in 11 files  

Yaks

Experimental replacement for mycpp

yaks/gen_cpp.py                                 104
yaks/transform.py                               250
yaks/yaks_main.py                               156

510 lines in 4 files  

Hand-written C++ Code

Includes OS bindings. Small C++ files like cpp/osh_arith_parse.{cc,h} correspond to larger Python files like osh/arith_parse.py.

cpp/core.cc                                     398
cpp/core.h                                      126
cpp/data_lang.cc                                273
cpp/data_lang.h                                  34
cpp/embedded_file.h                              18
cpp/fanos.cc                                     57
cpp/fanos.h                                      18
cpp/fanos_shared.h                               49
cpp/frontend_flag_spec.cc                       231
cpp/frontend_flag_spec.h                        151
cpp/frontend_match.cc                           155
cpp/frontend_match.h                             92
cpp/frontend_pyreadline.cc                      386
cpp/frontend_pyreadline.h                        86
cpp/libc.cc                                     241
cpp/libc.h                                       38
cpp/osh.cc                                      161
cpp/osh.h                                        50
cpp/osh_tdop.cc                                  16
cpp/osh_tdop.h                                   52
cpp/pgen2.cc                                     44
cpp/pgen2.h                                      86
cpp/preamble.h                                   48
cpp/pylib.cc                                     58
cpp/pylib.h                                      22
cpp/stdlib.cc                                   258
cpp/stdlib.h                                    177
cpp/translation_stubs.h                          11

3,336 lines in 29 files  

Garbage-Collected Runtime

Uses a fork-friendly Mark-Sweep collector.

mycpp/common.h                                   92
mycpp/comparators.h                              90
mycpp/gc_alloc.h                                258
mycpp/gc_builtins.cc                            461
mycpp/gc_builtins.h                             175
mycpp/gc_dict.h                                 471
mycpp/gc_iolib.cc                                46
mycpp/gc_iolib.h                                175
mycpp/gc_list.h                                 521
mycpp/gc_mops.cc                                 79
mycpp/gc_mops.h                                 120
mycpp/gc_mylib.cc                               311
mycpp/gc_mylib.h                                381
mycpp/gc_obj.h                                  179
mycpp/gc_slab.h                                  58
mycpp/gc_str.cc                                 645
mycpp/gc_str.h                                  219
mycpp/gc_tuple.h                                151
mycpp/hash.cc                                    45
mycpp/hash.h                                     22
mycpp/mark_sweep_heap.cc                        438
mycpp/mark_sweep_heap.h                         294
mycpp/probes.h                                   39
mycpp/runtime.h                                  25
mycpp/test_common.h                              24

5,319 lines in 26 files  

Unit tests in C++

The goal is to make the spec tests pass, but unit tests are helpful too.

cpp/core_test.cc                                361
cpp/data_lang_test.cc                           178
cpp/data_race_test.cc                           206
cpp/frontend_flag_spec_test.cc                  154
cpp/frontend_match_test.cc                       87
cpp/libc_test.cc                                403
cpp/obj_layout_test.cc                          215
cpp/osh_test.cc                                  50
cpp/pgen2_test.cc                                32
cpp/pylib_test.cc                                52
cpp/stdlib_test.cc                              158
cpp/unicode_test.cc                              71
mycpp/bump_leak_heap_test.cc                     62
mycpp/float_test.cc                             150
mycpp/gc_builtins_test.cc                       447
mycpp/gc_dict_test.cc                           745
mycpp/gc_heap_test.cc                           479
mycpp/gc_iolib_test.cc                          109
mycpp/gc_list_test.cc                           561
mycpp/gc_mops_test.cc                           173
mycpp/gc_mylib_test.cc                          327
mycpp/gc_str_test.cc                          1,558
mycpp/gc_stress_test.cc                         222
mycpp/gc_tuple_test.cc                          139
mycpp/mark_sweep_heap_test.cc                   370
mycpp/small_str_test.cc                         502

7,811 lines in 27 files  

Incremental C++ Build

NINJA-config.sh                                 137
asdl/NINJA_subgraph.py                           56
bin/NINJA_subgraph.py                           141
build/ninja-rules-cpp.sh                        390
build/ninja-rules-py.sh                         364
build/ninja_lib.py                              555
build/ninja_lib_test.py                         255
build/ninja_main.py                             440
core/NINJA_subgraph.py                           58
cpp/NINJA_subgraph.py                           227
data_lang/NINJA_subgraph.py                      66
display/NINJA_subgraph.py                        16
frontend/NINJA_subgraph.py                      175
mycpp/NINJA_subgraph.py                         446
osh/NINJA_subgraph.py                            32
pea/NINJA_subgraph.py                            19
prebuilt/NINJA_subgraph.py                       79
yaks/NINJA_subgraph.py                           93
ysh/NINJA_subgraph.py                            39

3,588 lines in 20 files  

Build Automation

Makefile                                        196
build/c_module_srcs.py                           98
build/c_module_toc.py                            51
build/clean.sh                                   65
build/common.sh                                  81
build/cpp.sh                                     40
build/cpython-defs.sh                           354
build/cpython_defs.py                           520
build/deps.sh                                 1,213
build/detect-cc.c                                 4
build/detect-pwent.c                              9
build/detect-readline.c                          19
build/detect-systemtap-sdt.c                      6
build/dev-setup-test.sh                          30
build/dev-shell-test.sh                          95
build/dev-shell.sh                              155
build/dev.sh                                      5
build/doc.sh                                    748
build/dynamic-deps.sh                           234
build/dynamic_deps.py                           155
build/hello.mk                                   51
build/make_zip.py                                53
build/native.sh                                 125
build/ninja-rules-cpp.sh                        390
build/ninja-rules-py.sh                         364
build/ninja_lib.py                              555
build/ninja_main.py                             440
build/oil.mk                                     61
build/oils-preamble.sh                          120
build/old-ovm-test.sh                            42
build/ovm-actions.sh                            190
build/ovm-compile.sh                            430
build/portable-rules.mk                          96
build/py.sh                                     437
build/py2.sh                                     14
build/stamp.sh                                   76
configure                                       492
install                                         174

8,188 lines in 39 files  

Release Automation

devtools/release-history.sh                     207
devtools/release-native.sh                      142
devtools/release-note.sh                        282
devtools/release-patch.sh                        26
devtools/release-test.sh                         36
devtools/release-version.sh                   1,059
devtools/release.sh                             818

2,570 lines in 8 files  

Soil: Multi-cloud CI with containers

soil/__init__.py                                  0
soil/admin.sh                                    28
soil/collect_json.py                             35
soil/common.sh                                  107
soil/cpp-tarball.sh                              84
soil/diagnose.sh                                139
soil/github-actions.sh                          142
soil/github-tool.sh                              43
soil/history.sh                                  78
soil/host-shim.sh                               389
soil/maybe-merge.sh                             153
soil/sourcehut.sh                                81
soil/tests.sh                                    20
soil/timeout-test.sh                             50
soil/wait.sh                                    142
soil/web-init.sh                                190
soil/web-remote-test.sh                          46
soil/web-worker.sh                              416
soil/web.py                                     782
soil/web.sh                                     236
soil/web_test.py                                 52
soil/worker.sh                                  645

3,858 lines in 23 files  

Benchmarks

benchmarks/__init__.py                            0
benchmarks/auto.sh                              106
benchmarks/autoconf.sh                          603
benchmarks/awk-python.sh                         97
benchmarks/builtin-io.sh                         60
benchmarks/cachegrind.sh                         30
benchmarks/cachegrind_to_tsv.py                  54
benchmarks/callgrind.sh                          56
benchmarks/common.R                              98
benchmarks/common.sh                            108
benchmarks/compute.sh                           651
benchmarks/fake_libc.py                          14
benchmarks/gc-cachegrind.sh                      92
benchmarks/gc.sh                                760
benchmarks/gc_stats_to_tsv.py                    51
benchmarks/gperftools.sh                         60
benchmarks/id-test.sh                            41
benchmarks/id.sh                                468
benchmarks/micro.sh                              45
benchmarks/mimalloc.sh                          107
benchmarks/mycpp.sh                             117
benchmarks/osh-parser.sh                        524
benchmarks/osh-runtime.sh                       581
benchmarks/osh_runtime.py                        64
benchmarks/ovm-build.sh                         491
benchmarks/parse-help.sh                         65
benchmarks/perf.sh                              483
benchmarks/pypy.sh                               42
benchmarks/pytrace.py                            92
benchmarks/pytrace.sh                           184
benchmarks/report.R                           1,366
benchmarks/report.sh                            112
benchmarks/report_test.R                         36
benchmarks/startup.sh                           165
benchmarks/systemtap.sh                          75
benchmarks/time-test.sh                         324
benchmarks/time_.py                             267
benchmarks/tokens.sh                             24
benchmarks/uftrace.sh                           349
benchmarks/uftrace_allocs.py                    180
benchmarks/uftrace_allocs_test.py                20
benchmarks/uftrace_plugin.py                     99
benchmarks/virtual_memory.py                     59
benchmarks/vm-baseline.sh                       180
benchmarks/wedge.R                               97
benchmarks/wedge.sh                              41
benchmarks/ysh-for.sh                           110
benchmarks/ysh-json.sh                          107

9,755 lines in 49 files  

Metrics

metrics/bytecode.R                              388
metrics/bytecode.sh                             143
metrics/cpython-defs.R                           72
metrics/native-code.R                            88
metrics/native-code.sh                          194
metrics/source-code.sh                          593
metrics/tarball.sh                              154

1,632 lines in 8 files  

Generated Python Code

For the Python App Bundle.

_devbuild/gen/__init__.py                         0
_devbuild/gen/arg_types.py                      410
_devbuild/gen/arith_nt.py                         4
_devbuild/gen/demo_lib_asdl.py                  277
_devbuild/gen/expr_asdl.py                      219
_devbuild/gen/find_asdl.py                    1,250
_devbuild/gen/find_nt.py                         10
_devbuild/gen/grammar_nt.py                      73
_devbuild/gen/help_meta.py                      476
_devbuild/gen/hnode_asdl.py                     132
_devbuild/gen/id_kind.py                        524
_devbuild/gen/id_kind_asdl.py                   928
_devbuild/gen/nil8_asdl.py                      450
_devbuild/gen/option_asdl.py                    408
_devbuild/gen/pretty_asdl.py                    582
_devbuild/gen/runtime_asdl.py                 2,140
_devbuild/gen/shared_variant_asdl.py            651
_devbuild/gen/syntax_asdl.py                 12,194
_devbuild/gen/typed_arith_asdl.py               706
_devbuild/gen/typed_demo_asdl.py              1,402
_devbuild/gen/types_asdl.py                     161
_devbuild/gen/value_asdl.py                   2,546
_devbuild/gen/yaks_asdl.py                    2,259

27,802 lines in 24 files  

Doc Tools

doctools/cmark.py                               498
doctools/doc_html.py                            106
doctools/fmt_check.py                            91
doctools/html_head.py                            98
doctools/html_lib.py                             48
doctools/micro_syntax.cc                      1,122
doctools/micro_syntax.re2c.h                    688
doctools/oils_doc.py                            671
doctools/ref_check.py                           182
doctools/spelling.py                            141
doctools/split_doc.py                           157
doctools/src_tree.py                            470
doctools/util.py                                 11
lazylex/html.py                                 406

4,689 lines in 15 files  

Web

web/ajax.js                                     149
web/osh-to-oil.js                                79
web/table/csv2html.py                           425
web/table/table-sort.js                         433

1,086 lines in 5 files