| 1 | ---
|
| 2 | in_progress: true
|
| 3 | ---
|
| 4 |
|
| 5 | TESTING
|
| 6 |
|
| 7 | Notes on Oils Architecture
|
| 8 | ===========================
|
| 9 |
|
| 10 | This doc is for contributors or users who want to understand the Oils codebase.
|
| 11 | These internal details are subject to change.
|
| 12 |
|
| 13 | <div id="toc">
|
| 14 | </div>
|
| 15 |
|
| 16 | ## Links
|
| 17 |
|
| 18 | - [Contributing][] (wiki) helps you change the code for the first time.
|
| 19 | - [README](oils-repo/README.html) describes how the code is organized.
|
| 20 | - [Interpreter State](interpreter-state.html) describes the interpreter's user-facing data
|
| 21 | structures.
|
| 22 | - [Parser Architecture](parser-architecture.html)
|
| 23 | - [OSH Word Evaluation Algorithm][word-eval] (wiki) describes shell's complex
|
| 24 | word evaluation. Oils uses [Simple Word Evaluation](simple-word-eval.html)
|
| 25 | instead.
|
| 26 |
|
| 27 | [Contributing]: https://github.com/oils-for-unix/oils/wiki/Contributing
|
| 28 | [word-eval]: https://github.com/oils-for-unix/oils/wiki/OSH-Word-Evaluation-Algorithm
|
| 29 |
|
| 30 | ## Source Code
|
| 31 |
|
| 32 | ### Build Dependencies
|
| 33 |
|
| 34 | - Essential: [libc]($xref)
|
| 35 | - Optional: GNU [readline]($xref) (TODO: other line editing libraries).
|
| 36 | - Only in the OVM build (as of March 2020): [yajl]($xref)
|
| 37 |
|
| 38 | ### Borrowed Code
|
| 39 |
|
| 40 | - [ASDL]($oils-src:asdl/) front end from [CPython]($xref:cpython) (heavily
|
| 41 | refactored)
|
| 42 | - [frontend/tdop.py]($oils-src): Adapted from tinypy, but almost no original code
|
| 43 | remains
|
| 44 | - [pgen2]($oils-src:pgen2/)
|
| 45 | - All of OPy (will be obsolete)
|
| 46 | - compiler2 from stdlib
|
| 47 | - byterun
|
| 48 | - Build Dependency: [MyPy]($xref:mypy)
|
| 49 |
|
| 50 | ### Metaprogramming / Generated Code
|
| 51 |
|
| 52 | - Try `ls */*_def.py */*_gen.py`
|
| 53 | - The `def.py` files are abstract definitions. They're not translated by
|
| 54 | [mycpp]($xref).
|
| 55 | - The `gen.py` files generate source code in Python and C++ from these
|
| 56 | definitions.
|
| 57 | - For example, we define the core `Id` type and the lexing rules abstractly.
|
| 58 | - *TODO: Details on each `def` / `gen` pair*.
|
| 59 | - See [build/dev.sh]($oils-src) and [build/codegen.sh]($oils-src)
|
| 60 |
|
| 61 |
|
| 62 | ## Other Cross-Cutting Observations
|
| 63 |
|
| 64 | ### Where $IFS is Used
|
| 65 |
|
| 66 | - Splitting of unquoted substitutions
|
| 67 | - The [read]($help) builtin
|
| 68 | - To split words in `compgen -W` (bash only)
|
| 69 |
|
| 70 | ### Shell Function Callbacks
|
| 71 |
|
| 72 | - Completion hooks registered by `complete -F ls_complete_func ls`
|
| 73 | - bash has a `command_not_found` hook; OSH doesn't yet
|
| 74 |
|
| 75 | ### Where Unicode is Respected
|
| 76 |
|
| 77 | See the doc on [Unicode](unicode.html).
|
| 78 |
|
| 79 | ### Parse-time and Runtime Pairs
|
| 80 |
|
| 81 | In OSH:
|
| 82 |
|
| 83 | - `echo -e '\x00\n'` and `echo $'\x00\n'` (OSH shares lexer rules between them)
|
| 84 | - `test` / `[` and `[[` (OSH shares the parser and evaluator)
|
| 85 | - Static vs. Dynamic Assignment. `local x=$y` vs. `s='x=$y'; local $s`.
|
| 86 | - All shells have both notions!
|
| 87 |
|
| 88 | Other Pairs:
|
| 89 |
|
| 90 | - `expr` and `$(( ))` (`expr` not in shell)
|
| 91 | - Later:
|
| 92 | - [printf]($help) can have a static variant like `${myfloat %.3f}`
|
| 93 | - `find` and our own language (although this may be done with blocks)
|
| 94 |
|
| 95 | ## State Machines
|
| 96 |
|
| 97 | - `$IFS` splitting in `osh/split.py`
|
| 98 | - [compadjust]($help) needs to split partial `argv` by user-defined delimiters,
|
| 99 | e.g. `:=`
|
| 100 |
|
| 101 | The point of a state machine is to make sure all cases are handled!
|
| 102 |
|
| 103 | <!--
|
| 104 | Idea:
|
| 105 | - Model the prompt state and completion as a state machine (?)
|
| 106 | - vtparse is another good example
|
| 107 | -->
|
| 108 |
|
| 109 | ## Error Locations and Fallbacks
|
| 110 |
|
| 111 | - `ExecuteAndCatch` uses mem.CurrentLocation()
|
| 112 | - `Failglob` uses mem.CurrentLocation()
|
| 113 | - `mem.GetVar $LINENO` uses `current_tok`, but it can be buggy
|
| 114 | - this is difference than `BASH_LINENO`, which is for the call stack!
|
| 115 |
|
| 116 | Other:
|
| 117 |
|
| 118 | - `ui.ctx_Location`
|
| 119 | - So builtins can call `errfmt.Print_()` without locations
|
| 120 | - `alloc.ctx_SourceCode` for assigning `source_t` to tokens
|
| 121 |
|
| 122 | ## Other Topics
|
| 123 |
|
| 124 | - [Dependency Injection]($xref:dependency-injection)
|
| 125 |
|