| 1 | ---
|
| 2 | in_progress: yes
|
| 3 | ---
|
| 4 |
|
| 5 | The Unix Shell Process Model - When Are Processes Created?
|
| 6 | =============
|
| 7 |
|
| 8 | OSH and YSH are both extensions of POSIX shell, and share its underlying "process model".
|
| 9 |
|
| 10 | Each Unix process has its **own** memory, that is not shared with other
|
| 11 | processes. (It's created by `fork()`, which means that the memory is
|
| 12 | "copy-on-write".)
|
| 13 |
|
| 14 | Understanding when a shell starts processes will make you a better shell
|
| 15 | programmer.
|
| 16 |
|
| 17 | As a concrete example, here is some code that behaves differently in
|
| 18 | [bash]($xref) and [zsh]($xref):
|
| 19 |
|
| 20 |
|
| 21 | $ bash -c 'echo hi | read x; echo x=$x'
|
| 22 | x=
|
| 23 |
|
| 24 | $ zsh -c 'echo hi | read x; echo x=$x'
|
| 25 | x=hi
|
| 26 |
|
| 27 | If you understand why they are different, then that means you understand the
|
| 28 | process model!
|
| 29 |
|
| 30 | (OSH behaves like zsh.)
|
| 31 |
|
| 32 | ---
|
| 33 |
|
| 34 | Related: [Interpreter State](interpreter-state.html). These two docs are the
|
| 35 | missing documentation for shell!
|
| 36 |
|
| 37 | <div id="toc">
|
| 38 | </div>
|
| 39 |
|
| 40 | ## Shell Constructs That Start Processes
|
| 41 |
|
| 42 | ### Pipelines `myproc | wc -l`
|
| 43 |
|
| 44 | - `shopt -s lastpipe`
|
| 45 | - `set -o pipefail`
|
| 46 |
|
| 47 | Note that functions Can Be Transparently Put in Pipelines:
|
| 48 |
|
| 49 | Hidden subshell:
|
| 50 |
|
| 51 | { echo 1; echo 2; } | wc -l
|
| 52 |
|
| 53 | A `SubProgramThunk` is started for the LHS of `|`.
|
| 54 |
|
| 55 | ### Command Sub `d=$(date)`
|
| 56 |
|
| 57 | d=$(date)
|
| 58 |
|
| 59 | ### Process Sub `<(sort left.txt)`
|
| 60 |
|
| 61 | diff -u <(sort left.txt) <(sort right.txt)
|
| 62 |
|
| 63 | ### Async - `fork` or `sleep 2 &`
|
| 64 |
|
| 65 | ### Explicit Subshell - `forkwait` or `( echo hi )`
|
| 66 |
|
| 67 | Explicit Subshells are Rarely Needed.
|
| 68 |
|
| 69 | - prefer `pushd` / `popd`, or `cd { }` in YSH.
|
| 70 |
|
| 71 |
|
| 72 | ## FAQ: "Subshells By Surprise"
|
| 73 |
|
| 74 | Sometimes subshells have no syntax.
|
| 75 |
|
| 76 | Common issues:
|
| 77 |
|
| 78 | ### shopt -s lastpipe
|
| 79 |
|
| 80 | Mentioned in the intro:
|
| 81 |
|
| 82 | $ bash -c 'echo hi | read x; echo x=$x'
|
| 83 | x=
|
| 84 |
|
| 85 | $ zsh -c 'echo hi | read x; echo x=$x'
|
| 86 | x=hi
|
| 87 |
|
| 88 | ### Other Pipelines
|
| 89 |
|
| 90 | myproc (&p) | grep foo
|
| 91 |
|
| 92 | ## Process Optimizations - `noforklast`
|
| 93 |
|
| 94 | Why does a Unix shell start processes? How many processes are started?
|
| 95 |
|
| 96 | Bugs / issues
|
| 97 |
|
| 98 | - job control:
|
| 99 | - restoring process state after the shell runs
|
| 100 | - `sh -i -c 'echo hi'`
|
| 101 | - traps
|
| 102 | - not run - issue #1853
|
| 103 | - Bug with `set -o pipefail`
|
| 104 | - likewise we have to disable process optimizations for `! false` and
|
| 105 | `! false | true`
|
| 106 |
|
| 107 | Oils/YSH specific:
|
| 108 |
|
| 109 | - `shopt -s verbose_errexit`
|
| 110 | - crash dump
|
| 111 | - because we don't get to test if it failed
|
| 112 | - stats / tracing - counting exit codes
|
| 113 |
|
| 114 |
|
| 115 | ## Process State
|
| 116 |
|
| 117 | ### Redirects
|
| 118 |
|
| 119 | ## Builtins
|
| 120 |
|
| 121 | ### [wait]($help)
|
| 122 |
|
| 123 | ### [fg]($help)
|
| 124 |
|
| 125 | ### [bg]($help)
|
| 126 |
|
| 127 | ### [trap]($help)
|
| 128 |
|
| 129 |
|
| 130 | ## Appendix: Non-Shell Tools
|
| 131 |
|
| 132 | These Unix tools start processes:
|
| 133 |
|
| 134 | - `xargs`
|
| 135 | - `xargs -P` starts parallel processes (but doesn't buffer output)
|
| 136 | - `find -exec`
|
| 137 | - `make`
|
| 138 | - `make -j` starts parallel processes (but doesn't buffer output)
|
| 139 | - `ninja` (buffers output)
|