| 1 | ---
|
| 2 | default_highlighter: oils-sh
|
| 3 | ---
|
| 4 |
|
| 5 | YSH FAQ
|
| 6 | =======
|
| 7 |
|
| 8 | Here are some common questions about [YSH]($xref). Many of the answers boil
|
| 9 | down to the fact that YSH is a **smooth upgrade** from [bash]($xref).
|
| 10 |
|
| 11 | Old and new constructs exist side-by-side. New constructs have fewer
|
| 12 | "gotchas".
|
| 13 |
|
| 14 | <!-- cmark.py expands this -->
|
| 15 | <div id="toc">
|
| 16 | </div>
|
| 17 |
|
| 18 | ## What's the difference `myvar`, `$myvar`, and `"$myvar"` ?
|
| 19 |
|
| 20 | YSH is more like Python/JavaScript rather than PHP/Perl, so it doesn't use the
|
| 21 | `$` sigil as much.
|
| 22 |
|
| 23 | Never use `$` on the left-hand side:
|
| 24 |
|
| 25 | var mystr = "foo" # not var $mystr
|
| 26 |
|
| 27 | Use `$` to **substitute** vars into commands:
|
| 28 |
|
| 29 | echo $mystr
|
| 30 | echo $mystr/subdir # no quotes in commands
|
| 31 |
|
| 32 | or quoted strings:
|
| 33 |
|
| 34 | echo "$mystr/subdir"
|
| 35 | var x = "$mystr/subdir"
|
| 36 |
|
| 37 | Rarely use `$` on the right-hand side:
|
| 38 |
|
| 39 | var x = mystr # preferred
|
| 40 | var x = $mystr # ILLEGAL -- use remove $
|
| 41 | var x = ${mystr:-} # occasionally useful
|
| 42 |
|
| 43 | var x = $? # allowed
|
| 44 |
|
| 45 | See [Command vs. Expression Mode](command-vs-expression-mode.html) for more
|
| 46 | details.
|
| 47 |
|
| 48 | ## How do I write `~/src` or `~bob/git` in a YSH assignment?
|
| 49 |
|
| 50 | This should cover 80% of cases:
|
| 51 |
|
| 52 | var path = "$HOME/src" # equivalent to ~/src
|
| 53 |
|
| 54 | The old shell style will cover the remaining cases:
|
| 55 |
|
| 56 | declare path=~/src
|
| 57 | readonly other=~bob/git
|
| 58 |
|
| 59 | ---
|
| 60 |
|
| 61 | This is only in issue in *expressions*. The traditional shell idioms work in
|
| 62 | *command* mode:
|
| 63 |
|
| 64 | echo ~/src ~bob/git
|
| 65 | # => /home/alice/src /home/bob/git
|
| 66 |
|
| 67 | The underlying design issue is that the YSH expression `~bob` looks like a
|
| 68 | unary operator and a variable, not some kind of string substitution.
|
| 69 |
|
| 70 | Also, quoted `"~"` is a literal tilde, and shells disagree on what `~""` means.
|
| 71 | The rules are subtle, so we avoid inventing new ones.
|
| 72 |
|
| 73 | <!--
|
| 74 | TODO: I want the ${ ~/src } syntax though it's complicated by ksh command sub
|
| 75 | -->
|
| 76 |
|
| 77 | ## How do I write the equivalent of `echo -e` or `echo -n`?
|
| 78 |
|
| 79 | To echo special characters denoted by backslash escapes, use a
|
| 80 | statically-parsed string literal, not `echo -e`:
|
| 81 |
|
| 82 | echo u'tab \t newline \n' # YES: J8 style string is recommended in YSH
|
| 83 | echo $'tab \t newline \n' # bash-style string is also accepted
|
| 84 |
|
| 85 | These styles don't work in YSH:
|
| 86 |
|
| 87 | echo -e "tab \\t newline \\n" # NO: -e is printed literally
|
| 88 | echo -e "tab \t newline \n" # Error: Invalid char escape
|
| 89 |
|
| 90 | To omit the trailing newline, use the `write` builtin:
|
| 91 |
|
| 92 | write -n -- $prefix # YES
|
| 93 | write --end '' -- $prefix # synonym
|
| 94 |
|
| 95 | echo -n $prefix # NO: -n is printed literally
|
| 96 |
|
| 97 | ### Why Were `-e` and `-n` Removed?
|
| 98 |
|
| 99 | The idioms with `u''` and `write` are more powerful and consistent.
|
| 100 |
|
| 101 | Moreover, shell's `echo` is the *only* builtin that doesn't accept `--` to stop
|
| 102 | flag processing.
|
| 103 |
|
| 104 | That is, `echo "$flag"` always has a few bugs: when `$flag` is `-e`, `-n`,
|
| 105 | `-en`, or `-ne`. There's **no** way to fix this bug in POSIX shell.
|
| 106 |
|
| 107 | So portable shell scripts use:
|
| 108 |
|
| 109 | printf '%s\n' "$x" # print $x "unmolested" in POSIX shell
|
| 110 |
|
| 111 | We could have chosen to respect `echo -- $x`, but YSH already has:
|
| 112 |
|
| 113 | write -- $x # print $x "unmolested" in YSH
|
| 114 |
|
| 115 | That means YSH has:
|
| 116 |
|
| 117 | echo $x # an even shorter way
|
| 118 |
|
| 119 | So `echo` is technically superfluous in YSH, but it's also short, familiar, and
|
| 120 | correct.
|
| 121 |
|
| 122 | YSH isn't intended to be compatible with POSIX shell; only OSH is.
|
| 123 |
|
| 124 | ### How do I write a string literal with both `$myvar` and `\n`?
|
| 125 |
|
| 126 | In YSH, either use `$[ \n ]` inside a double-quoted string:
|
| 127 |
|
| 128 | $ echo "$myvar $[ \n ] two" # expression sub wraps \n
|
| 129 | value_of_myvar
|
| 130 | two
|
| 131 |
|
| 132 | Or use the concatenation operator `++` with two styles of string literal:
|
| 133 |
|
| 134 | echo $[u'newline \n' ++ " $year/$month/$day"]
|
| 135 |
|
| 136 | This POSIX shell behavior is probably not what you want:
|
| 137 |
|
| 138 | $ echo "\n"
|
| 139 | \n # not a newline!
|
| 140 |
|
| 141 | ### How do I find all the `echo` invocations I need to change when using YSH?
|
| 142 |
|
| 143 | A search like this can statically find most usages:
|
| 144 |
|
| 145 | $ egrep -n 'echo (-e|-n|-en|-ne)' *.sh
|
| 146 | test/syscall.sh:58: echo -n hi
|
| 147 | test/syscall.sh:76: echo -e '\t'
|
| 148 |
|
| 149 | ## What's the difference between `$(dirname $x)` and `$[len(x)]` ?
|
| 150 |
|
| 151 | Superficially, both of these syntaxes take an argument `x` and return a
|
| 152 | string. But they are different:
|
| 153 |
|
| 154 | - `$(dirname $x)` is a shell command substitution that returns a string, and
|
| 155 | **starts another process**.
|
| 156 | - `$[len(x)]` is an expression sub containing a function call expression.
|
| 157 | - It doesn't need to start a process.
|
| 158 | - Note that `len(x)` evaluates to an integer, and `$[len(x)]` converts it to
|
| 159 | a string.
|
| 160 |
|
| 161 | <!--
|
| 162 | (Note: builtin subs like `${.myproc $x}` are meant to eliminate process
|
| 163 | overhead, but they're not yet implemented.)
|
| 164 | -->
|
| 165 |
|
| 166 | ## Why doesn't a raw string work here: `${array[r'\']}` ?
|
| 167 |
|
| 168 | This boils down to the difference between OSH and YSH, and not being able to
|
| 169 | mix the two. Though they look similar, `${array[i]}` syntax (with braces) is
|
| 170 | fundamentally different than `$[array[i]]` syntax (with brackets).
|
| 171 |
|
| 172 | - OSH supports `${array[i]}`.
|
| 173 | - The index is legacy/deprecated shell arithmetic like `${array[i++]}` or
|
| 174 | `${assoc["$key"]}`.
|
| 175 | - The index **cannot** be a raw string like `r'\'`.
|
| 176 | - YSH supports both, but [expression substitution][expr-sub] syntax
|
| 177 | `$[array[i]]` is preferred.
|
| 178 | - It accepts YSH expressions like `$[array[i + 1]` or `$[mydict[key]]`.
|
| 179 | - A raw string like `r'\'` is a valid key, e.g. `$[mydict[r'\']]`.
|
| 180 |
|
| 181 | [expr-sub]: ref/chap-expr-lang.html#expr-sub
|
| 182 |
|
| 183 | Of course, YSH style is preferred when compatibility isn't an issue.
|
| 184 |
|
| 185 | No:
|
| 186 |
|
| 187 | echo ${array[r'\']}
|
| 188 |
|
| 189 | Yes:
|
| 190 |
|
| 191 | echo $[array[r'\']]
|
| 192 |
|
| 193 | A similar issue exists with arithmetic.
|
| 194 |
|
| 195 | Old:
|
| 196 |
|
| 197 | echo $((1 + 2)) # shell arithmetic
|
| 198 |
|
| 199 | New:
|
| 200 |
|
| 201 | echo $[1 + 2] # YSH expression
|
| 202 |
|
| 203 | <!--
|
| 204 |
|
| 205 | ## Why doesn't the ternary operator work here: `${array[0 if cond else 5]}`?
|
| 206 |
|
| 207 | The issue is the same as above. YSH expression are allowed within `$[]` but
|
| 208 | not `${}`.
|
| 209 |
|
| 210 | -->
|
| 211 |
|
| 212 | ## How do I combine conditional commands and expressions: `if (myvar)` and `if test -f`?
|
| 213 |
|
| 214 | You can use the `--true` and `--false` flags to the [YSH test][ysh-test]
|
| 215 | builtin:
|
| 216 |
|
| 217 | if test --true $[myvar] && test --file x {
|
| 218 | echo ok
|
| 219 | }
|
| 220 |
|
| 221 | They test if their argument is literally the string `"true"` or `"false"`.
|
| 222 |
|
| 223 | This works because the boolean `true` *stringifies* to `"true"`, and likewise
|
| 224 | with `false`.
|
| 225 |
|
| 226 | [ysh-test]: ref/chap-builtin-cmd.html#ysh-test
|
| 227 |
|
| 228 |
|
| 229 | ## Why do I lose the value of `p` in `myproc (&p) | grep foo`?
|
| 230 |
|
| 231 | In a pipeline, most components are **forked**. This means that `myproc (&p)`
|
| 232 | runs in a different process from the main shell.
|
| 233 |
|
| 234 | The main shell can't see the memory of a subshell.
|
| 235 |
|
| 236 | ---
|
| 237 |
|
| 238 | In general, you have to restructure your code to avoid this. You could use a proc with multiple outputs:
|
| 239 |
|
| 240 | myproc (&p, &grepped_output)
|
| 241 |
|
| 242 | Or you could use a function:
|
| 243 |
|
| 244 | var out1, out2 = myfunc(io)
|
| 245 |
|
| 246 | ---
|
| 247 |
|
| 248 | [The Unix Shell Process Model - When Are Processes
|
| 249 | Created?](process-model.html) may help.
|
| 250 |
|
| 251 | This issue is similar to the `shopt -s lastpipe` issue:
|
| 252 |
|
| 253 | $ bash -c 'echo hi | read x; echo x=$x'
|
| 254 | x=
|
| 255 |
|
| 256 | $ zsh -c 'echo hi | read x; echo x=$x'
|
| 257 | x=hi
|
| 258 |
|
| 259 | In bash, `read` runs in a subshell, but in `zsh` and OSH, it runs in the main
|
| 260 | shell.
|
| 261 |
|
| 262 | ## Why are `Dict` and `Obj` different types?
|
| 263 |
|
| 264 | *JavaScript has a single Object type, while Python has separate dicts and
|
| 265 | objects.*
|
| 266 |
|
| 267 | In YSH, we draw a line between data and code.
|
| 268 |
|
| 269 | - A `Dict` is pure **data**, and may correspond to JSON from untrusted sources.
|
| 270 | - An `Obj` bundles both data and **code**, and can't be serialized by default.
|
| 271 |
|
| 272 | You can create an `Obj` from a `Dict` with the `Obj` constructor. Conversely,
|
| 273 | you can get the first Dict in an object with [first(myobj)][first].
|
| 274 |
|
| 275 | There is no special `__proto__` or `prototype` name, which reduces the
|
| 276 | likelihood of "prototype pollution" vulnerabilities.
|
| 277 |
|
| 278 | ---
|
| 279 |
|
| 280 | This is essentially the [Interior vs. Exterior][interior-exterior] distinction:
|
| 281 | An Obj lives inside the shell process, while a Dict may come from outside the
|
| 282 | process (user input).
|
| 283 |
|
| 284 | [first]: ref/chap-builtin-func.html#first
|
| 285 | [interior-exterior]: https://www.oilshell.org/blog/2023/06/ysh-design.html
|
| 286 |
|
| 287 | ## Why are `Command` and `Proc` different types?
|
| 288 |
|
| 289 | *Could a `Command` be a `Proc` with no arguments? Similarly, could an `Expr` be a
|
| 290 | `Func` with no arguments?*
|
| 291 |
|
| 292 | Procs and Funcs both push a new stack frame, and bind arguments to declared
|
| 293 | parameters.
|
| 294 |
|
| 295 | On the other hand, `Command` and `Expr` are more "raw" and flexible:
|
| 296 |
|
| 297 | - They can be evaluated in different stack frames —
|
| 298 | e.g. `io->eval(b, in_captured_frame)`
|
| 299 | - They can have "undeclared" variable bindings —
|
| 300 | e.g. `io->eval(b, vars={x: 42})`.
|
| 301 |
|
| 302 | In other words, they're low-level, reflective types that allow users to create
|
| 303 | expressive APIs, like:
|
| 304 |
|
| 305 | cd /tmp { # Command literal (block)
|
| 306 | ls -l
|
| 307 | }
|
| 308 | my-table | where [size > 3] # Expr literal
|
| 309 |
|
| 310 | ---
|
| 311 |
|
| 312 | Another way to think about it: we could have removed procs from the core YSH
|
| 313 | language, and implemented them in terms of command blocks and `io->eval()`.
|
| 314 | But that seems too low-level!
|
| 315 |
|
| 316 | ## Related
|
| 317 |
|
| 318 | - [Oil Language FAQ]($wiki) on the wiki has more answers. They may be migrated
|
| 319 | here at some point.
|
| 320 |
|