OILS / doc / ref / chap-cmd-lang.md View on Github | oils.pub

781 lines, 503 significant
1---
2title: Command Language (Oils Reference)
3all_docs_url: ..
4body_css_class: width40
5default_highlighter: oils-sh
6preserve_anchor_case: yes
7---
8
9<div class="doc-ref-header">
10
11[Oils Reference](index.html) &mdash;
12Chapter **Command Language**
13
14</div>
15
16This chapter describes the command language for OSH, and some YSH extensions.
17
18<span class="in-progress">(in progress)</span>
19
20<div id="dense-toc">
21</div>
22
23## Quick Sketch: What's a Command?
24
25OSH:
26
27 print-files() {
28 for name in *.py; do
29 if test -x "$name"; then
30 echo "$name is executable"
31 fi
32 done
33 }
34
35YSH:
36
37 proc print-files {
38 for name in *.py {
39 if test -x $name { # no quotes needed
40 echo "$name is executable"
41 }
42 }
43 }
44
45
46<h2 id="Commands">Commands</h2>
47
48### simple-command
49
50Commands are composed of words. The first word may be the name of
51
521. An alias
531. A builtin command
541. A YSH `proc`
551. A shell "function"
561. An external command
57
58Examples:
59
60 echo hi # a shell builtin doesn't start a process
61 ls /usr/bin ~/src # starts a new process
62 myproc "hello $name"
63 myshellfunc "hello $name"
64 myalias -l
65<!-- TODO: document lookup order -->
66
67Redirects are also allowed in any part of the command:
68
69 echo 'to stderr' >&2
70 echo >&2 'to stderr'
71
72 echo 'to file' > out.txt
73 echo > out.txt 'to file'
74
75### command-lookup-order
76
77The first word in a command can mean many things.
78
79(1) An [alias][] may expand into shell code at **parse** time. This includes
80keywords!
81
82Aliases are **disabled** in YSH (`shopt --unset expand_aliases`).
83
84(2) Shell keywords like `if` have their own **parsing** rules:
85
86 if test -d /; then # OSH style
87 echo yes
88 fi
89
90 if test --dir / { # YSH style
91 echo yes
92 }
93
94(3) The first word of a [simple command][simple-command] is looked up in this
95order:
96
971. Special builtins like `eval`
981. YSH Procs or Shell Functions
991. Normal Builtins like `cd`
1001. External Processes like `/bin/ls`
101
102So special builtins can't be redefined as functions, but normal builtins can.
103
104YSH also adds the concept of a **private** builtin. Private
105builtins like [sleep][] are **not** consulted as the first word:
106
107 sleep 1 # run external command
108 builtin sleep 1 # run private builtin with explicit prefix
109
110[alias]: chap-builtin-cmd.html#alias
111[simple-command]: chap-cmd-lang.html#simple-command
112[sleep]: chap-builtin-cmd.html#sleep
113
114To summarize, OSH commands are looked up in this order:
115
1161. alias
1171. keyword
1181. special builtin
1191. shell function
1201. normal builtin
1211. external
122
123YSH commands are looked up in this order:
124
1251. keyword
1261. special builtin
1271. YSH proc
1281. normal builtin
1291. external
130
131Logically separate:
132
1336. private builtin (not the first word)
134
135Use [invoke --show][invoke] to see different meanings for a given name.
136
137[invoke]: chap-builtin-cmd.html#invoke
138
139### prefix-binding
140
141Bindings are allowed before a simple command:
142
143 PYTHONPATH=. mydir/myscript.py
144
145These bindings set a variable and mark it exported. This binding is usually
146temporary, but when used with certain [special builtins][special], it persists.
147
148[special]: https://www.gnu.org/software/bash/manual/html_node/Special-Builtins.html
149
150- Related: [ysh-prefix-binding](ysh-prefix-binding)
151
152### ysh-prefix-binding
153
154YSH prefix bindings look exactly like they do in shell:
155
156 PYTHONPATH=. mydir/myscript.py
157
158However, they temporarily set `ENV.PYTHONPATH`, not `$PYTHONPATH`. This is
159done by adding a new `Dict` to the prototype chain of the `Obj`.
160
161The new `ENV` then becomes the environment of the child processes for the
162command.
163
164In YSH, prefix bindings are always temporary ENV bindings, and they don't
165persist. This is enforced by `shopt --set strict_env_binding`.
166
167- Related: [ENV](chap-special-var.html#ENV), [prefix-binding](chap-cmd-lang.html#prefix-binding)
168
169
170<h3 id="semicolon" class="osh-ysh-topic">semicolon ;</h3>
171
172Run two commands in sequence like this:
173
174 echo one; echo two
175
176or this:
177
178 echo one
179 echo two
180
181<h2 id="Conditional">Conditional</h2>
182
183<h3 id="case" class="osh-topic">case</h3>
184
185Match a string against a series of glob patterns. Execute code in the section
186below the matching pattern.
187
188 path='foo.py'
189 case "$path" in
190 *.py)
191 echo 'python'
192 ;;
193 *.sh)
194 echo 'shell'
195 ;;
196 esac
197
198For bash compatibility, the `;;` terminator can be substituted with either:
199
200- `;&` - fall through to next arm, ignoring the condition
201- `;;&` - fall through to next arm, respecting the condition
202
203<h3 id="if" class="osh-topic">if</h3>
204
205Test if a command exited with status zero (true). If so, execute the
206corresponding block of code.
207
208Shell:
209
210 if test -d foo; then
211 echo 'foo is a directory'
212 elif test -f foo; then
213 echo 'foo is a file'
214 else
215 echo 'neither'
216 fi
217
218YSH:
219
220 if test -d foo {
221 echo 'foo is a directory'
222 } elif test -f foo {
223 echo 'foo is a file'
224 } else {
225 echo 'neither'
226 }
227
228<h3 id="dbracket" class="osh-topic">dbracket [[</h3>
229
230Statically parsed boolean expressions, from bash and other shells:
231
232 x=42
233 if [[ $x -eq 42 ]]; then
234 echo yes
235 fi # => yes
236
237Compare with the [test][] builtin, which is dynamically parsed.
238
239See [bool-expr][] for the expression syntax.
240
241[test]: chap-builtin-cmd.html#test
242[bool-expr]: chap-mini-lang.html#bool-expr
243
244
245<h3 id="bang" class="osh-ysh-topic">bang !</h3>
246
247Invert an exit code:
248
249 if ! test -d /tmp; then
250 echo "No temp directory
251 fi
252
253<h3 id="and" class="osh-ysh-topic">and &amp;&amp;</h3>
254
255 mkdir -p /tmp && cp foo /tmp
256
257<h3 id="or" class="osh-ysh-topic">or ||</h3>
258
259 ls || die "failed"
260
261<h2 id="Iteration">Iteration</h2>
262
263<h3 id="while" class="osh-ysh-topic">while</h3>
264
265POSIX
266
267<h3 id="until" class="osh-topic">until</h3>
268
269POSIX
270
271<h3 id="for" class="osh-ysh-topic">for</h3>
272
273For loops iterate over words.
274
275YSH style:
276
277 var mystr = 'one'
278 var myarray = :| two three |
279
280 for i in $mystr @myarray *.py {
281 echo $i
282 }
283
284
285Shell style:
286
287 local mystr='one'
288 local myarray=(two three)
289
290 for i in "mystr" "${myarray[@]}" *.py; do
291 echo $i
292 done
293
294Both fragments output 3 lines and then Python files on remaining lines.
295
296<h3 id="for-expr-sh" class="osh-topic">for-expr-sh</h3>
297
298A bash/ksh construct:
299
300 for (( i = 0; i < 5; ++i )); do
301 echo $i
302 done
303
304<h2 id="Control Flow">Control Flow</h2>
305
306These are keywords in Oils, not builtins!
307
308### break
309
310Break out of a loop. (Not used for case statements!)
311
312If optional argument N is specified, breaks N enclosing loops.
313
314### continue
315
316Continue to the next iteration of a loop.
317
318If optional argument N is specified, resumes the Nth enclosing loop.
319
320### return
321
322Return from a function with the given status (if value is omitted, the return
323status is that of the last command executed).
324
325### exit
326
327Exit the shell process with the given status:
328
329 exit 2
330
331<h2 id="Grouping">Grouping</h2>
332
333### sh-func
334
335POSIX:
336
337 f() {
338 echo args "$@"
339 }
340 f 1 2 3
341
342### sh-block
343
344POSIX:
345
346 { echo one; echo two; }
347
348The trailing `;` is necessary in OSH, but not YSH. In YSH, `parse_brace` makes
349`}` is more of a special word.
350
351
352### subshell
353
354 ( echo one; echo two )
355
356In YSH, use [forkwait](chap-builtin-cmd.html#forkwait) instead of parentheses.
357
358<h2 id="Concurrency">Concurrency</h2>
359
360### pipe
361
362Pipelines are a traditional POSIX shell construct:
363
364 ls /tmp | grep ssh | sort
365
366Related:
367
368- [`PIPESTATUS`]() in OSH
369- [`_pipeline_status`]() in YSH
370
371[PIPESTATUS]: chap-special-var.html#PIPESTATUS
372[_pipeline_status]: chap-special-var.html#_pipeline_status
373
374<h3 id="ampersand" class="osh-topic">ampersand &amp;</h3>
375
376Start a command as a background job. Don't wait for it to finish, and return
377control to the shell.
378
379The PID of the job is recorded in the `$!` variable.
380
381 sleep 1 &
382 echo pid=$!
383 { echo two; sleep 2 } &
384 wait
385 wait
386
387In YSH, use the [fork][] builtin.
388
389[fork]: chap-builtin-cmd.html#fork
390
391
392<h2 id="Redirects">Redirects</h2>
393
394### redir-file
395
396The operators `>` and `>>` redirect the `stdout` of a process to a disk file.
397The `<` operator redirects `stdin` from a disk file.
398
399---
400
401Examples of redirecting the `stdout` of a command:
402
403 echo foo > out.txt # overwrite out.txt
404 date >> stamp.txt # append to stamp.txt
405
406<!--
407 echo foo >| out.txt # clobber the file even if set -o noclobber
408-->
409
410Redirect to the `stdin` of a command:
411
412 cat < in.txt
413
414Redirects are compatible with POSIX and bash, so they take descriptor numbers
415on the left:
416
417 make 2> stderr.txt # '2>' is valid, but '2 >' is not
418
419Note that the word argument to **file** redirects is evaluated like bash, which
420is different than other arguments to other redirects:
421
422 tar -x -z < Python* # glob must expand to exactly 1 file
423 tar -x -z < $myvar # $myvar is split because it's unquoted
424
425In other words, it's evaluated **as** a sequence of 1 word, which **produces**
426zero to N strings. But redirects are only valid when it produces exactly 1
427string.
428
429(Related: YSH uses `shopt --set simple_word_eval`, which means that globs that
430match nothing evaluate to zero strings, not themselves.)
431
432<!-- They also take a file descriptor on the left -->
433
434
435### redir-desc
436
437Redirect to a file descriptor:
438
439 echo 'to stderr' >&2
440
441<!--
442NOTE: >&2 is just like <&2
443There's no real difference.
444-->
445
446### here-doc
447
448Here documents let you write the `stdin` of a process in the shell program.
449
450Specify a delimiter word (like EOF) after the redir operator (like `<<`).
451
452If it's unquoted, then `$` expansion happens, like a double-quoted string:
453
454 cat <<EOF
455 here doc with $double ${quoted} substitution
456 EOF
457
458If the delimiter is quoted, then `$` expansion does **not** happen, like a
459single-quoted string:
460
461 cat <<'EOF'
462 price is $3.99
463 EOF
464
465Leading tabs can be stripped with the `<<-` operator:
466
467 myfunc() {
468 cat <<-EOF
469 here doc with one tab leading tab stripped
470 EOF
471 }
472
473### here-str
474
475The `<<<` operator means that the argument is a `stdin` string, not a
476chosen delimiter.
477
478 cat <<< 'here string'
479
480The string **plus a newline** is the `stdin` value, which is consistent with
481GNU bash.
482
483### ysh-here-str
484
485You can also use YSH multi-line strings as "here strings". For example:
486
487Double-quoted:
488
489 cat <<< """
490 double
491 quoted = $x
492 """
493
494Single-quoted:
495
496 cat <<< '''
497 price is
498 $3.99
499 '''
500
501J8-style with escapes:
502
503 cat <<< u'''
504 j8 style string price is
505 mu = \u{3bc}
506 '''
507
508In these cases, a trailing newline is **not** added. For example, the first
509example is equivalent to:
510
511 write --end '' -- """
512 double
513 quoted = $x
514 """
515
516## Other Command
517
518<h3 id="dparen" class="osh-topic">dparen ((</h3>
519
520<h3 id="time" class="osh-ysh-topic">time</h3>
521
522 time [-p] pipeline
523
524Measures the time taken by a command / pipeline. It uses the `getrusage()`
525function from `libc`.
526
527Note that time is a KEYWORD, not a builtin!
528
529<!-- Note: bash respects TIMEFORMAT -->
530
531
532## YSH Simple
533
534### typed-arg
535
536Internal commands (procs and builtins) accept typed arguments in parentheses:
537
538 json write (myobj)
539
540Redirects can also appear after the typed args:
541
542 json write (myobj) >out.txt
543
544### lazy-expr-arg
545
546Expressions in brackets like this:
547
548 assert [42 === x]
549
550Are syntactic sugar for:
551
552 assert (^[42 === x])
553
554That is, it's single arg of type `value.Expr`.
555
556Redirects can also appear after the lazy typed args:
557
558 assert [42 === x] >out.txt
559
560- Related: [Expr][] type
561
562[Expr]: chap-type-method.html#Expr
563
564### block-arg
565
566Blocks can be passed to simple commands, either literally:
567
568 cd /tmp {
569 echo $PWD # prints /tmp
570 }
571 echo $PWD
572
573Or as an expression:
574
575 var block = ^(echo $PWD)
576 cd /tmp (; ; block)
577
578Note that `cd` has no typed or named arguments, so the two semicolons are
579preceded by nothing.
580
581When passed to procs, blocks capture the enclosing stack frame:
582
583 var x = 42
584 myproc {
585 # lexical scope is respected
586 echo "x = $x" # x = 42
587 }
588
589---
590
591Redirects can appear after the block arg:
592
593 cd /tmp {
594 echo $PWD # prints /tmp
595 } >out.txt
596
597
598Related:
599
600- [sh-block](#sh-block) in OSH.
601- [Command][] and [CommandFrag][] types.
602
603[Command]: chap-type-method.html#Command
604[CommandFrag]: chap-type-method.html#CommandFrag
605
606## YSH Cond
607
608### ysh-case
609
610Like the shell case statement, the Ysh case statement has **string/glob** patterns.
611
612 var s = 'README.md'
613 case (s) {
614 *.py { echo 'Python' }
615 *.cc | *.h { echo 'C++' }
616 * { echo 'Other' }
617 }
618 # => Other
619
620We also generated it to **typed data** within `()`:
621
622 var x = 43
623 case (x) {
624 (30 + 12) { echo 'the integer 42' }
625 (else) { echo 'neither' }
626 }
627 # => neither
628
629The `else` is a special keyword that matches any value.
630
631 case (s) {
632 / dot* '.md' / { echo 'Markdown' }
633 (else) { echo 'neither' }
634 }
635 # => Markdown
636
637### ysh-if
638
639Like shell, you can use a command:
640
641 if test --file $x {
642 echo "$x is a file"
643 }
644
645You can also use an expression:
646
647 if (x > 0) {
648 echo 'positive'
649 }
650
651## YSH Iter
652
653### ysh-for
654
655#### Words
656
657This is a shell-style loop over "words":
658
659 for word in 'oils' $num_beans {pea,coco}nut {
660 echo $word
661 }
662 # =>
663 # oils
664 # 13
665 # peanut
666 # coconut
667
668You can ask for the loop index with `i,`:
669
670 for i, name in README.md *.py {
671 echo "$i $name"
672 }
673 # => 0 README.md
674 # => 1 foo.py
675
676#### Expressions Over Typed Data
677
678Expressions are enclosed in `()`. You can iterate over a `Range`, `List`,
679`Dict`, or `io.stdin`.
680
681Range:
682
683 for i in (3 ..< 5) { # range operator ..<
684 echo "i = $i"
685 }
686 # =>
687 # i = 3
688 # i = 4
689
690List:
691
692 var foods = ['ale', 'bean']
693 for item in (foods) {
694 echo $item
695 }
696 # =>
697 # ale
698 # bean
699
700---
701
702There are **three** ways of iterating over a `Dict`:
703
704 var mydict = {pea: 42, nut: 10}
705 for key in (mydict) {
706 echo $key
707 }
708 # =>
709 # pea
710 # nut
711
712 for key, value in (mydict) {
713 echo "$key $value"
714 }
715 # =>
716 # pea - 42
717 # nut - 10
718
719 for i, key, value in (mydict) {
720 echo "$i $key $value"
721 }
722 # =>
723 # 0 - pea - 42
724 # 1 - nut - 10
725
726That is, if you ask for two things, you'll get the key and value. If you ask
727for three, you'll also get the index.
728
729(One way to think of it: `for` loops in YSH have the functionality Python's
730`enumerate()`, `items()`, `keys()`, and `values()`.)
731
732---
733
734The `io.stdin` object iterates over lines:
735
736 for line in (io.stdin) {
737 echo $line
738 }
739 # lines are buffered, so it's much faster than `while read --raw-line`
740
741---
742
743(This section is based on [A Tour of YSH](../ysh-tour.html).)
744
745#### Closing Over the Loop Variable
746
747Each iteration of a `for` loop creates a new frame, which may be captured.
748
749 var x = 42 # outside the loop
750 for i in (0 ..< 3) {
751 var j = i + 2
752
753 var expr = ^"$x: i = $i, j = $j" # captures x, i, and j
754
755 my-task {
756 echo "$x: i = $i, j = $j" # also captures x, i, and j
757 }
758 }
759
760#### Mutating Containers in a `for` Loop
761
762- If you append or remove from a `List` while iterating over it, the loop **will** be affected.
763- If you mutate a `Dict` while iterating over it, the loop will **not** be
764 affected.
765
766### ysh-while
767
768You can use an expression as the condition:
769
770 var x = 5
771 while (x < 0) {
772 setvar x -= 1
773 }
774
775You or a command:
776
777 while test -f myfile {
778 echo 'myfile'
779 sleep 1
780 }
781