OILS / devtools / release-note.sh View on Github | oils.pub

323 lines, 65 significant
1#!/usr/bin/env bash
2#
3# Usage:
4# devtools/release-note.sh <function name>
5#
6# Example:
7# $0 fetch-issues
8# $0 write-template
9
10set -o nounset
11set -o pipefail
12set -o errexit
13
14source build/dev-shell.sh # PYTHONPATH
15source devtools/release-version.sh # for escape-segments
16
17readonly OILS_VERSION=$(head -n 1 oils-version.txt)
18readonly PREV_VERSION='0.36.0'
19
20if false; then # replaced
21
22# adapted from release-version.sh
23_git-changelog-body() {
24 local commit=$1
25
26 # - a trick for HTML escaping (avoid XSS): surround %s with unlikely bytes,
27 # \x00 and \x01. Then pipe Python to escape.
28 # --reverse makes it go in forward chronlogical order.
29
30 # %x00 generates the byte \x00
31 local format='<tr>
32 <td><a class="checksum"
33 href="https://github.com/oils-for-unix/oils/commit/%H">%h</a>
34 </td>
35 <td>%x00%an%x01</td>
36 <td class="subject">%x00%s%x01</td>
37 </tr>'
38 git log \
39 --reverse \
40 --pretty="format:$format" \
41 --date=short \
42 -n 1 \
43 $commit \
44 | escape-segments
45}
46
47contrib-commit-table() {
48 # Filter out my commits, then pass back to git log
49
50 # 2023-07: Deoptimized due to git breakage
51 # https://stackoverflow.com/questions/6889830/equivalence-of-git-log-exclude-author
52
53 git log --format='%H %an' "release/$PREV_VERSION..release/$OILS_VERSION" |
54 grep -v 'Andy C' |
55 cut -d ' ' -f 1 |
56 xargs -n 1 $0 _git-changelog-body
57
58 #xargs -n 1 -- git log -n 1
59}
60fi
61
62fetch-issues() {
63 local label=${1:-pending-release}
64 local url="https://api.github.com/repos/oils-for-unix/oils/issues?labels=$label"
65 curl "$url" > _tmp/issues.json
66}
67
68issues-table() {
69 cat _tmp/issues.json | devtools/services/github_issues.py
70}
71
72readonly DATE_DIR=$(date +%Y/%m)
73readonly BLOG_DIR="../oils.pub/blog/$DATE_DIR"
74
75write-template() {
76 ### New blog post
77
78 local out=$BLOG_DIR/_release-$OILS_VERSION.md
79 print-template > $out
80 echo "Wrote $out"
81}
82
83write-zulip-thread() {
84 local bot_email=$1
85 local bot_api_key=$2
86 local version=${3:-$OILS_VERSION}
87
88 local out=$BLOG_DIR/release-thread-$version.txt
89 devtools/services/zulip.sh print-thread \
90 "$bot_email" "$bot_api_key" oil-dev "Oils $version Release" \
91 | tee $out
92
93 echo
94 echo "Wrote $out"
95}
96
97preview-template() {
98 local out=_tmp/release-note.html
99
100 # This isn't right because it doesn't split the --- front matter
101 # But good enough for now
102
103 print-template | doctools/cmark.py > $out
104 log "Wrote $out"
105}
106
107print-template() {
108 local metric_prev=${1:-$PREV_VERSION}
109
110 cat <<EOF
111---
112title: Oils $OILS_VERSION - Foo Foo
113date: $(date +%Y/%m/%d)
114css_file: blog-bundle-v7.css
115body_css_class: width40
116default_highlighter: oils-sh
117tags: oils-release
118comments_url: TODO
119published: no
120---
121
122<style>
123 /* shortlog style adapted from devtools/release-version.sh */
124 .checksum {
125 font-family: monospace;
126 /* margin-left: 2em; /* indent */
127 }
128 .shortlog {
129 font-size: large;
130 }
131 .shortlog td {
132 vertical-align: top;
133 }
134 .author-cell {
135 padding-top: 1em;
136 padding-bottom: 1em;
137 font-weight: bold;
138 color: darkgreen;
139 }
140</style>
141
142This is the latest version of Oils, a Unix shell that's our upgrade path from
143[bash][]:
144
145[bash]: \$xref
146
147<div class="attention">
148
149[Oils version $OILS_VERSION][release-index] - Source tarballs and documentation.
150
151</div>
152
153To build and run it, follow the instructions in [INSTALL.txt][]. The wiki has
154tips on [How To Test OSH](\$wiki).
155
156If you're new to the project, see [Why Create a New Shell?][why-create] and posts
157tagged #[FAQ][tagged-faq].
158
159[INSTALL.txt]: /release/$OILS_VERSION/doc/INSTALL.html
160
161[why-create]: https://www.oilshell.org/blog/2021/01/why-a-new-shell.html
162[tagged-faq]: https://www.oilshell.org/blog/tags.html?tag=FAQ#FAQ
163
164[github-bugs]: https://github.com/oils-for-unix/oils/issues
165[release-index]: /release/$OILS_VERSION/
166[oilshell.zulipchat.com]: http://oilshell.zulipchat.com/
167
168<div id="toc">
169</div>
170
171I'm trying something different this release. These release notes are
172semi-automated with a shell script! See the [last
173post](../02/good-parts-sketch.html) in #[shell-the-good-parts](\$blog-tag).
174
175## Closed Issues
176
177<table>
178EOF
179
180 issues-table
181
182 cat <<EOF
183</table>
184
185## Commit Log
186
187Here are the commits from other contributors. You can also view the [full
188changelog][changelog].
189
190[changelog]: /release/$OILS_VERSION/changelog.html
191
192<table class="shortlog">
193EOF
194
195 #contrib-commit-table
196 devtools/release-version.sh git-shortlog-html release/$PREV_VERSION release/$OILS_VERSION
197
198 cat <<EOF
199</table>
200
201## Documentation Updated
202
203- [Known Differences](/release/$OILS_VERSION/doc/known-differences.html)
204- [Interpreter State](/release/$OILS_VERSION/doc/interpreter-state.html) - still
205 a draft.
206
207### Wiki Pages
208
209- [How Interactive Shells Work](https://github.com/oils-for-unix/oils/wiki/How-Interactive-Shells-Work)
210
211
212## What's Next?
213
214Here are some notable Open Issues
215
216- [Provide APIs to allow users to write their own line editor / interactive
217interface](\$issue:663)
218
219## Appendix: Metrics for the $OILS_VERSION Release
220
221Let's review release metrics, which help me keep track of the project. The last review was in May, in [Metrics for the 0.29.0 Release](../05/metrics.html).
222
223### Docs
224
225We continue to improve the [Oils Reference](/release/0.35.0/doc/ref/). In addition to adding new topics, I polished existing topics.
226
227- [Doc Metrics for 0.29.0](//oils.pub/release/0.29.0/doc/metrics.txt) - **428** topics with first pass, **455** marked implemented, **496** unique
228- [Doc Metrics for 0.35.0](//oils.pub/release/0.35.0/doc/metrics.txt) - **442** topics with first pass, **466** marked implemented, **510** unique
229
230[NLnet]: https://nlnet.nl
231
232### Spec Tests
233
234[OSH][] made good progress, with **91** new passing spec tests.
235
236
237
238- [OSH spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/osh-py/index.html): **2517** tests, **2248** passing, **134** failing
239- [OSH spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/osh-py/index.html): **2608** tests, **2328** passing, **156** failing
240
241But it also feels like there are too many failing tests.
242
243Let's look at the C++ translation:
244
245[CPython]: \$xref:cpython
246
247- [C++ spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/osh-cpp/compare.html) - **2240** of **2248** passing - delta **8**
248- [C++ spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/osh-cpp/compare.html) - **2322** of **2328** passing - delta **6**
249
250---
251
252[YSH][] has **40** more tests passing:
253
254- [YSH spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/ysh-py/index.html): **1090** tests, **1022** passing, **68** failing
255- [YSH spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/ysh-py/index.html): **1133** tests, **1062** passing, **71** failing
256
257They all pass in the C++ translation:
258
259- [YSH C++ spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/ysh-cpp/compare.html): **1020** of **1022** passing, delta **2**
260- [YSH C++ spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/ysh-cpp/compare.html): **1060** of **1062** passing, delta **2**
261
262(The delta is again due to a \`Dict\` ordering issue in [Hay](\$xref:hay), which is harmless right now)
263
264[YSH]: \$xref
265
266### Benchmarks
267
268No significant changes in parser speed:
269
270- [Parser Performance for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/osh-parser/): **12.9** thousand irefs per line
271- [Parser Performance for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/osh-parser/): **12.3**
272 thousand irefs per line
273
274Or memory usage:
275
276- [benchmarks/gc for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/gc/): \`parse.configure-coreutils\` **1.65 M** objects comprising **41.0 MB**, max RSS **46.6 MB**
277- [benchmarks/gc for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/gc/): \`parse.configure-coreutils\` **1.65 M** objects comprising **44.6 MB**, max RSS **50.8 MB**
278
279#### Runtime
280
281A compute-bound workload is the same speed:
282
283- [benchmarks/gc-cachegrind for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/gc-cachegrind/) - \`fib\` takes **25.4** million irefs, mut+alloc+free+gc
284- [benchmarks/gc-cachegrind for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/gc-cachegrind/) - \`fib\` takes **25.4** million irefs, mut+alloc+free+gc
285
286This is the **infamous** autotools \`configure\` workload, which unfortunately has **noisy** measurements:
287
288- [Runtime Performance for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/osh-runtime/) on 2 machines
289 - **0.96x** and **1.05x** the speed of bash on \`configure.cpython\`
290 - **1.04x** and **1.10x** the speed of bash on \`configure.util-linux\`
291- [Runtime Performance for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/osh-runtime/) on 2 machines
292 - **0.95x** and **0.87x** the speed of bash on \`configure.cpython\`
293 - **1.04x** and **1.00x** the speed of bash on \`configure.util-linux\`
294
295#### Code Size
296
297Even though many new tests pass, our code is still short:
298
299- [cloc for 0.29.0](//oils.pub/release/0.29.0/pub/metrics.wwz/line-counts/cloc-report.txt)
300 - **25,185** significant lines in [OSH][], **6,380** in [YSH](\$xref), **1,691** in data languages
301 - **6,112** lines of hand-written C++
302- [cloc for 0.35.0](//oils.pub/release/0.35.0/pub/metrics.wwz/line-counts/cloc-report.txt)
303 - **25,594** significant lines in [OSH][], **6,617** in [YSH](\$xref), **1,690** in data languages
304 - **6,401** lines of hand-written C++
305
306The generated C++ is proportional:
307
308- [oils-cpp for 0.29.0](//oils.pub/release/0.29.0/pub/metrics.wwz/line-counts/oils-cpp.txt) - **128,449** physical lines
309- [oils-cpp for 0.35.0](//oils.pub/release/0.35.0/pub/metrics.wwz/line-counts/oils-cpp.txt) - 193,303 - 62,484 = **130,819** physical lines (accounting another source file)
310
311And compiled binary size is proportional:
312
313- [ovm-build for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/ovm-build/):
314 **2.40 MB** of native code (hoover, under GCC, on Debian 12)
315- [ovm-build for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/ovm-build/):
316 **2.42 MB** of native code (hoover, under GCC, on Debian 12)
317
318&nbsp;
319
320EOF
321}
322
323"$@"