OILS / devtools / release-note.sh View on Github | oils.pub

321 lines, 65 significant
1#!/usr/bin/env bash
2#
3# Usage:
4# devtools/release-note.sh <function name>
5#
6# Example:
7# $0 fetch-issues
8# $0 write-template
9
10set -o nounset
11set -o pipefail
12set -o errexit
13
14source build/dev-shell.sh # PYTHONPATH
15source devtools/release-version.sh # for escape-segments
16
17readonly OILS_VERSION=$(head -n 1 oils-version.txt)
18readonly PREV_VERSION='0.36.0'
19
20if false; then # replaced
21
22# adapted from release-version.sh
23_git-changelog-body() {
24 local commit=$1
25
26 # - a trick for HTML escaping (avoid XSS): surround %s with unlikely bytes,
27 # \x00 and \x01. Then pipe Python to escape.
28 # --reverse makes it go in forward chronlogical order.
29
30 # %x00 generates the byte \x00
31 local format='<tr>
32 <td><a class="checksum"
33 href="https://github.com/oils-for-unix/oils/commit/%H">%h</a>
34 </td>
35 <td>%x00%an%x01</td>
36 <td class="subject">%x00%s%x01</td>
37 </tr>'
38 git log \
39 --reverse \
40 --pretty="format:$format" \
41 --date=short \
42 -n 1 \
43 $commit \
44 | escape-segments
45}
46
47contrib-commit-table() {
48 # Filter out my commits, then pass back to git log
49
50 # 2023-07: Deoptimized due to git breakage
51 # https://stackoverflow.com/questions/6889830/equivalence-of-git-log-exclude-author
52
53 git log --format='%H %an' "release/$PREV_VERSION..release/$OILS_VERSION" |
54 grep -v 'Andy C' |
55 cut -d ' ' -f 1 |
56 xargs -n 1 $0 _git-changelog-body
57
58 #xargs -n 1 -- git log -n 1
59}
60fi
61
62fetch-issues() {
63 local label=${1:-pending-release}
64 local url="https://api.github.com/repos/oils-for-unix/oils/issues?labels=$label"
65 curl "$url" > _tmp/issues.json
66}
67
68issues-table() {
69 cat _tmp/issues.json | devtools/services/github_issues.py
70}
71
72readonly DATE_DIR=$(date +%Y/%m)
73readonly BLOG_DIR="../oils.pub/blog/$DATE_DIR"
74
75write-template() {
76 ### New blog post
77
78 local out=$BLOG_DIR/_release-$OILS_VERSION.md
79 print-template > $out
80 echo "Wrote $out"
81}
82
83write-zulip-thread() {
84 local bot_email=$1
85 local bot_api_key=$2
86 local version=${3:-$OILS_VERSION}
87
88 local out=$BLOG_DIR/release-thread-$version.txt
89 devtools/services/zulip.sh print-thread \
90 "$bot_email" "$bot_api_key" oil-dev "Oils $version Release" \
91 | tee $out
92
93 echo
94 echo "Wrote $out"
95}
96
97preview-template() {
98 local out=_tmp/release-note.html
99
100 # This isn't right because it doesn't split the --- front matter
101 # But good enough for now
102
103 print-template | doctools/cmark.py > $out
104 log "Wrote $out"
105}
106
107print-template() {
108 local metric_prev=${1:-$PREV_VERSION}
109
110 cat <<EOF
111---
112title: Oils $OILS_VERSION - Foo Foo
113date: $(date +%Y/%m/%d)
114css_file: blog-bundle-v7.css
115body_css_class: width40
116default_highlighter: oils-sh
117tags: oils-release
118comments_url: TODO
119published: no
120---
121
122<style>
123 /* shortlog style adapted from devtools/release-version.sh */
124 .checksum {
125 font-family: monospace;
126 /* margin-left: 2em; /* indent */
127 }
128 #shortlog {
129 font-size: large;
130 }
131 #shortlog td {
132 vertical-align: top;
133 }
134 .author-cell {
135 padding-top: 1em;
136 padding-bottom: 1em;
137 font-weight: bold;
138 color: darkgreen;
139 }
140</style>
141
142This is the latest version of Oils, a Unix shell that's our upgrade path from
143[bash][]:
144
145[bash]: \$xref
146
147<div class="attention">
148
149[Oils version $OILS_VERSION][release-index] - Source tarballs and documentation.
150
151</div>
152
153To build and run it, follow the instructions in [INSTALL.txt][]. The wiki has
154tips on [How To Test OSH](\$wiki).
155
156If you're new to the project, see [Why Create a New Shell?][why-oil] and posts
157tagged #[FAQ](\$blog-tag).
158
159[INSTALL.txt]: /release/$OILS_VERSION/doc/INSTALL.html
160[github-bugs]: https://github.com/oils-for-unix/oils/issues
161[why-oil]: ../../2021/01/why-a-new-shell.html
162[release-index]: /release/$OILS_VERSION/
163
164[oilshell.zulipchat.com]: http://oilshell.zulipchat.com/
165
166<div id="toc">
167</div>
168
169I'm trying something different this release. These release notes are
170semi-automated with a shell script! See the [last
171post](../02/good-parts-sketch.html) in #[shell-the-good-parts](\$blog-tag).
172
173## Closed Issues
174
175<table>
176EOF
177
178 issues-table
179
180 cat <<EOF
181</table>
182
183## Commit Log
184
185Here are the commits from other contributors. You can also view the [full
186changelog][changelog].
187
188[changelog]: /release/$OILS_VERSION/changelog.html
189
190<table id="shortlog">
191EOF
192
193 #contrib-commit-table
194 devtools/release-version.sh git-shortlog-html release/$PREV_VERSION release/$OILS_VERSION
195
196 cat <<EOF
197</table>
198
199## Documentation Updated
200
201- [Known Differences](/release/$OILS_VERSION/doc/known-differences.html)
202- [Interpreter State](/release/$OILS_VERSION/doc/interpreter-state.html) - still
203 a draft.
204
205### Wiki Pages
206
207- [How Interactive Shells Work](https://github.com/oils-for-unix/oils/wiki/How-Interactive-Shells-Work)
208
209
210## What's Next?
211
212Here are some notable Open Issues
213
214- [Provide APIs to allow users to write their own line editor / interactive
215interface](\$issue:663)
216
217## Appendix: Metrics for the $OILS_VERSION Release
218
219Let's review release metrics, which help me keep track of the project. The last review was in May, in [Metrics for the 0.29.0 Release](../05/metrics.html).
220
221### Docs
222
223We continue to improve the [Oils Reference](/release/0.35.0/doc/ref/). In addition to adding new topics, I polished existing topics.
224
225- [Doc Metrics for 0.29.0](//oils.pub/release/0.29.0/doc/metrics.txt) - **428** topics with first pass, **455** marked implemented, **496** unique
226- [Doc Metrics for 0.35.0](//oils.pub/release/0.35.0/doc/metrics.txt) - **442** topics with first pass, **466** marked implemented, **510** unique
227
228[NLnet]: https://nlnet.nl
229
230### Spec Tests
231
232[OSH][] made good progress, with **91** new passing spec tests.
233
234
235
236- [OSH spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/osh-py/index.html): **2517** tests, **2248** passing, **134** failing
237- [OSH spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/osh-py/index.html): **2608** tests, **2328** passing, **156** failing
238
239But it also feels like there are too many failing tests.
240
241Let's look at the C++ translation:
242
243[CPython]: \$xref:cpython
244
245- [C++ spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/osh-cpp/compare.html) - **2240** of **2248** passing - delta **8**
246- [C++ spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/osh-cpp/compare.html) - **2322** of **2328** passing - delta **6**
247
248---
249
250[YSH][] has **40** more tests passing:
251
252- [YSH spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/ysh-py/index.html): **1090** tests, **1022** passing, **68** failing
253- [YSH spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/ysh-py/index.html): **1133** tests, **1062** passing, **71** failing
254
255They all pass in the C++ translation:
256
257- [YSH C++ spec tests for 0.29.0](//oils.pub/release/0.29.0/test/spec.wwz/ysh-cpp/compare.html): **1020** of **1022** passing, delta **2**
258- [YSH C++ spec tests for 0.35.0](//oils.pub/release/0.35.0/test/spec.wwz/ysh-cpp/compare.html): **1060** of **1062** passing, delta **2**
259
260(The delta is again due to a \`Dict\` ordering issue in [Hay](\$xref:hay), which is harmless right now)
261
262[YSH]: \$xref
263
264### Benchmarks
265
266No significant changes in parser speed:
267
268- [Parser Performance for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/osh-parser/): **12.9** thousand irefs per line
269- [Parser Performance for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/osh-parser/): **12.3**
270 thousand irefs per line
271
272Or memory usage:
273
274- [benchmarks/gc for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/gc/): \`parse.configure-coreutils\` **1.65 M** objects comprising **41.0 MB**, max RSS **46.6 MB**
275- [benchmarks/gc for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/gc/): \`parse.configure-coreutils\` **1.65 M** objects comprising **44.6 MB**, max RSS **50.8 MB**
276
277#### Runtime
278
279A compute-bound workload is the same speed:
280
281- [benchmarks/gc-cachegrind for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/gc-cachegrind/) - \`fib\` takes **25.4** million irefs, mut+alloc+free+gc
282- [benchmarks/gc-cachegrind for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/gc-cachegrind/) - \`fib\` takes **25.4** million irefs, mut+alloc+free+gc
283
284This is the **infamous** autotools \`configure\` workload, which unfortunately has **noisy** measurements:
285
286- [Runtime Performance for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/osh-runtime/) on 2 machines
287 - **0.96x** and **1.05x** the speed of bash on \`configure.cpython\`
288 - **1.04x** and **1.10x** the speed of bash on \`configure.util-linux\`
289- [Runtime Performance for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/osh-runtime/) on 2 machines
290 - **0.95x** and **0.87x** the speed of bash on \`configure.cpython\`
291 - **1.04x** and **1.00x** the speed of bash on \`configure.util-linux\`
292
293#### Code Size
294
295Even though many new tests pass, our code is still short:
296
297- [cloc for 0.29.0](//oils.pub/release/0.29.0/pub/metrics.wwz/line-counts/cloc-report.txt)
298 - **25,185** significant lines in [OSH][], **6,380** in [YSH](\$xref), **1,691** in data languages
299 - **6,112** lines of hand-written C++
300- [cloc for 0.35.0](//oils.pub/release/0.35.0/pub/metrics.wwz/line-counts/cloc-report.txt)
301 - **25,594** significant lines in [OSH][], **6,617** in [YSH](\$xref), **1,690** in data languages
302 - **6,401** lines of hand-written C++
303
304The generated C++ is proportional:
305
306- [oils-cpp for 0.29.0](//oils.pub/release/0.29.0/pub/metrics.wwz/line-counts/oils-cpp.txt) - **128,449** physical lines
307- [oils-cpp for 0.35.0](//oils.pub/release/0.35.0/pub/metrics.wwz/line-counts/oils-cpp.txt) - 193,303 - 62,484 = **130,819** physical lines (accounting another source file)
308
309And compiled binary size is proportional:
310
311- [ovm-build for 0.29.0](//oils.pub/release/0.29.0/benchmarks.wwz/ovm-build/):
312 **2.40 MB** of native code (hoover, under GCC, on Debian 12)
313- [ovm-build for 0.35.0](//oils.pub/release/0.35.0/benchmarks.wwz/ovm-build/):
314 **2.42 MB** of native code (hoover, under GCC, on Debian 12)
315
316&nbsp;
317
318EOF
319}
320
321"$@"