<article>
<h2 id="til-1-incrementing-times-with-the-date-command">TIL 1:
Incrementing times with the <code>date</code> command</h2>
<ul>
<li>I have been working on an HPR project which is restoring external
files to shows which lost them when we migrated to the current static
site.</li>
<li>As I make changes I want to be able to check that they are correct,
but to make this check I need to wait for the next update of the static
site.</li>
<li>Ken was away recently, and set up a <code>cron</code> job to refresh
the site every three hours. Each show page shows the refresh time in UTC
form in the header.</li>
<li>Being a bit numerically challenged I wanted a way of computing the
next refresh time in my timezone from the previous refresh time.</li>
<li>The GNU <code>date</code> command accepts a date and time expression
after the <code>-d</code> option (or using the
alternative<code>--date=STRING</code> option). The contents of the
<code>STRING</code> here are very flexible but quite complex since you
can include time zone data, offsets, day and month names, etc. See the
links below for links to the GNU manual.</li>
<li>My first attempt used the <code>date</code> command like this and
got the wrong answer (using the output format <code>+%T</code> which
writes the time in a default form):</li>
</ul>
<pre><code>    $ date -d &#39;16:27:16 + 3 hours&#39; +%T
    15:27:16</code></pre>
<ul>
<li>It is not clear why this fails, but the GNU function which parses
these date parameters is obviously confused. The second try included the
time zone after the time, and worked better, but is a little
confusing:</li>
</ul>
<pre><code>    $ date -d &#39;16:27:16 UTC + 3 hours&#39; +%T
    20:27:16</code></pre>
<ul>
<li>The time returned is local time for me. The <code>date</code>
command has added three hours to the UTC date to get 19:27:16, but since
I am in the UK, which is in DST (called BST - British Summer Time - UTC
plus 1 hour), an hour is added.</li>
<li>The final try used the <code>-u</code> option which writes UTC
time:</li>
</ul>
<pre><code>    $ date -u -d &#39;16:27:16 UTC + 3 hours&#39; +%T
    19:27:16</code></pre>
<ul>
<li>I actually ended up using and re-using these commands (though a
script would have been better):</li>
</ul>
<pre><code>    $ current=&#39;06:27:55&#39;
    $ next=$(date -u -d &quot;${current}UTC + 3 hours 3 minutes&quot; +%T); echo &quot;$next UTC / $(date -d &quot;${next} UTC&quot; +&#39;%T %Z&#39;)&quot;
    09:30:55 UTC / 10:30:55 BST
    $ current=$next
    $ next=$(date -u -d &quot;${current}UTC + 3 hours 3 minutes&quot; +%T); echo &quot;$next UTC / $(date -d &quot;${next} UTC&quot; +&#39;%T %Z&#39;)&quot;
    12:33:55 UTC / 13:33:55 BST
    $ current=$next
    $ next=$(date -u -d &quot;${current}UTC + 3 hours 3 minutes&quot; +%T); echo &quot;$next UTC / $(date -d &quot;${next} UTC&quot; +&#39;%T %Z&#39;)&quot;
    15:36:55 UTC / 16:36:55 BST</code></pre>
<h2 id="til-1-links">TIL 1: Links</h2>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/index.html">GNU
Coreutils manual</a>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/System-context.html">System
context</a>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/date-invocation.html">date:
Print or set system date and time</a>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/Date-input-formats.html">Date
input formats</a> (The quotation at the start of this section of the
manual is quite interesting!)</li>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/Examples-of-date.html">Date
examples</a></li>
</ul></li>
</ul></li>
</ul></li>
</ul>
<hr />
<h2 id="til-2-merging-lines-of-files-with-paste">TIL 2: Merging lines of
files with <code>paste</code></h2>
<ul>
<li>While processing and "repairing" shows I came across the need to
generate a list of show numbers separated by commas. In the past I have
loaded these into a Bash array and turned them into a comma-delimited
string, using the parameter substitution capabilities of Bash which can
add a comma to each element. The trouble with this is that it leaves a
trailing comma which has to be removed. I stumbled upon
<code>paste</code> as an alternative way of doing this.</li>
<li>The GNU <code>paste</code> command is another from the <a
href="https://www.gnu.org/software/coreutils/manual/html_node/index.html">GNU
Coreutils</a> group. This one merges lines of files. Its synopsis
is:</li>
</ul>
<pre><code>    paste [OPTION]... [FILE]...</code></pre>
<ul>
<li>It merges lines consisting of the corresponding lines from each file
provided as an argument, by default separated by TABs, and writes them
to standard output. This means it produces lines consisting of the first
line from each of the files, separated by tabs, then the second lines,
and so on.</li>
<li>Any of the files can be linked to standard in by using a
<code>-</code> (hyphen) as the file name.</li>
<li>The delimiters can be changed with the <code>-d LIST</code> or
<code>--delimiters=LIST</code> option. The use of a list of delimiters
causes the characters in the list to be used sequentially for each
delimiter.</li>
<li>The merged lines can be visualised as rows in a matrix, where each
file provides a column.</li>
<li>The "matrix" is rotated by using the <code>-s</code> or
<code>--serial</code> option. Here the lines from one file at a time are
merged - the files are processed <em>serially</em> rather than in
<em>parallel</em>.</li>
<li>The <code>paste</code> command can be used to generate the
comma-delimited list I wanted by using the options <code>-s</code> and
<code>-d ','</code>:</li>
</ul>
<pre><code>    $ printf &#39;%s&bsol;n&#39; {1..10} | paste -s -d, -
    1,2,3,4,5,6,7,8,9,10</code></pre>
<ul>
<li>Note that you can't get the same result with
<code>echo {1..10}</code> because all the numbers will be written to one
line rather than being the separate lines that <code>paste</code>
requires.</li>
<li>The file arguments to <code>paste</code> may also be Bash
<em>process substitution</em> expressions:</li>
</ul>
<pre><code>    $ paste -s -d, &lt;(printf &#39;%s&bsol;n&#39; {1..10})
    1,2,3,4,5,6,7,8,9,10</code></pre>
<ul>
<li>This means you can generate more complex output by using multiple
<em>process substitution</em> expressions where each is seen as a
file:</li>
</ul>
<pre><code>    $ paste -d&#39;|&#39; &lt;(printf &#39;%d&bsol;n&#39; {1..7}) &lt;(printf &#39;%s&bsol;n&#39; {A..G}) &lt;(printf &#39;%d&bsol;n&#39; {100..106})
    1|A|100
    2|B|101
    3|C|102
    4|D|103
    5|E|104
    6|F|105
    7|G|106</code></pre>
<ul>
<li>Note how using the <code>-s</code> option "rotates" this:</li>
</ul>
<pre><code>    $ paste -s -d&#39;|&#39; &lt;(printf &#39;%d&bsol;n&#39; {1..7}) &lt;(printf &#39;%s&bsol;n&#39; {A..G}) &lt;(printf &#39;%d&bsol;n&#39; {100..106})
    1|2|3|4|5|6|7
    A|B|C|D|E|F|G
    100|101|102|103|104|105|106</code></pre>
<ul>
<li>This is what I was actually trying to do, so I could feed show
numbers to another script which will accept a CSV list as an
option:</li>
</ul>
<pre><code>    $ echo &quot;select episode_id from repairs where repair_date is null order by episode_id desc limit 5&quot; |&bsol;
    sqlite3 -list ~/HPR/InternetArchive/ia.db | paste -s -d&#39;,&#39; -
    1959,1952,1951,1946,1941</code></pre>
<h2 id="til-2-links">TIL 2: Links</h2>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/index.html">GNU
Coreutils manual</a>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/Operating-on-fields.html">Operating
on fields</a>
<ul>
<li><a
href="https://www.gnu.org/software/coreutils/manual/html_node/paste-invocation.html">paste:
Merge lines of files</a></li>
</ul></li>
</ul></li>
</ul>
</article>
