
<h1>
More Command line fun: downloading a podcast</h1>
<p>
In the show <a href="https://hackerpublicradio.org/eps/hpr4398/index.html" rel="noopener noreferrer" target="_blank">
hpr4398 :: Command line fun: downloading a podcast</a>
 Kevie walked us through a command to download a podcast.</p>
<p>
He used some techniques here that I hadn't used before, and it's always great to see how other people approach the problem.</p>
<p>
Let's have a look at the script and walk through what it does, then we'll have a look at some "traps for young players" as the <a href="https://www.eevblog.com/" rel="noopener noreferrer" target="_blank">
EEVBlog</a>
 is fond of saying.</p>
<p>
<br>
</p>
<h2>
Analysis of the Script</h2>
<pre class="ql-syntax" spellcheck="false">
wget `curl https://tuxjam.otherside.network/feed/podcast/ | grep -o 'https*://[^"]*ogg' | head -1`
</pre>
<p>
It chains four different commands together to "Save the latest file from a feed".</p>
<p>
Let's break it down so we can have checkpoints between each step.</p>
<p>
I often do this when writing a complex one liner - first do it as steps, and then combine it.</p>
<p>
<br>
</p>
<ol>
<li>
The curl command gets <a href="https://tuxjam.otherside.network/feed/podcast/" rel="noopener noreferrer" target="_blank">
https://tuxjam.otherside.network/feed/podcast/</a>
.</li>
</ol>
<p>
To do this ourselves we will call <code>
curl https://tuxjam.otherside.network/feed/podcast/ --output tuxjam.xml</code>
, as the default file name is index.html.</p>
<p>
This gives us a xml file, and we can confirm it's valid xml with the <code>
xmllint</code>
 command.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmllint --format tuxjam.xml &gt;/dev/null
$ echo $?
0
</pre>
<p>
Here the output of the command is ignored by redirecting it to <code>
/dev/null</code>
 Then we check the error code the last command had. As it's <code>
0</code>
 it completed sucessfully.</p>
<p>
<br>
</p>
<ol>
<li>
Kevie then passes the output to the <code>
grep</code>
 search command with the option <code>
-o</code>
 and then looks for any string starting with https followed by anything then followed by two forward slashes, then</li>
</ol>
<pre class="ql-syntax" spellcheck="false">
-o, --only-matching
      Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line
</pre>
<p>
We can do the same with. I was not aware that grep defaulted to regex, as I tend to add the <code>
--perl-regexp</code>
 to explicitly add it.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
grep --only-matching 'https*://[^"]*ogg' tuxjam.xml

    http matches the characters http literally (case sensitive)
    s* matches the character s literally (case sensitive)
        Quantifier: * Between zero and unlimited times, as many times as possible, giving back as needed [greedy]
    : matches the character : literally
    / matches the character / literally
    / matches the character / literally
    [^"]* match a single character not present in the list below
        Quantifier: * Between zero and unlimited times, as many times as possible, giving back as needed [greedy]
        " a single character in the list " literally (case sensitive)
    ogg matches the characters ogg literally (case sensitive)
</pre>
<p>
When we run this ourselves we get the following</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ grep --only-matching 'https*://[^"]*ogg' tuxjam.xml 
https://archive.org/download/tuxjam-121/tuxjam_121.ogg
https://archive.org/download/tuxjam-120/TuxJam_120.ogg
https://archive.org/download/tux-jam-119/TuxJam_119.ogg
https://archive.org/download/tuxjam_118/tuxjam_118.ogg
https://archive.org/download/tux-jam-117-uncut/TuxJam_117.ogg
https://tuxjam.otherside.network/tuxjam-115-ogg
https://archive.org/download/tuxjam_116/tuxjam_116.ogg
https://tuxjam.otherside.network/tuxjam-115-ogg
https://tuxjam.otherside.network/tuxjam-115-ogg
https://tuxjam.otherside.network/tuxjam-115-ogg
https://ogg
http://tuxjam.otherside.network/wp-content/uploads/sites/5/2024/10/tuxjam_115_OggCamp2024.ogg
https://ogg
https://archive.org/download/tuxjam_114/tuxjam_114.ogg
https://archive.org/download/tuxjam_113/tuxjam_113.ogg
https://archive.org/download/tuxjam_112/tuxjam_112.ogg
</pre>
<ol>
<li>
The last command returns the first line, so therefore <code>
https://archive.org/download/tuxjam-121/tuxjam_121.ogg</code>
</li>
<li>
Finally that line is used as the input to the <code>
wget</code>
 command.</li>
</ol>
<h2>
Problems with the approach</h2>
<p>
Relying on grep with structured data like xml or json can lead to problems.</p>
<p>
When we looked at the output of the command in step 2, some of the results gave <code>
https://ogg</code>
.</p>
<p>
When run the same command without the <code>
--only-matching</code>
 argument we see what was matched.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ grep 'https*://[^"]*ogg' tuxjam.xml 
                &lt;enclosure url="https://archive.org/download/tuxjam-121/tuxjam_121.ogg" length="66760045" type="audio/ogg" /&gt;
                &lt;enclosure url="https://archive.org/download/tuxjam-120/TuxJam_120.ogg" length="71038808" type="audio/ogg" /&gt;
                &lt;enclosure url="https://archive.org/download/tux-jam-119/TuxJam_119.ogg" length="77851752" type="audio/ogg" /&gt;
                &lt;enclosure url="https://archive.org/download/tuxjam_118/tuxjam_118.ogg" length="87356918" type="audio/ogg" /&gt;
                &lt;enclosure url="https://archive.org/download/tux-jam-117-uncut/TuxJam_117.ogg" length="69092470" type="audio/ogg" /&gt;
&lt;p&gt;This episode may not be live as in &lt;a href="https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/"&gt;TuxJam 115 from Oggcamp&lt;/a&gt; but your friendly foursome of Al, Dave (thelovebug), Kevie and Andrew (mcnalu) are very much alive to treats of Free and Open Source Software and Creative Commons tunes.&lt;/p&gt;
                &lt;enclosure url="https://archive.org/download/tuxjam_116/tuxjam_116.ogg" length="80274046" type="audio/ogg" /&gt;
                &lt;link&gt;https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/&lt;/link&gt;
                &lt;comments&gt;https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/#respond&lt;/comments&gt;
                &lt;wfw:commentRss&gt;https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/feed/&lt;/wfw:commentRss&gt;
&lt;p&gt;With the group meeting up together for the first time in person, it was decided that a live recording would be an appropriate venture. With the quartet squashed around a table and a group of adoring fans crowded into a room at the &lt;a href="https://www.pendulumhotel.co.uk/"&gt;Pendulum Hotel&lt;/a&gt; in Manchester, the discussion turns to TuxJam reviews that become regularly used applications, what we enjoyed about &lt;a href="https://oggcamp.org/"&gt;OggCamp 2024&lt;/a&gt; and for the third section the gang put their reputation on the line and allow open questions from the sea of dedicated fans.&lt;/p&gt;
                &lt;enclosure url="http://tuxjam.otherside.network/wp-content/uploads/sites/5/2024/10/tuxjam_115_OggCamp2024.ogg" length="62603883" type="audio/ogg" /&gt;
&lt;li&gt;&lt;a href="https://oggcamp.org/"&gt;OggCamp 2024&lt;/a&gt; on Saturday 12 and Sunday 13 October 2024, Manchester UK.&lt;/li&gt;
                &lt;enclosure url="https://archive.org/download/tuxjam_114/tuxjam_114.ogg" length="75051959" type="audio/ogg" /&gt;
                &lt;enclosure url="https://archive.org/download/tuxjam_113/tuxjam_113.ogg" length="72892480" type="audio/ogg" /&gt;
                &lt;enclosure url="https://archive.org/download/tuxjam_112/tuxjam_112.ogg" length="82495846" type="audio/ogg" /&gt;
</pre>
<p>
Two of the hits are not enclosures at all, they are references in the text to OggCamp <code>
what we enjoyed about &lt;a href="https://oggcamp.org/"&gt;OggCamp 2024&lt;/a&gt;</code>
</p>
<p>
Normally running <code>
grep</code>
 will only get one entry per line, and if the xml is minimised it can miss entries on a file that comes across as one big line.</p>
<p>
I did this myself using</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
xmllint --noblanks tuxjam.xml &gt; tuxjam-min.xml
</pre>
<p>
I then edited it and replaced the new lines with spaces. I have to say that the <code>
--only-matching</code>
 argument is doing a great job at pulling out the matches.</p>
<p>
That said the results were not perfect either.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ grep --only-matching 'https*://[^"]*ogg' tuxjam-min.xml
https://archive.org/download/tuxjam-121/tuxjam_121.ogg
https://archive.org/download/tuxjam-120/TuxJam_120.ogg
https://archive.org/download/tux-jam-119/TuxJam_119.ogg
https://archive.org/download/tuxjam_118/tuxjam_118.ogg
https://archive.org/download/tux-jam-117-uncut/TuxJam_117.ogg
https://tuxjam.otherside.network/tuxjam-115-ogg
https://archive.org/download/tuxjam_116/tuxjam_116.ogg
https://tuxjam.otherside.network/tuxjam-115-ogg
https://tuxjam.otherside.network/?p=1029&lt;/guid&gt;&lt;comments&gt;https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/#respond&lt;/comments&gt;&lt;wfw:commentRss&gt;https://tuxjam.otherside.network/tuxjam-115-ogg
https://ogg
http://tuxjam.otherside.network/wp-content/uploads/sites/5/2024/10/tuxjam_115_OggCamp2024.ogg
https://ogg
https://archive.org/download/tuxjam_114/tuxjam_114.ogg
https://archive.org/download/tuxjam_113/tuxjam_113.ogg
https://archive.org/download/tuxjam_112/tuxjam_112.ogg
</pre>
<p>
You could fix it by modifying the <code>
grep</code>
 arguments and add additional searches looking for <code>
enclosure</code>
. The problem with that approach is that you'll forever and a day be chasing issues when someone changes something.</p>
<p>
So the approach is officially "Grand", but it's a very likely to break if you're not babysitting it.</p>
<p>
<br>
</p>
<h2>
Suggested Applications.</h2>
<p>
I recommend never parsing <a href="https://en.wikipedia.org/wiki/Structured_document" rel="noopener noreferrer" target="_blank">
structured documents</a>
, like xml or json with grep.</p>
<p>
You should use dedicated parsers that understands the document markup, and can intelligently address parts of it.</p>
<p>
I recommend:</p>
<p>
<br>
</p>
<ul>
<li>
<code>
xml</code>
 use <a href="https://en.wikipedia.org/wiki/XMLStarlet" rel="noopener noreferrer" target="_blank">
xmlstarlet</a>
</li>
<li>
<code>
json</code>
 use <a href="https://jqlang.org/download/" rel="noopener noreferrer" target="_blank">
jq</a>
</li>
<li>
<code>
yaml</code>
 use <a href="https://github.com/mikefarah/yq" rel="noopener noreferrer" target="_blank">
yq</a>
</li>
</ul>
<p>
Of course anyone that looks at my code on the <a href="https://repo.anhonesthost.net/HPR" rel="noopener noreferrer" target="_blank">
hpr gittea</a>
 will know this is a case of "do what I say, not what I do."</p>
<p>
Never parse xml with grep, where the only possible exception is to see if a string is in a file in the first place.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
grep --max-count=1 --files-with-matches
</pre>
<p>
That's justified under the fact that <code>
grep</code>
 is going to be faster than having to parse, and build a <a href="https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Using_the_Document_Object_Model" rel="noopener noreferrer" target="_blank">
XML Document Object Model</a>
 when you don't have to.</p>
<p>
<br>
</p>
<h2>
Some Tips</h2>
<h3>
Always refer to examples and specification</h3>
<p>
A specification is just a set of rules that tell you how the document is formatted.</p>
<p>
There is a danger in just looking at example files, and not reading the specifications. I had a situation once where a software developer raised a bug as the files didn't begin with <code>
ken-test-</code>
 followed by a <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier" rel="noopener noreferrer" target="_blank">
uuid</a>
. They were surprised when the supplied files did not follow this convention as per the examples. Suffice to say that was rejected.</p>
<p>
For us there are the rules from the <a href="https://www.rssboard.org/rss-specification" rel="noopener noreferrer" target="_blank">
RSS specification</a>
 itself, but as it's a XML file there are <a href="https://www.w3.org/TR/REC-xml/" rel="noopener noreferrer" target="_blank">
XML Specifications</a>
. While the RSS spec is short, the XML is not, so people tend to use dedicated libraries to parse XML. Using a dedicated tool like <a href="https://en.wikipedia.org/wiki/XMLStarlet" rel="noopener noreferrer" target="_blank">
xmlstarlet</a>
 will allow us to mostly ignore the details of XML.</p>
<p>
<br>
</p>
<blockquote>
<a href="https://www.rssboard.org/rss-specification#whatIsRss" rel="noopener noreferrer" target="_blank">
RSS is a dialect of XML</a>
. All RSS files must conform to the XML 1.0 specification, as published on the World Wide Web Consortium (W3C) website.</blockquote>
<p>
The first line of the tuxjam feed shows it's an XML file.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
</pre>
<p>
The specification goes on to say "At the top level, a RSS document is a  element, with a mandatory attribute called version, that specifies the version of RSS that the document conforms to. If it conforms to this specification, the version attribute must be 2.0." And sure enough then the second line show that it's a RSS file.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
&lt;rss version="2.0" ...
</pre>
<h3>
Use the best tool for the job</h3>
<p>
You wouldn't grep a Excel File ? Why would you grep an XML file ?</p>
<p>
We could go on all day but I want to get across the idea that there is structure in the file. As XML is everywhere you should have a tool to process it. More than likely <code>
xmlstarlet</code>
 is in all the distro repos, so just install it.</p>
<p>
The help looks like this:</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet --help
XMLStarlet Toolkit: Command line utilities for XML
Usage: xmlstarlet [&lt;options&gt;] &lt;command&gt; [&lt;cmd-options&gt;]
where &lt;command&gt; is one of:
  ed    (or edit)      - Edit/Update XML document(s)
  sel   (or select)    - Select data or query XML document(s) (XPATH, etc)
  tr    (or transform) - Transform XML document(s) using XSLT
  val   (or validate)  - Validate XML document(s) (well-formed/DTD/XSD/RelaxNG)
  fo    (or format)    - Format XML document(s)
  el    (or elements)  - Display element structure of XML document
  c14n  (or canonic)   - XML canonicalization
  ls    (or list)      - List directory as XML
  esc   (or escape)    - Escape special XML characters
  unesc (or unescape)  - Unescape special XML characters
  pyx   (or xmln)      - Convert XML into PYX format (based on ESIS - ISO 8879)
  p2x   (or depyx)     - Convert PYX into XML
&lt;options&gt; are:
  -q or --quiet        - no error output
  --doc-namespace      - extract namespace bindings from input doc (default)
  --no-doc-namespace   - don't extract namespace bindings from input doc
  --version            - show version
  --help               - show help
Wherever file name mentioned in command help it is assumed
that URL can be used instead as well.

Type: xmlstarlet &lt;command&gt; --help &lt;ENTER&gt; for command help

XMLStarlet is a command line toolkit to query/edit/check/transform
XML documents (for more information see http://xmlstar.sourceforge.net/)
</pre>
<p>
You can get more help on a given topic by calling the <code>
xmlstarlet</code>
 <code>
command</code>
 like this:</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet el --help
XMLStarlet Toolkit: Display element structure of XML document
Usage: xmlstarlet el [&lt;options&gt;] &lt;xml-file&gt;
where
  &lt;xml-file&gt; - input XML document file name (stdin is used if missing)
  &lt;options&gt; is one of:
  -a    - show attributes as well
  -v    - show attributes and their values
  -u    - print out sorted unique lines
  -d&lt;n&gt; - print out sorted unique lines up to depth &lt;n&gt;

XMLStarlet is a command line toolkit to query/edit/check/transform
XML documents (for more information see http://xmlstar.sourceforge.net/)
</pre>
<p>
To prove that it's a structured document we can run the command <code>
xmlstarlet el -u</code>
 - show me unique elements</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet el -u tuxjam.xml 
rss
rss/channel
rss/channel/atom:link
rss/channel/copyright
rss/channel/description
rss/channel/generator
rss/channel/image
rss/channel/image/link
rss/channel/image/title
rss/channel/image/url
rss/channel/item
rss/channel/item/category
rss/channel/item/comments
rss/channel/item/content:encoded
rss/channel/item/description
rss/channel/item/enclosure
rss/channel/item/guid
rss/channel/item/itunes:author
rss/channel/item/itunes:duration
rss/channel/item/itunes:episodeType
rss/channel/item/itunes:explicit
rss/channel/item/itunes:image
rss/channel/item/itunes:subtitle
rss/channel/item/itunes:summary
rss/channel/item/link
rss/channel/item/pubDate
rss/channel/item/slash:comments
rss/channel/item/title
rss/channel/item/wfw:commentRss
rss/channel/itunes:author
rss/channel/itunes:category
rss/channel/itunes:explicit
rss/channel/itunes:image
rss/channel/itunes:owner
rss/channel/itunes:owner/itunes:name
rss/channel/itunes:subtitle
rss/channel/itunes:summary
rss/channel/itunes:type
rss/channel/language
rss/channel/lastBuildDate
rss/channel/link
rss/channel/podcast:guid
rss/channel/podcast:license
rss/channel/podcast:location
rss/channel/podcast:medium
rss/channel/podcast:podping
rss/channel/rawvoice:frequency
rss/channel/rawvoice:location
rss/channel/sy:updateFrequency
rss/channel/sy:updatePeriod
rss/channel/title
</pre>
<p>
That is the <a href="https://en.wikipedia.org/wiki/XPath" rel="noopener noreferrer" target="_blank">
xpath</a>
 representation of the xml structure. It's very similar to a unix filesystem tree. There is one <code>
rss</code>
 branch, of that is one <code>
channel</code>
 branch, and that can have many <code>
item</code>
 branches.</p>
<p>
<br>
</p>
<h3>
"Save the latest file from a feed"</h3>
<p>
The ask here is to "Save the latest file from a feed".</p>
<p>
The solution Kevie gave gets the "first entry in the feed", which is correct for his feed but is not safe.</p>
<p>
However let's see how we could replace <code>
grep</code>
 with <code>
xmlstarlet</code>
.</p>
<p>
The definition of <a href="https://www.rssboard.org/rss-specification#ltenclosuregtSubelementOfLtitemgt" rel="noopener noreferrer" target="_blank">
enclosure</a>
 is:</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
&lt;enclosure&gt; is an optional sub-element of &lt;item&gt;.

It has three required attributes. url says where the enclosure is located, 
length says how big it is in bytes, and type says what its type is, a standard MIME type.

The url must be an http url.

&lt;enclosure url="http://www.scripting.com/mp3s/weatherReportSuite.mp3" length="12216320" type="audio/mpeg" /&gt;
</pre>
<p>
The location of the files must be in <code>
rss/channel/item/enclosure</code>
 or it's not a Podcast feed.</p>
<p>
In each <code>
enclosure</code>
 there has to be a xml attribute called <code>
url</code>
 which points to the media.</p>
<p>
<code>
xmlstarlet</code>
 has the select command to select locations.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet sel --help
XMLStarlet Toolkit: Select from XML document(s)
Usage: xmlstarlet sel &lt;global-options&gt; {&lt;template&gt;} [ &lt;xml-file&gt; ... ]
where
  &lt;global-options&gt; - global options for selecting
  &lt;xml-file&gt; - input XML document file name/uri (stdin is used if missing)
  &lt;template&gt; - template for querying XML document with following syntax:

&lt;global-options&gt; are:
  -Q or --quiet             - do not write anything to standard output.
  -C or --comp              - display generated XSLT
  -R or --root              - print root element &lt;xsl-select&gt;
  -T or --text              - output is text (default is XML)
  -I or --indent            - indent output
  -D or --xml-decl          - do not omit xml declaration line
  -B or --noblanks          - remove insignificant spaces from XML tree
  -E or --encode &lt;encoding&gt; - output in the given encoding (utf-8, unicode...)
  -N &lt;name&gt;=&lt;value&gt;         - predefine namespaces (name without 'xmlns:')
                              ex: xsql=urn:oracle-xsql
                              Multiple -N options are allowed.
  --net                     - allow fetch DTDs or entities over network
  --help                    - display help

Syntax for templates: -t|--template &lt;options&gt;
where &lt;options&gt;
  -c or --copy-of &lt;xpath&gt;   - print copy of XPATH expression
  -v or --value-of &lt;xpath&gt;  - print value of XPATH expression
  -o or --output &lt;string&gt;   - output string literal
  -n or --nl                - print new line
  -f or --inp-name          - print input file name (or URL)
  -m or --match &lt;xpath&gt;     - match XPATH expression
  --var &lt;name&gt; &lt;value&gt; --break or
  --var &lt;name&gt;=&lt;value&gt;      - declare a variable (referenced by $name)
  -i or --if &lt;test-xpath&gt;   - check condition &lt;xsl:if test="test-xpath"&gt;
  --elif &lt;test-xpath&gt;       - check condition if previous conditions failed
  --else                    - check if previous conditions failed
  -e or --elem &lt;name&gt;       - print out element &lt;xsl:element name="name"&gt;
  -a or --attr &lt;name&gt;       - add attribute &lt;xsl:attribute name="name"&gt;
  -b or --break             - break nesting
  -s or --sort op xpath     - sort in order (used after -m) where
  op is X:Y:Z, 
      X is A - for order="ascending"
      X is D - for order="descending"
      Y is N - for data-type="numeric"
      Y is T - for data-type="text"
      Z is U - for case-order="upper-first"
      Z is L - for case-order="lower-first"
</pre>
<p>
Options we will need are:</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
-T or --text              - output is text (default is XML)
-t|--template &lt;options&gt; where &lt;options&gt;
-m or --match &lt;xpath&gt;     - match XPATH expression
-v or --value-of &lt;xpath&gt;  - print value of XPATH expression
-n or --nl                - print new line
</pre>
<p>
So putting it together we will get:</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet sel --text --template --match 'rss/channel/item' --value-of 'enclosure/@url' --nl  tuxjam.xml 
https://archive.org/download/tuxjam-121/tuxjam_121.ogg
https://archive.org/download/tuxjam-120/TuxJam_120.ogg
https://archive.org/download/tux-jam-119/TuxJam_119.ogg
https://archive.org/download/tuxjam_118/tuxjam_118.ogg
https://archive.org/download/tux-jam-117-uncut/TuxJam_117.ogg
https://archive.org/download/tuxjam_116/tuxjam_116.ogg
http://tuxjam.otherside.network/wp-content/uploads/sites/5/2024/10/tuxjam_115_OggCamp2024.ogg
https://archive.org/download/tuxjam_114/tuxjam_114.ogg
https://archive.org/download/tuxjam_113/tuxjam_113.ogg
https://archive.org/download/tuxjam_112/tuxjam_112.ogg
</pre>
<p>
We match for all the <code>
rss/channel/item</code>
 and look for any <code>
enclosure/@url</code>
s, simple enough.</p>
<p>
We could replace the grep in Kevie's script,</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
wget `curl https://tuxjam.otherside.network/feed/podcast/ | grep -o 'https*://[^"]*ogg' | head -1`
</pre>
<p>
with,</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
wget "$( curl --silent https://tuxjam.otherside.network/feed/podcast/ | xmlstarlet sel --text --template --match 'rss/channel/item' --value-of 'enclosure/@url' --nl  - | head -1 )"
</pre>
<p>
Which would guarantee to give the "first entry in the feed". Some additions is the use of <code>
$()</code>
 instead of back ticks, as it's easier to nest them, and I think it's clearer that something is been executed. I also added <code>
--silent</code>
 to <code>
curl</code>
 to suppress the progress bar. Also replaced the file name with <code>
-</code>
 dash to tell <code>
xmlstarlet</code>
 that the input will be piped from standard in.</p>
<p>
<br>
</p>
<h3>
How about the latest feed ?</h3>
<p>
There is nothing to stop someone producing a RSS feed where the latest entries are at the end, or even sorted alphabetically, or random. They are all valid use cases and are allowed under the Specification. So how would we find the "latest podcast" ?.</p>
<p>
While defined as optional, the <code>
items</code>
 <a href="https://www.rssboard.org/rss-specification#ltpubdategtSubelementOfLtitemgt" rel="noopener noreferrer" target="_blank">
pubDate</a>
 found at <code>
rss/channel/item/pubDate</code>
, is usually also always defined in podcast feeds.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
&lt;pubDate&gt; is an optional sub-element of &lt;item&gt;.

Its value is a date, indicating when the item was published. If it's a date in the future, aggregators may choose to not display the item until that date.

&lt;pubDate&gt;Sun, 19 May 2002 15:21:36 GMT&lt;/pubDate&gt;
</pre>
<p>
Unfortunately they picked a completely stupid <a href="https://www.w3.org/Protocols/rfc822/" rel="noopener noreferrer" target="_blank">
email date format</a>
, as opposed to a <a href="https://en.wikipedia.org/wiki/Atom_%28web_standard%29%23Date_formats" rel="noopener noreferrer" target="_blank">
sane format</a>
 like <a href="https://www.rfc-editor.org/rfc/rfc3339" rel="noopener noreferrer" target="_blank">
rfc3339</a>
 a subset of <a href="https://en.wikipedia.org/wiki/ISO_8601" rel="noopener noreferrer" target="_blank">
iso8601</a>
.</p>
<p>
So if you want to get the "latest podcast" you need to parse then convert the <code>
pubDate</code>
 to sortable format like <a href="https://en.wikipedia.org/wiki/ISO_8601" rel="noopener noreferrer" target="_blank">
iso8601</a>
 or <a href="https://en.wikipedia.org/wiki/Unix_time" rel="noopener noreferrer" target="_blank">
Unix epoch</a>
.</p>
<p>
I'll do the first part here, the second is it's own (series) of shows. <a href="https://www.youtube.com/watch?v=-5wpm-gesOY" rel="noopener noreferrer" target="_blank">
The Problem with Time &amp; Timezones - Computerphile</a>
</p>
<p>
So could it be as simple as replacing the xpath <code>
enclosure/@url</code>
 of with <code>
pubDate</code>
 ? Yes, yes it is.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet sel --text --template --match 'rss/channel/item' --value-of 'pubDate' --nl  tuxjam.xml 
Fri, 23 May 2025 17:54:17 +0000
Fri, 28 Feb 2025 15:48:57 +0000
Mon, 03 Feb 2025 09:57:26 +0000
Sat, 21 Dec 2024 17:08:26 +0000
Thu, 05 Dec 2024 12:57:52 +0000
Sat, 30 Nov 2024 20:31:20 +0000
Wed, 30 Oct 2024 08:10:58 +0000
Mon, 26 Aug 2024 15:11:51 +0000
Fri, 05 Jul 2024 18:15:44 +0000
Sat, 08 Jun 2024 09:21:50 +0000
</pre>
<p>
But we will need both the <code>
enclosure/@url</code>
 and <code>
pubDate</code>
, and we can do this using the <code>
concat</code>
 option. For more information on this see the <a href="https://xmlstar.sourceforge.net/doc/UG/xmlstarlet-ug.html" rel="noopener noreferrer" target="_blank">
XmlStarlet Command Line XML Toolkit User's Guide</a>
</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
$ xmlstarlet sel --text --template --match 'rss/channel/item' --value-of 'concat(pubDate, ";", enclosure/@url)' --nl  tuxjam.xml 
Fri, 23 May 2025 17:54:17 +0000;https://archive.org/download/tuxjam-121/tuxjam_121.ogg
Fri, 28 Feb 2025 15:48:57 +0000;https://archive.org/download/tuxjam-120/TuxJam_120.ogg
Mon, 03 Feb 2025 09:57:26 +0000;https://archive.org/download/tux-jam-119/TuxJam_119.ogg
Sat, 21 Dec 2024 17:08:26 +0000;https://archive.org/download/tuxjam_118/tuxjam_118.ogg
Thu, 05 Dec 2024 12:57:52 +0000;https://archive.org/download/tux-jam-117-uncut/TuxJam_117.ogg
Sat, 30 Nov 2024 20:31:20 +0000;https://archive.org/download/tuxjam_116/tuxjam_116.ogg
Wed, 30 Oct 2024 08:10:58 +0000;http://tuxjam.otherside.network/wp-content/uploads/sites/5/2024/10/tuxjam_115_OggCamp2024.ogg
Mon, 26 Aug 2024 15:11:51 +0000;https://archive.org/download/tuxjam_114/tuxjam_114.ogg
Fri, 05 Jul 2024 18:15:44 +0000;https://archive.org/download/tuxjam_113/tuxjam_113.ogg
Sat, 08 Jun 2024 09:21:50 +0000;https://archive.org/download/tuxjam_112/tuxjam_112.ogg
</pre>
<p>
I use the <code>
;</code>
 as delimiter.</p>
<p>
To tackle the date we will need to use the <a href="https://www.man7.org/linux/man-pages/man1/date.1.html" rel="noopener noreferrer" target="_blank">
date</a>
 command with the following options:</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
-d, --date=STRING
      display time described by STRING, not 'now'
-u, --utc, --universal
      print or set Coordinated Universal Time (UTC)
</pre>
<p>
The once it's in a sane format we can use the <a href="https://www.man7.org/linux/man-pages/man1/sort.1.html" rel="noopener noreferrer" target="_blank">
sort</a>
 to sort them with the newest on the top.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
-n, --numeric-sort
      compare according to string numerical value; see full documentation for supported strings
-r, --reverse
      reverse the result of comparisons
</pre>
<p>
Putting that into a script</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
#!/bin/bash
# (c) CC-0 Ken Fallon 2025

podcast="https://tuxjam.otherside.network/feed/podcast/"


wget --quiet $( curl --silent "${podcast}" | \
xmlstarlet sel --text --template --match 'rss/channel/item' --value-of 'concat(pubDate, ";", enclosure/@url)' --nl - | \
while read item
do
  pubDate="$( echo ${item} | awk -F ';' '{print $1}' )"
  pubDate="$( \date -d "${pubDate}" --universal +%Y-%m-%dT%H:%M:%S )"
  url="$( echo ${item} | awk -F ';' '{print $2}' )"
  echo -e "${pubDate}\t${url}"
done | \
sort --numeric-sort --reverse | \
head -1 | \
awk '{print $NF}' )
</pre>
<p>
Admittedly it's a smidgen linger than Kevies in it's one liner format.</p>
<p>
<br>
</p>
<pre class="ql-syntax" spellcheck="false">
wget "$( curl --silent "https://tuxjam.otherside.network/feed/podcast/" | xmlstarlet sel --text --template --match 'rss/channel/item' --value-of 'concat(pubDate, ";", enclosure/@url)' --nl - | while read item; do pubDate="$( echo ${item} | awk -F ';' '{print $1}' )"; pubDate="$( \date -d "${pubDate}" --universal +%Y-%m-%dT%H:%M:%S )"; url="$( echo ${item} | awk -F ';' '{print $2}' )"; echo -e "${pubDate}\t${url}"; done | sort --numeric-sort --reverse | head -1 | awk '{print $NF}' )"
</pre>
<p>
<br>
</p>


