
<article>

<h1 id="overview">
Overview</h1>

<p>
Several years ago I wrote a Bash script to perform a task I need to
perform almost every day - find the newest file in a series of
files.</p>

<p>
At this point I was running a camera on a Raspberry Pi which was
attached to a window and viewed my back garden. I was taking a picture
every 15 minutes, giving them names containing the date and time, and
storing them in a directory. It was useful to be able to display the
latest picture.</p>

<p>
Since then, I have found that searching for newest files useful in
many contexts:</p>

<ul>

<li>
<p>
Find the image generated by my random recipe chooser, put in the
clipboard and send it to the Telegram channel for my family.</p>
</li>

<li>
<p>
Generate a weather report from <code>
wttr.in</code>
 and send it
to Matrix.</p>
</li>

<li>
<p>
Find the screenshot I just made and put it in the
clipboard.</p>
</li>

</ul>

<p>
Of course, I could just use the same name when writing these various
files, rather than accumulating several, but I often want to look back
through such collections. If I am concerned about such files
accumulating in an unwanted way I write <code>
cron</code>
 scripts which
run every day and delete the oldest ones.</p>

<h2 id="original-script">
Original script</h2>

<p>
The first iteration of the script was actually written as a Bash
function which was loaded at login time. The function is called
<code>
newest_matching_file</code>
 and it takes two arguments:</p>

<ul>

<li>
<p>
A file glob expression to match the file I am looking
for.</p>
</li>

<li>
<p>
An optional directory to look for the file. If this is omitted,
then the current directory will be used.</p>
</li>

</ul>

<p>
The first version of this function was a bit awkward since it used a
<code>
for</code>
 loop to scan the directory, using the glob pattern to
find the file. Since Bash glob pattern searches will return the search
pattern when they fail, it was necessary to use the
<code>
nullglob</code>
 (see references) option to prevent this, turning
it on before the search and off afterwards.</p>

<p>
This technique was replaced later with a pipeline using the
<code>
find</code>
 command.</p>

<h2 id="improved-bash-script">
Improved Bash script</h2>

<p>
The version using <code>
find</code>
 is what I will explain here.</p>

<pre>
<code>
function newest_matching_file {
    local glob_pattern=${1-}
    local dir=${2:-$PWD}

    # Argument number check
    if [[ $# -eq 0 || $# -gt 2 ]]; then
        echo &#39;Usage: newest_matching_file GLOB_PATTERN [DIR]&#39; &gt;&amp;2
        return 1
    fi

    # Check the target directory
    if [[ ! -d $dir ]]; then
        echo &quot;Unable to find directory $dir&quot; &gt;&amp;2
        return 1
    fi

    local newest_file

    # shellcheck disable=SC2016
    newest_file=$(find &quot;$dir&quot; -maxdepth 1 -name &quot;$glob_pattern&quot; &bsol;
        -type f -printf &quot;%T@ %p&bsol;n&quot; | sort | sed -ne &#39;${s/.&bsol;+ //;p}&#39;)

    # Use printf instead of echo in case the file name begins with &#39;-&#39;
    [[ -n $newest_file ]] &amp;&amp; printf &#39;%s&bsol;n&#39; &quot;$newest_file&quot;

    return 0
}</code>
</pre>

<p>
The function is in the file <code>
newest_matching_file_1.sh</code>
,
and it's loaded ("sourced", or declared) like this:</p>

<pre>
<code>
. newest_matching_file_1.sh</code>
</pre>

<p>
The <code>
'.'</code>
 is a short-hand version of the command
<code>
source</code>
.</p>

<p>
I actually have two versions of this function, with the second one
using a regular expression, which the <code>
find</code>
 command is able
to search with, but I prefer this one.</p>

<h3 id="explanation">
Explanation</h3>

<ul>

<li>
<p>
The first two lines beginning with <code>
local</code>
 define
variables local to the function holding the arguments. The first,
<code>
glob_pattern</code>
 is expected to contain something like
<code>
screenshot_2025-04-*.png</code>
. The second will hold the
directory to be scanned, or if omitted, will be set to the current
directory.</p>
</li>

<li>
<p>
Next, an <code>
if</code>
 statement checks that there are the
right number of arguments, aborting if not. Note that the
<code>
echo</code>
 command writes to STDERR (using
<code>
'&gt;&amp;2'</code>
), the error channel.</p>
</li>

<li>
<p>
Another <code>
if</code>
 statement checks that the target
directory actually exists, and aborts if not.</p>
</li>

<li>
<p>
Another local variable <code>
newest_file</code>
 is defined. It's
good practice not to create global variables in functions since they
will "leak" into the calling environment.</p>
</li>

<li>
<p>
The variable <code>
newest_file</code>
 is set to the result of a
command substitution containing a pipeline:</p>

<ul>

<li>
The <code>
find</code>
 command searches the target directory.
<ul>

<li>
Using <code>
-maxdepth 1</code>
 limits the search to the chosen
directory and does not descend into sub-directories.</li>

<li>
The search pattern is defined by
<code>
-name "$glob_pattern"</code>
</li>

<li>
Using <code>
-type f</code>
 limits the search to files</li>

<li>
The <code>
-printf "%T@ %p&bsol;n"</code>
 argument returns the file's last
modification time as the number of seconds since the Unix epoch
<code>
'%T@'</code>
. This is a number which is larger if the file is
older. This is followed, after a space, by the full path to the file
(<code>
'%p'</code>
), and a newline.</li>

</ul>
</li>

<li>
The matching file names are sorted. Because each is preceded by a
numeric time value, they will be sorted in ascending order of age.</li>

<li>
Finally <code>
sed</code>
 is used to return the last file in the
sorted list with the program <code>
'${s/.&bsol;+ //;p}'</code>
:
<ul>

<li>
The use of the <code>
-n</code>
 option ensures that only lines which
are explicitly printed will be shown.</li>

<li>
The <code>
sed</code>
 program looks for the last line (using
<code>
'$'</code>
). When found the leading numeric time is removed with
'<code>
s/.&bsol;+ //'</code>
 and the result is printed (with
<code>
'p'</code>
).</li>

</ul>
</li>

<li>
The end result will either be the path to the newest file or nothing
(because there was no match).</li>

</ul>
</li>

<li>
<p>
The expression <code>
'[[ -n $newest_file ]]'</code>
 will be true
if <code>
$newest_file</code>
 variable is not empty, and if that is the
case, the contents of the variable will be printed on STDOUT, otherwise
nothing will be printed.</p>
</li>

<li>
<p>
Note that the script returns 1 (false) if there is a failure, and
0 (true) if all is well. A null return is regarded as success.</p>
</li>

</ul>

<h4 id="script-update">
Script update</h4>

<p>
While editing the audio for this show I realised that there is a flaw
in the Bash function <code>
newest_matching_file</code>
. This is in the
<code>
sed</code>
 script used to process the output from
<code>
find</code>
.</p>

<p>
The <code>
sed</code>
 commands used in the script delete all
characters up to a space, assuming that this is the only space in the
last line. However, if the file name itself contains spaces, this will
not work because regular expressions in <code>
sed</code>
 are
<em>
greedy</em>
. What is deleted in this case is everything up to and
including the <em>
last</em>
 space.</p>

<p>
I created a directory called <code>
tests</code>
 and added the
following files:</p>

<pre>
<code>
&#39;File 1 with spaces.txt&#39;
&#39;File 2 with spaces.txt&#39;
&#39;File 3 with spaces.txt&#39;</code>
</pre>

<p>
I then ran the <code>
find</code>
 command as follows:</p>

<pre>
<code>
$ find tests -maxdepth 1 -name &#39;File*&#39; -type f -printf &quot;%T@ %p&bsol;n&quot; | sort | sed -ne &#39;${s/.&bsol;+ //;p}&#39;
spaces.txt</code>
</pre>

<p>
I adjusted the <code>
sed</code>
 call to
<code>
sed -ne '${s/[^ ]&bsol;+ //;p}'</code>
. This uses the regular
expression:</p>

<pre>
<code>
s/[^ ]&bsol;+ //</code>
</pre>

<p>
This now specifies that what it to be removed is every
<em>
non-space</em>
 up to and including the first space. The result
is:</p>

<pre>
<code>
$ find tests -maxdepth 1 -name &#39;File*&#39; -type f -printf &quot;%T@ %p&bsol;n&quot; | sort | sed -ne &#39;${s/[^ ]&bsol;+ //;p}&#39;
tests/File 3 with spaces.txt</code>
</pre>

<p>
This change has been propagated to the copy on
<code>
GitLab</code>
.</p>

<h3 id="usage">
Usage</h3>

<p>
This function is designed to be used in commands or other
scripts.</p>

<p>
For example, I have an alias defined as follows:</p>

<pre>
<code>
alias copy_screenshot=&quot;xclip -selection clipboard -t image/png -i &bsol;$(newest_matching_file &#39;Screenshot_*.png&#39; ~/Pictures/Screenshots/)&quot;</code>
</pre>

<p>
This uses <code>
xclip</code>
 to load the latest screenshot into the
clipboard, so I can paste it into a social media client for example.</p>

<h2 id="perl-alternative">
Perl alternative</h2>

<p>
During the history of this family of scripts I wrote a Perl version.
This was originally because the Bash function gave problems when run
under the Bourne shell, and I was using <code>
pdmenu</code>
 a lot which
internally runs scripts under that shell.</p>

<pre>
<code>
#!/usr/bin/env perl

use v5.40;

use open &#39;:std&#39;, &#39;:encoding(UTF-8)&#39;;    # Make all IO UTF-8

use Cwd;
use File::Find::Rule;

#
# Script name
#
( my $PROG = $0 ) =~ s|.*/||mx;

#
# Use a regular expression rather than a glob pattern
#
my $regex = shift;

#
# Get the directory to search, defaulting to the current one
#
my $dir = shift // getcwd();

#
# Have to have the regular expression
#
die &quot;Usage: $PROG regex [DIR]&bsol;n&quot; unless $regex;

#
# Collect all the files in the target directory without recursing. Include the
# path and let the caller remove it if they want.
#
my @files =  File::Find::Rule-&gt;file()
    -&gt;name(qr/$regex/)
    -&gt;maxdepth(1)
    -&gt;in($dir);

die &quot;Unsuccessful search&bsol;n&quot; unless @files;

#
# Sort the files by ascending modification time, youngest first
#
@files = sort {-M($a) &lt;=&gt; -M($b)} @files;

#
# Report the one which sorted first
#
say $files[0];

exit;</code>
</pre>

<h3 id="explanation-1">
Explanation</h3>

<ul>

<li>
<p>
This is fairly straightforward Perl script, run out of an
executable file with a <em>
shebang</em>
 line at the start indicating
what is to be used to run it - <code>
perl</code>
.</p>
</li>

<li>
<p>
The preamble defines the Perl version to use, and indicates that
<code>
UTF-8</code>
 (character sets like Unicode) will be acceptable for
reading and writing.</p>
</li>

<li>
<p>
Two modules are required:</p>

<ul>

<li>
<code>
Cwd</code>
: provides functions for determining the pathname of
the current working directory.</li>

<li>
<code>
File::Find::Rule</code>
: provides tools for searching the file
system (similar to the <code>
find</code>
 command, but with more
features).</li>

</ul>
</li>

<li>
<p>
Next the variable <code>
$PROG</code>
 is set to the name under
which the script has been invoked. This is useful when giving a brief
summary of usage.</p>
</li>

<li>
<p>
The first argument is then collected (with <code>
shift</code>
)
and placed into the variable <code>
$regex</code>
.</p>
</li>

<li>
<p>
The second argument is optional, but if omitted, is set to the
current working directory. We see the use of <code>
shift</code>
 again,
but if this returns nothing (is undefined), the <code>
'//'</code>

operator invokes the <code>
getcwd()</code>
 function to get the current
working directory.</p>
</li>

<li>
<p>
If the <code>
$regex</code>
 variable is not defined, then
<code>
die</code>
 is called to terminate the script with an error
message.</p>
</li>

<li>
<p>
The search itself is invoked using <code>
File::Find::Rule</code>

and the results are added to the array <code>
@files</code>
. The
multi-line call shows several methods being called in a "chain" to
define the rules and invoke the search:</p>

<ul>

<li>
<code>
file()</code>
: sets up a file search</li>

<li>
<code>
name(qr/$regex/)</code>
: a rule which applies a regular
expression match to each file name, rejecting any that do not match</li>

<li>
<code>
maxdepth(1)</code>
: a rule which prevents the search from
descending below the top level into sub-directories</li>

<li>
<code>
in($dir)</code>
: defines the directory to search (and also
begins the search)</li>

</ul>
</li>

<li>
<p>
If the search returns no files (the array is empty), the script
ends with an error message.</p>
</li>

<li>
<p>
Otherwise the <code>
@files</code>
 array is sorted. This is done
by comparing modification times of the files, with the array being
reordered such that the "youngest" (newest) file is sorted first. The
<code>
&lt;=&gt;</code>
 operator checks if the value of the left operand
is greater than the value of the right operand, and if yes then the
condition becomes true. This operator is most useful in the Perl
<code>
sort</code>
 function.</p>
</li>

<li>
<p>
Finally, the newest file is reported.</p>
</li>

</ul>

<h3 id="usage-1">
Usage</h3>

<p>
This script can be used in almost the same way as the Bash variant.
The difference is that the pattern used to match files is a Perl regular
expression. I keep this script in my <code>
~/bin</code>
 directory, so it
can be invoked just by typing its name. I also maintain a symlink called
<code>
nmf</code>
 to save typing!</p>

<p>
The above example, using the Perl version, would be:</p>

<pre>
<code>
alias copy_screenshot=&quot;xclip -selection clipboard -t image/png -i &bsol;$(nmf &#39;Screenshot_.*&bsol;.png&#39; ~/Pictures/Screenshots/)&quot;</code>
</pre>

<p>
In regular expressions <code>
'.*'</code>
 means "any character zero or
more times". The <code>
'.'</code>
 in <code>
'.png'</code>
 is escaped
because we need an actual dot character.</p>

<h2 id="conclusion">
Conclusion</h2>

<p>
The approach in both cases is fairly simple. Files matching a pattern
are accumulated, in the Bash case including the modification time. The
files are sorted by modification time and the one with the lowest time
is the answer. The Bash version has to remove the modification time
before printing.</p>

<p>
This algorithm could be written in many ways. I will probably try
rewriting it in other languages in the future, to see which one I think
is best.</p>

<h1 id="references">
References</h1>

<ul>

<li>
Glob expansion:
<ul>

<li>
<a
href="https://en.wikipedia.org/wiki/Glob_%28programming%29">
Wikipedia
article on glob patterns</a>
</li>

</ul>
</li>

</ul>

<ul>

<li>
HPR shows covering <code>
glob</code>
 expansion:
<ul>

<li>
<a
href="https://hackerpublicradio.org/eps/hpr2278/index.html">
Finishing
off the subject of expansion in Bash (part 1)</a>
</li>

<li>
<a
href="https://hackerpublicradio.org/eps/hpr2293/index.html">
Finishing
off the subject of expansion in Bash (part 2)</a>
</li>

</ul>
</li>

</ul>

<ul>

<li>
GitLab repository holding these files:
<ul>

<li>
<a href="https://gitlab.com/oss_tools/hprmisc">
<em>
hprmisc</em>
 -
Miscellaneous scripts, notes, etc pertaining to HPR episodes which I
have contributed</a>
</li>

</ul>
</li>

</ul>

</article>


