[Original document date: 11 October 2019] # Some really cool things I discovered in the date(1) command and related Some cool features I have discovered in the `date(1)' command that I don't want to forget. The reason I discovered these is because of the rubbish that is dnf, the package manager for Red Hat systems. Because I am in the process of a server upgrade and the fact I have had nothing but trouble with reinstalling I am looking into CentOS 8. In fact I actually installed CentOS 7 again but still at least this was instructional here. Anyway unfortunately CentOS 8 have migrated to the dnf package manager. It's touted as an improvement but one thing is certain and that is the logging is a serious regression: far too verbose (with seemingly no way to disable it) and not having a simple log file that the yum package manager did have. Instead those prat developers think using the utility itself should suffice. More crap from Linux developers - much like systemd using (by default) binary logging which means simple command line utilities we're all (except maybe the developers of these atrocious software that has subjugated Linux sadly and ironically working for Red Hat) no longer are helpful. Instead more utilities are needed! Whatever. That's a digression and not important. One of the problems is that the fact the logging is not like it used to be (i.e. with yum) logwatch did not work. I found a patch and it works - after I modified the timestamp format. However because logs should be local server time - for accountability - the fact the idiots made it UTC means that everyone - meaning more people than those in - who's not in UTC have incorrect logging information! So then how can we get the date format to be local server time? What does it look like? It looks like thus: 2019-10-11T14:41:00Z Yes that's very helpful. Well a comment on the bugzilla report about dnf logging (inadequacies) stated that it's ISO 8601 UTC. The trouble of course is the offset differences. Worse is that not all time zones are even on the hour; in Australia there is a time zone on the half hour and another at 45 past, for two examples. Now I encountered an interesting way to modify the TZ variable for `date(1)' that changes the offset - though it wasn't explained so I experimented. The way it works is: $ TZ=UTC6 date For example. This would print as if it was UTC but 6 hours behind. Okay but then how to get the offset on the system? That's the +%z format. However it looks like this: [-+]XXXX Where the first one is a 0 and then the hour difference and then the minutes. For example if you do: $ TZ=Australia/North date +%z +0930 Because it's 9 hours and 30 minutes ahead of UTC. If however you are to set a time zone behind UTC it would look like: $ TZ=Canada/Pacific date +%z -0700 But then does this mean you can simply replace (in the example above) the '6' (from the UTC) say -0700 and get UTC - 7 hours? No. Actually at this time: $ TZ=UTC date ; TZ=UTC-0700 date Fri Oct 11 17:53:33 UTC 2019 Sat Oct 12 17:53:33 UTC 2019 Observe that only the day has changed. And not backwards but forwards! It's currently Friday 11 October 2019 in UTC but -0700 it's Saturday 12 October 2019! I played with different variations and none of them seemed to do it. So what is there to do? I looked at the source code of course! That's when I discovered something very interesting. I will not include the code because it doesn't matter since they also added a comment which here I cite: /* Guard against falsely reporting errors near the time_t boundaries when parsing times in other time zones. For example, suppose the input string "1969-12-31 23:00:00 -0100", the current time zone is 8 hours ahead of UTC, and the min time_t value is 1970-01-01 00:00:00 UTC. Then the min localtime value is 1970-01-01 08:00:00, and mktime will therefore fail on 1969-12-31 23:00:00. To work around the problem, set the time zone to 1 hour behind UTC temporarily by setting TZ="XXX1:00" and try mktime again. */ Now that is interesting; adding TZ=XXX1:00 works? Looking at the code and just understanding how these things work it was obvious that you could do this at the command line. Okay but what about off hours? Yes it works: $ TZ=UTC date ; TZ=XXX1:30 date Fri Oct 11 17:59:48 UTC 2019 Fri Oct 11 16:29:48 XXX 2019 Observe that it's an hour and a half behind on the second line of output. Also note that the time zone has changed to 'XXX'. And yes you can make it whatever you like: it just prints out what's passed to it (converted to the abbreviation). So for instance you could if you wanted to change it to ELF: $ TZ=ELF date Fri Oct 11 18:01:48 ELF 2019 But then how does it determine the *real* time to print? Well actually that's an interesting thing. If say you had exported TZ to be UTC: $ export TZ=UTC; date ; TZ=ELF date Fri Oct 11 18:05:20 UTC 2019 Fri Oct 11 18:05:20 ELF 2019 If you see the time is the same: in other words UTC! But then what if TZ is unset? $ export TZ=UTC; date ; TZ=ELF date ; unset TZ ; TZ=ELF date Fri Oct 11 18:06:41 UTC 2019 Fri Oct 11 18:06:41 ELF 2019 Fri Oct 11 18:06:41 ELF 2019 In other words still UTC. In the tzset(3) function (also the command, tzset(1)): If the TZ variable does appear in the environment, but its value is empty, or its value cannot be interpreted using any of the formats specified below, then Coordinated Universal Time (UTC) is used. I seem to recall though that another function does something like this. However given my experiments here I don't think that's so clear. Well what about the dnf issue? Actually date(1) (at least GNU date) is pretty amazing with parsing date and time. All you have to do is pass in that format above (in the log) and it will convert it to local time. For instance: $ TZ=Canada/Pacific date -d 2019-10-11T14:41:00Z Fri Oct 11 07:41:00 PDT 2019 To invert it: $ TZ=UTC date -d "Fri Oct 11 07:41:00 PDT 2019" Fri Oct 11 14:41:00 UTC 2019 Invert at least in so far as converting to UTC. So what about dnf.rpm.log (the only somewhat - and only in comparison to the other dnf files is this true - sane file, and I stress 'somewhat') and the time? An example looks like this (minus the irrelevant lines that can be easily filtered out with grep -v): # date -d $(grep Installed /var/log/dnf.rpm.log | tail -n 1 |cut -d' ' -f1-1) Would convert the UTC time in the last line that includes the text 'Installed' in the log file. You could indeed specify the TZ to change it. Still this does not include the other information but I decided to write this instead of resolving that fact lest I forget these interesting things. # Bonus tips for the date command (some are GNU specific features) For these I am exporting TZ to be UTC: $ export TZ=UTC ## If you want the number of seconds since the UNIX epoch: $ date +%s 1570818088 What use is this? Well actually it's quite useful. In programming you can keep track of exactly when an account was created (for example). But then what if you want to convert it to the date and time in English? GNU date can do this (but it's certainly not the only way to do it it's just an easy way): $ date -d @1570818088 Fri Oct 11 18:21:28 UTC 2019 Let's take it a step further though: what if you want to do command substitution? Throughout this document I used the ';' feature of the shell so as to have the exact time match in order to make things consistent and easier to follow. # What is the exact time right now? This is a very cool feature of GNU date (actually this part might be POSIX, I don't recall immediately at hand) that can be taken much further (which I will get to below): you can specify what you want in English (in fact the touch(1) command also has this feature). This includes 'now' but much more: $ date ; date -d now Fri Oct 11 18:26:48 UTC 2019 Fri Oct 11 18:26:48 UTC 2019 As you can see using `date' by itself is the same as `date -d now'! But what if you want to subtract time? If you look at the man page for date: DATE STRING The --date=STRING is a mostly free format human readable date string such as "Sun, 29 Feb 2004 16:21:42 -0800" or "2004-02-29 16:21:42" or even "next Thursday". A date string may contain items indicating calendar date, time of day, time zone, day of week, relative time, relative date, and numbers. An empty string indicates the beginning of the day. The date string format is more complex than is easily documented here but is fully described in the info documentation. What about that info? I'd like to share an amusing quote that the info documentation includes in the relevant section which I am including at the end of this document. Anyway I'm not going to include the documentation in full but rather give an excerpt of how flexible it really is (rules follow and there is more information before this subsection): A "calendar date item" specifies a day of the year. It is specified differently, depending on whether the month is specified numerically or literally. All these strings specify the same calendar date: 1972-09-24 # ISO 8601. 72-9-24 # Assume 19xx for 69 through 99, # 20xx for 00 through 68. 72-09-24 # Leading zeros are ignored. 9/24/72 # Common U.S. writing. 24 September 1972 24 Sept 72 # September has a special abbreviation. 24 Sep 72 # Three-letter abbreviations always allowed. Sep 24, 1972 24-sep-72 24sep72 I'll include a few more that pop into my head including about the future: $ date -d tomorrow Sat Oct 12 18:39:03 UTC 2019 $ date -d yesterday Thu Oct 10 18:39:06 UTC 2019 $ date -d 'next week' Fri Oct 18 18:39:11 UTC 2019 An interesting titbit I just noticed: More generally, the time of day may be given as 'HOUR:MINUTE:SECOND', where HOUR is a number between 0 and 23, MINUTE is a number between 0 and 59, and SECOND is a number between 0 and 59 possibly followed by '.' or ',' and a fraction containing one or more digits. Alternatively, ':SECOND' can be omitted, in which case it is taken to be zero. On the rare hosts that support leap seconds, SECOND may be 60. What is so interesting? Well I can imagine that some hosts do not support leap seconds but time is vitally important and even a second off can cause massive problems. Never mind clock skewing causing problems back in 2012 (30 June if I recall correctly) some versions of the Linux kernel had a bug that did not deal with the leap second (if I recall correctly there was also one at the end of the year and pretty sure also one in 2016) - with a result of consuming cpu% and causing hosts to stop responding. There was a fix but this required being able to run a command (setting the time) and I believe also it might have been on a weekend. # Try the --debug option $ date --debug -d now date: parsed relative part: today/this/now date: input timezone: TZ="UTC" environment value date: using current time as starting value: '18:43:43' date: using current date as starting value: '(Y-M-D) 2019-10-11' date: starting date/time: '(Y-M-D) 2019-10-11 18:43:43' date: '(Y-M-D) 2019-10-11 18:43:43' = 1570819423 epoch-seconds date: timezone: TZ="UTC" environment value date: final: 1570819423.030376926 (epoch-seconds) date: final: (Y-M-D) 2019-10-11 18:43:43 (UTC) date: final: (Y-M-D) 2019-10-11 18:43:43 (UTC+00) Fri Oct 11 18:43:43 UTC 2019 # The quote on the insanity of time Our units of temporal measurement, from seconds on up to months, are so complicated, asymmetrical and disjunctive so as to make coherent mental reckoning in time all but impossible. Indeed, had some tyrannical god contrived to enslave our minds to time, to make it all but impossible for us to escape subjection to sodden routines and unpleasant surprises, he could hardly have done better than handing down our present system. It is like a set of trapezoidal building blocks, with no vertical or horizontal surfaces, like a language in which the simplest thought demands ornate constructions, useless particles and lengthy circumlocutions. Unlike the more successful patterns of language and science, which enable us to face experience boldly or at least level-headedly, our system of temporal calculation silently and persistently encourages our terror of time. ... It is as though architects had to measure length in feet, width in metres and height in ells; as though basic instruction manuals demanded a knowledge of five different languages. It is no wonder then that we often look into our own immediate past or future, last Tuesday or a week from Sunday, with feelings of helpless confusion. ... --Robert Grudin, 'Time and the Art of Living'. Well I suppose that's a good way of putting it for most people. I don't know that it applies to me but it does to many people I am sure. Anyway there are some fun and interesting things about the date(1) command.