Hello, and welcome back to CS631 "Advanced Programming in the UNIX Environment". Today, we'll start looking a little bit closer into what our systems think of "time", something that experts most accurately describe as a big ball of wibbly wobbly timey wimey... stuff. I know, I know, you thought you were done with the struct stat, yet here we are again... --- So let's look at the struct stat again. We have at least three time-related fields, the file's atime, mtime, and ctime, with the atime representing the time of last access. Well, - data access, really. Recall our visualization of the data structures representing a file: accessing the data blocks, pointed to from the inode in the vnode structure is what we're talking about here. Then there's the time of last modification - again, _data_ modification and the time of last file status change, the ctime. This represents a change in the struct stat itself, more or less. All three of these times can be displayed using the ls(1) command, as I'm sure you know by now. - --- So, trivial example: by default, ls(1) uses the file's mtime when displaying or sorting by date. --- If you specify the '-u' flag, you get the atime. In this example, the atime for all these files is identical because I recently re-created the apue-code tarball: archiving the files requires reading them, so no surprise, all files have their atime updated. --- To display the time of last inode change, we use '-c'. --- Let's take a look at how the timestamps change when we perform common operations. We create a new file and use stat(1) to display the file times. The filesystem in use here supports an inode "birthtime", so we display that, too. Note that all four times shown here are identical: this should make sense, since creating the file implies access and modification of the file data as well as modification of the new file's inode data. --- If we then cat(1) the file, we read it, meaning we are accessing the file data blocks, and the atime gets updated. --- If we append to the file, we are obviously writing data to the file, so the mtime is updated. We are extending the file in size, so at the very least the st_size field of the inode changes, so our ctime is updated as well. So far, so good. --- Those operations seemed straight forward. Now let's look at what happens when change anything _about_ the file. Creating a second hard link to the file increments the st_nlink field, so necessarily requires an update of the ctime. But since we didn't do anything to the data, the atime and mtime remain unchanged. --- Similarly, changing permissions only modify the st_mode, and again the ctime is updated, but the atime and mtime remain the same. --- Changing file ownership follows along the same lines with the same results. --- Ok, now we have a file that's owned by 'fred' with no write permissions for anybody but the owner. The 'touch' command by default touches a file's atime and mtime - but not surprisingly you can modify these members of the struct stat of fred's file without write permissions. --- ...but if you _do_ have read and write permissions, you can change those timestamps. That makes sense, since if you have read permissions, you could just open the file and read a few bytes, and that would update the atime, and if you have write permissions, you could write to the file, and that would update the mtime. Either way, making those changes also update the ctime, since now something about the file has changed. --- But the touch command also allows you to set these time stamps to _any_ date, not just the current time. Let's try to set those times to the epoch, but that fails, even though we have write permissions. That also should make sense: we have write permissions to the file, but we are not the owner, so modifying those timestamps to arbitrary dates seems reasonable to restrict to the owner. --- If we _are_ the owner, we can set those timestamps. Note that in this case the birthtime was changed as well! Whenever the mtime is modified to a date that's _before_ the current birthtime, the birthtime is updated to match, to avoid us appearing to require timetravel to modify a file before it is created. Also note that the ctime was updated when we set the other times. There is no option for us to set the ctime -- think about it: changing the times is a change of the file status information, and the ctime tracks whenever the file status information changed, so if you could update the ctime to an arbitrary date, it would immediately have to be set to the current time to reflect that change in the file status. --- We can also use touch(1) to selectively update one or the other times, atime or mtime, and each time the ctime will be updated as promised. Once for the atime... --- ...and once for the mtime. --- Ok, so we saw how the mtime was updated when we appended data to the file, which included a change to the st_size field. But what if we don't append data, and merely change some data in place? Here, we use our good friend ed(1) to flip a single byte, and our various time fields are... - ...updated, all three of them. This is because to make the change, we had to first read the file (which updated the atime), make the change, then write the file (which updates the mtime), and so all three times have been changed. --- Here, let's read the file again, and this time keep a close eye on the ctime. Reading the file changes the atime, as we observed before, while the ctime did not change. But wait - the ctime should be updated whenever anything _about_ the file changes, so a change to the atime should trigger an update of the ctime, shouldn't it? After all, that's exactly what we saw when we called 'touch'! Well, it turns out that ctime and atime are a little bit trickier than that. A mere update of the atime as a side effect of any of the file operations -- such as a read(2) -- does not update the ctime; an _intentional_ modification of the atime by way of e.g., touch(1), however, does! POSIX has in all system calls a note about which of the time fields it should update (if any). --- By the way, we just used the ed(1) line editor to edit the file in place. This wasn't done to impress the Unix young 'uns -- well, not only, anyway. One might think to use the sed(1) command with the '-í' flag to edit a file in place. Here, it'd look like that: - Now what's that? All times are the same now, even the birthtime! That suggests... - That's right -- this file is a different file now! 'sed -i' does not actually edit a file in place: it creates a temporary file, then replaces the current file with the temporary file. Depending on the version of sed(1), that may include creating a new file or truncating the old file. Anyway, so let's think a bit more about the atime. Any read on any file will trigger an update of the atime, which the filesystem then has to write to disk. That means that you constantly are triggering comparatively expensive I/O. This not only impacts overall I/O performance, but especially on solid state drives, this can actually reduce the lifespan of the drive. --- So let's find a way to avoid this. Many filesystem support a mount option to disable atime updates altogether - noatime. Let's try this out with our separate disk. Here, we see a file with the various times as displayed. --- Reading the file here will - _not_ update the atime. --- Even trying to explicitly update the atime via the touch(1) command does _not_ update the atime. 'noatime' means 'noatime'. But! The touch(1) command, even though it did not actually change the atime, did not fail, and the ctime was updated as if the atime had been changed. That is, the touch(1) command is not aware of the filesystem's mount options or behavior: it issues a call to make the change, and that is all. So why don't we use 'noatime' everywhere? Well, sometimes it _is_ useful to know when a file was last accessed, and some programs do in fact depend on that. For example, many mail programs use the comparison of the mtime to the atime to determine whether or not there is new mail in a given file. --- But there are other ways to solve the dilemma with the atime and its impact on performance. Let's take a look at a Linux system. Here, we see the output of the stat(1) command differ between the different Unix versions; as usual, compare the manual pages for the details. But we see that reading the file here does not update the atime - so perhaps this filesystem also has the 'noatime' option enabled? --- Writing to the file, like we did before, updateѕ the mtime and ctime here, but not the atime, even though clearly we did read the file... --- ...but, and this is where it gets interesting, reading the file _now_ _does_ update the atime! This is the default behavior on most Linux systems these days, and the filesystem mount option is known as 'relatime', or "relative atime". That is, the atime is only updated if the mtime or ctime is newer than the atime, or if the atime is older than 24 hours. This allows the filesystem to continue to support applications that depend on the atime change, but to not have to thrash the disk with I/O on read-only operations. Different versions of Unix and different filesystems may or may not support this or other, similar options. --- Ok, so having seen the touch(1) command in action, we then might ask just how it manages to perform the actions, and the answer is provided by the utimes family of functions. As with other calls, we get the familiar variations: lutimes to operate on a symlink, futimes to operate on a file descriptor, and, a utimensat variation. The second argument to utimes is a two-element array of time values, one for the atime and one for the mtime. - If you pass in NULL, then you get the current time, and for that write permissions are sufficient, as we've seen. - If you do pass a time value, the function sets the atime and mtime accordingly, but requires ownership of the file in question. - Either way, as we also observed, the ctime is always updated by this function. - Finally, there are variations that allow for nanosecond precision, and the utimensat call supports those via the timespec instead of a timeval argument. --- Alright, time for an intermission. Let's recap: - The atime is a bit odd; it desribes the time of last _data_ access. - but its logic seems to have a number of negative side effects, so that - we have found ways to work around that via mount options and different implementations. - The mtime makes a bit more sense: whenever we write to the file, it gets updated. - The ctime indicates when anything about the file has changed, and it's the only field we can't influence -- the different system calls will trigger an update automatically. - Calling utimes(2) to set the timestamps requires file ownership, and with all of what we covered here - you can now implement the touch(1) command. Or at least take a look at the sources from the NetBSD operating system. - Oh, and by the way, bananas really are berries, botanically speaking. Anyway, that's it - we're out of time for today. But we're not done with time by far. More on this big ball of wibbly, wobbly, timey, wimey stuff in our next video. Until then - thanks for watching! Cheers! cd 04 ls -l ls -lu ls -lc echo bananas > file ls -l file ls -lu file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file All the same, that makes sense: - we just created the file - we accessed it to write to it - we modified it - a whole lot changed about the file, so ctime changes cat file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file only atime changed, because by calling read(2), we accessed the data blocks it. Note: merely calling open(2) does not count as "access", since atime is last _data_ access. echo are berries. >> file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file a time remains the same, we didn't access the data; we merely opened the file in append mode, causing a seek to the end before writing data. Writing data modifies the file, so the mtime gets updated. Writing the data changed (at least) the st_size of the file, so metadata changed, so the ctime changed. ln file file2 stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file Creating a second link to a file changes the st_nlink count, so updated the ctime. chmod go-r file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file changing permissions changes information about the file, so updated ctime. touch(1) lets you modify the atime and mtime of a file: touch file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file touch -t 197001010000 file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file Note: birthtime is updated because mtime is prior to previous birthtime. this changes the inode data, so ctime is updated touch -a -t 197111111111 file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file this changes the inode data, so ctime is updated touch -m -t 202002020202 file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file ctime updated What about writing to the data without changing the size? ( echo 1s/b/B/; echo wq; ) | ed -s file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file atime, mtime, ctime are udpated because: we opened the file and read it -> atime we modified the file -> mtime modifying the mtime changes the inode data, so we update ctime (btw, can't use "sed -i" here, because "-i" actually writes to a new file, then moves the file into place; we observe this by noticing a changed birthtime) Anyway, so updating the mtime updates the ctime. But: when we cat the file, atime gets changed, but not ctime? sudo mount -o noatime /dev/wd1a /mnt stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file cat file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file touch file stat -f "birth: %SB size: %z%natime: %Sa inode: %i%nmtime: %Sm%nctime: %Sc%n" file On Linux, relatime: write via ed does not update atime --- utimes: note: if you have read access, you can always set the atime by opening it and reading a byte