Hello, and welcome back to CS631 "Advanced Programming in the UNIX Environment". In this video, we will conclude our coverage of advanced I/O topics with a quick look at asynchronous and memory-mapped I/O. Both of these topics are laregly mentioned here for completeness's sake and as a nudge to get you to begin researching additional topics moving forward. --- So what exactly is Asynchronous I/O? Let's compare to the other forms of I/O we've already seen. Programs that perform any I/O are usually performance bound by the efficiency of the I/O done, rather than being CPU bound. Actual I/O is an order of magnitude slower than handling data in memory or performing CPU instructions, so we are looking to find ways to increase efficiency in this area. - In Week 2, when we began our discussion of regular I/O, we covered the standard read(2) and write(2) system calls, which we saw are synchronous and blocking, meaning we have to wait for them to complete before we can do anything else. We issue the read(2) system call, which then leads to a context switch into kernel space where the kernel performs the actual I/O. Whenever the data has been fetched, the kernel delivers it into userspace to us, and in the mean time, we just sat there, twiddling our thumbs, unable to do anything else. Remember, while this may seem instantaneous to us, from the computer's perspective there elapsed a lot of time that would have been used to complete other work while we were waiting for the I/O. --- In a previous video from this week, we saw ways to avoid blocking when I/O cannot be performed by using non-blocking mode. If we specified O_NONBLOCK, then the kernel would immediately return to us when it can't complete the I/O. When this happens, we don't give up -- we still want to get the data, presumably, so we try again and again, context switching multiple times until we finally can get the data. This looks rather inefficient and expensive. --- In Week 9, we discussed ways to multiplex I/O on multiple descriptors by using the select(2) system call, thereby accomplishing a form of asynchronous I/O: while we can inspect multiple file descriptors and be notified when one of them is ready, the select call itself is still blocking (although we have an option to specify no wait time - but that only means that we told it to block for zero seconds). So we are minimizing the number of context switches, which is a good thing and certainly sufficient for many use cases, but if you want high-performance I/O, you do not even want to wait for select(2). --- So what we're looking at right now is then true asynchronous, non-blocking I/O. We can issue the I/O request, and then go and do other stuff. Whenever the I/O operation is complete, we get notified and can consume the data right away. --- So in a nutshell, this is what we're looking at. read(2) and write(2) perform synchronous, blocking I/O. read(2) and write(2) with O_NONBLOCK perform synchronous, non-blocking I/O. We saw I/O multiplexing using select(2) and poll(2) as a form of asynchronous, but still blocking I/O, and finally we're looking at asynchronous non-blocking I/O. Now having true asynchronous, non-blocking I/O seems like a rather useful thing to have, but it's worth noting that on the other hand it is far from trivial to implement. --- Before POSIX specified asynchronous I/O, we began with what we just mentioned: somewhat semi-async I/O via select(2) or poll(2). - System V had some form of true asynchronous I/O, but that was limited to "STREAMS", a framework developed on and for System V for character devices, network communications and IPC. STREAMS, however, did never get adopted by the BSD lineage nor in Linux, and nowadays is marked as obsolete. You probably won't come across STREAMS other than in historical Unix jeopardy or as an obscure reference when somebody wants to sound really smart in a meeting or on a mailing klist somewhere for extra neckbeard credits. - But we also have some form of asynchronous I/O that's derived from the BSD lineage and now present in most modern Unix versions. This form is, however, limited to terminals and network only. In this approach, you can mark a file descriptor as being ready for async I/O by setting O_ASYNC in the open(2) flags, or F_SETOWN via fcntl(2). When I/O becomes available, your program will then receive SIGIO or, in the case of network communications, SIGURG, where that event is generated when a TCP packets arrives with the out-of-band flag set. This, however, is still not full fledged async I/O, and so --- we also have a full POSIX compliant implementation available on NetBSD, which you can read up in the aio(7) manual page. This form of asynchronous I/O is implemented in the kernel but provided to you via the POSIX Real-time Library, librt, which we've previously seen when we talked about POSIX message queues. Here, - the kernel managed queued I/O events using an "asynchronous I/O Control Block" as its basic operational unit, and the library functions are all prefixed with "aio", such as "aio_read(3)", "aio_write(3)" etc. - Notification of a calling process happens via a signal or sigevent structure, and - the calling process still has the option to block. Unfortunately, the different implementations of POSIX AIO vary slightly across Unix versions, making portable implementation difficult. - For example, Linux has multiple implementations: a POSIX aio implementation via GNU libc as well as a separate library -- libaio. In addition, there are plenty of discussions to be found online about the merits of asynchronous I/O and whether we really need it, which is why we don't provide full code examples here and leave that as an exercise for you to satisfy your own curiosity. So let's move on to... --- Memory-mapped I/O. Memory mapped I/O is just what it sounds like -- instead of using read(2) and write(2) to go out to disk and read or write the data, we are - grabbing a chunk of data from a file and map it into our memory space in the process. We can then perform I/O directly on that region, and the changes will be reflected on disk when we sync it. This makes for a very efficient method of performing I/O, although of course it requires sufficient memory to be available. That is, the data actually has to fit, or you're thrashing your memory, possibly leading to your process swapping. --- So how do we do map pages from a file into memory? We call mmap(2). mmap(2) maps 'len' bytes of data starting at the given address and returns to you a pointer to the mapped region. When mapping the data into memory, you can specify a protection to restrict what can be done with the region. The protections available are quite self-explanatory: PROT_READ to allow reading, - PROT_WRITE to allow writing, - PROT_EXEC to allow execution, and - PROT_NONE to request no access. On NetBSD, you can also change these permissions later on via the mprotect(2) system call, but that is non-portable. Next, you - specify either MAP_SHARED or MAP_PRIVATE, depending on whether you want other processes to have access to the mapped region, combined with any additional flags you may need -- see the manual page for a description of the available options. --- Now as we've mentioned a few times before, having context switcheѕ between user space and kernel space is expensive, and standard I/O suffers from this. Using memory mapped I/O can thus significantly boost your performance here. The author of our course textbook did run a benchmark comparison on copying data using read(2)/write(2) versus mmap(2)/memcpy(2), and found some rather notable differences. Now this benchmark was done a few years ago, but the findings still should remain relevant. Specifically, we notice that memory mapped I/O appears to perform between 30 and 40% better than standard I/O. --- Ok, to conclude this quick summary of asynchronous and memory mapped I/O, let me suggest a few exercises for you: - If you look up the aio(7) manual page on a Linux system, it includes a nice code example. Go ahead and run that, then see if you can take the same code and run it on NetBSD. If not, what changes are necessary? Are the outcomes the same? - Take your HW1 from the beginning of the semester and rewrite it to use memory mapped I/O. This isn't very difficult, but a good exercise to internalize how this works. Make sure to account for the edge cases, such as copying a zero-sized file. - As just mentioned, the benchmark comparison of mmap(2) versus read(2)/write(2) done by the textbook author is a few years old, but now you have two implementations of the cp(1) program that use both approaches -- perform a benchmark test and see what the outcomes are. Remember to copy random and distinct data, to ensure the filesystem buffer cache doesn't help your standard I/O implementation. - And lastly, as I've said repeatedly, make sure you're comfortable reading code. The NetBSD operating system provides you with all the source code to all the tools -- take a look at the cp(1) program, you should find that it makes use of mmap(2), but not for all I/O. Research why that might be the case, and compare to other Open Source implementations. Alright, this brings us to the end of our series on advanced I/O. In our next set of videos, we'll discuss how we can restrict processes from interfering with one another. Until then - thanks for watching! Cheers.