Hello, and welcome back to CS631 "Advanced Programming
in the UNIX Environment".  With this video, we
complete our discussion of interprocess communication.
We've come a long way from when we started with
signals as the most trivial, asynchronous method to
inform a process that some condition occurred to
full, bi-directional, connection-oriented
communications between two systems spread out over the
internet using a modern network protocol such as IPv6.

In our last two videos, we moved from simple
interprocess communications to a client-server
communications model, but we still used primarily
one-to-one communications.

That is, we had a single process accepting connections
and another process send messages.  But this isn't a
realistic communications model on the internet, so
let's see how we can build on this to move towards a
more typical client-server model with a server side
process capable of handling multiple simultaneous
clients.

So let's quickly revisit our last example and see
where we might run into problems:

Here's our streamreader code, where we create a
socket, bind it, listen, and then accept connections.

Suppose that we want to allow multiple connections.
We might consider opening additional sockets, and so
we might split out the functionality to create a
socket and to then handle a socket and accept
connections into subroutines, so that our new 'main'
might look like so:

We create two sockets, then handle both of them in
succession, hoping to allow multiple connections to
our server.

So now when we run this program, we will have two
sockets open.  They are port 65446 and port
65445.

So now we should be able to connect to either port and
send messages to our server.  Let's give it a try.

Here we create one client connection from our laptop
to port 65445 and send a message.

But we don't see our server receiving it!  Why is
that?

If we try to connect to the other port - 65446 -
and send a message, it is immediately received.

So what happened to the other connection?

It seems like we can send messages, but nothing
happens, because our program is still blocked inside
the first call to the 'handleSocket' function.

Only when we terminate this connection can our program
move on and handle the second socket, which it then
does immediately, as shown here.

So our model of creating two sockets and then trying
to process them sequentially is not going to work so
well -- the 'accept' in the first call to
'handleSocket' will block, even if there is already a
connection waiting on the second socket.

So what can we do when we have multiple file
descriptors that we wish to perform I/O on?

Well, the first option is what we've just seen: we
simply open the file descriptor, block, waiting for
I/O, and then move on to the next.  But that's clearly
not a very good option.

We could fork a new process for each file descriptor
we want to handle.

That would seem reasonable, but what if you want to
have information be passed between the different
channels?  Then you need additional IPC, so that seems
a bit cumbersome, too.

We could try non-blocking mode, something we'll
discuss in a bit more detail in a future lecture.  But
in a nutshell, if we mark the file descriptor as
non-blocking, then any call we make will return
immediately, and we can move on if it's not ready for
I/O.

Or we can use asynchronous I/O, another form of I/O
we'll discuss in more detail in the future as well.
This approach requires us to get notified by the
kernel when the file descriptors we're interested in
are ready for I/O.

But each of these options has at least some drawbacks:

Blocking forever clearly is undesirable, as we've
seen.

Using non-blocking mode, on the other hand, means that
if there is nothing connecting to us, we end up
busy-polling.  That is, we loop and test and loop and
test etc. etc., constantly making calls and wasting
CPU cycles.

Asynchronous I/O, while sounding promising, is often
times very limited, as we will see in the future.  But
let's instead consider another option:

We build a set of file descriptors we care about, and
then

call a special function that will tell us whether or
not any of them are ready for I/O.

And that is precisely what the select(2) syscall does.

We pass it the file descriptor number up to which we
want it to inspect.  That is, the descriptors from 0
through nfds - 1 is examined, which is why you will
always see the argument given being set as a file
descriptor plus one.

Then we pass in a file descriptor set for each of the
fields readfds, writefds, and exceptfds, and select(2)
will, respectively, check whether any file descriptors
in those sets are ready for reading, writing, or if an
exceptional condition has occurred.

Exceptional conditions include for example out-of-band
messages arriving on a socket, something we may
revisit in a future lecture on advanced I/O.

Finally, we can control for how long select(2) should
block when waiting for such conditions:

we can have it block forever by passing NULL

we can specify a granularity of seconds and
microseconds to wait

or we can have it not block at all by setting both
seconds and microseconds of the struct timeval to
zero.

When using select(2) for this form of synchronous I/O
multiplexing, we will operate on 

file descriptor _sets_, which we manipulate using the
provided FD_ macros, FD_SET, FD_CLR, FD_ISSET, and
FD_ZERO, which we'll see in action in a moment.

As mentioned, the read, write, and except fd sets are
used to check for read, write, and exceptional
conditions.

One thing to remember about reading from a file
descriptor is that EOF is a form of I/O -- that is,
the file descriptor set would be marked as ready for
I/O only for you to then get back 0, but that is not
really surprising.

If you need more granularity when blocking, you can
use the pselect(2) variation, which provides
nanosecond granularity as well as an option to specify
a signal mask to apply while blocking.

But let's see if we can use this in action to resolve
our problem of having multiple sockets that we'd like
to be able to handle without blocking communications
via the second socket while waiting on the first.

So here we updated our example program from before to
add a call to select(2).

We initialize a file descriptor set, then add our
sockets s1 and s2 to that set.

Then we call select(2), providing the max file
descriptor of the two plus one as the first argument.

After select returns, we can then test each socket
individually using FD_ISSET to see if either is ready
for reading.

So let's see if this helps us.

When we start, we again create two sockets, one
listening on port 65441 and one on port 65440.

Let's pull up the code while we make our connections.

Ok, now we can connect to our server on port 65440.

In our previous example, this connection to the second
socket would be blocked, as we'd be waiting for a
client to connect to the first socket before moving on
to the second, but now we notice that we were able to
pick up the second socket because select(2) identified
that this socket is ready for I/O, while the first
socket did not yet have any connections.

Of course we can also connect to the first socket and
have that connection not be blocked.

Ok, so here we've seen how the use of select(2)
allowed our program to pick up whichever socket was
ready, which is useful if you have multiple file
descriptors to juggle I/O on.

But more often, if you're writing a server, for
example, you wouldn't want to have multiple sockets
open.  Think about a web server -- it's listening on
port 80 (or on port 443 for TLS connections), but you
still have multiple clients connecting at the same
time.

So if we use select(2) in this example, we only have
one socket, but we won't be blocked waiting for
connections, meaning if there is no client connecting,
we can do something else.

This is what this example looks like.

Now running the program, we'll see that while we're
waiting for connections the server has determined that
no connections are ready, so it can print this
incredibly useful message while waiting.

When we connect to the server, it will now handle our
connection and not enter the other code block to print
the 'waiting' message.

Only after we disconnect does the server again get a
chance to do other things as now no file descriptors
are ready for I/O until the next client connects.

Ok, now let's see what happens when we have two
simultaneous client connections:

We connect from localhost, and of course that
connection is picked up right away.

But if we then initiate a second connection -- over
here from our laptop -- that connection remains
blocked since the first connection is still active,
and to have our second connection get handled, we need
to finish and terminate the first connection.

At that point, all the messages from the second client
are delivered, since our second socket was still in
the backlog of our server.

We'll see that over here in a second.

There. All messages from the second client were
immediately delivered.

So that's not going to be very helpful if you're
writing a web server: imagine a website that can only
handle a single connection at a time and you have to
wait until the other person has finished.

So that's no good.  Let's think how we could improve
on this...

Suppose we redo our program such that whenever a
connection is ready on our socket, we fork a new child
process, let _that_ handle the request and do whatever
the server is supposed to do while the parent goes
back to waiting for new connections...

For this, we first need to establish a signal handler
to let the parent wait(3) for any children that have
terminated, as otherwise whenever a client disconnects
we have a zombie left.

Then we create out socket and make our select(2) call.
Since we might get interrupted when a client
disconnects, we simply ignore those conditions and
loop around, calling select(2) again.

Our function 'handleSocket' now gets invoked whenever
there's a client connecting; it can then accept(2) the
new connection, but instead of handling it directly,
we now fork a new process and let _that_ process
handle it.  The parent returns and can then proceed to
pick up the next connection whenever it comes in.

The 'handleConnection' function then does what we're
used to, with the difference to before being that
after we are done and disconnect, we exit.

Ok, let's see if that helped us.

The server is now listening on port 65427.  While no
client connections are ready, the server can go and do
other stuff, and when our first client connects, of
course it is being handled as expected.

But now note that while we are communicating with the
first client, the server can _still_ go and do other
things.  That is, it's not blocked from moving on
since the client connection is now handled by a
dedicated child process.

And while we are still connected to the server in the
upper right, we can now initiate a second connection
in the bottom right, and _this_ time, the server
immediately picks it up.

So now both clients can simultaneously communicate
with the server without being blocked, and the server
is still ready to accept new connections.

If we terminate the server, all client connections are
of course terminated as well.

Ok, so we've seen how to handle I/O multiplexing in
this video:

We can use select(2) to check a set of file
descriptors for specific conditions, which allows us
to either juggle multiple file descriptors or sockets,
or to avoid blocking on a single socket and do other
things while we wait for clients to connect.

We've also seen how we can then handle multiple
simultaneous clients on a single socket by spawning
child processes or separate threads to have them
perform the interactions.  This works well if each
child process does not need to communicate with any of
the other clients, and is a common pattern in standard
server design.

Sometimes you may come across server programs that
pre-fork a bunch of processes to handle the
connections, which may improve on this approach.

There are a few other alternative interfaces an
options for being notified of when I/O is ready or,
more generically speaking, an event occurred and you
want to handle it.  Each call or API listed here has
its own advantages, and you can read up on their use
cases and comparison with others on your own.

They are:

poll(2)

epoll(2) on Linux

The kqueue(2) kernel event notification mechanism
available on the various BSD systems, including macOS.

And libevent or libev, which may wrap some of these
interfaces to allow for a platform agnostic I/O
multiplexing mechanism.

Each of those tries to overcome some of the
shortcomings of the others in an effort to improve
efficiency when handling large numbers of simultaneous
connections, which isn't a trivial problem to solve.

This link over here describes the problem space and
the various approaches well.


Alright, I think that about concludes our coverage of
interprocess communications, and with the discussion
from this video, you're now well prepared to begin
work on your final group project: writing an HTTP
server.

We'll discuss the project in class, and then continue
our video lecture series with a look at daemon
processes and shared libraries.

Until next time - thanks for watching.  Cheers!