CS615 -- System Administration -- EBS Backup

Summary:

In this group project, we will work together to develop the ebs-backup tool as specified in this manual page.

In the process, we will (hopefully) learn a few things about cloud storage, using the AWS API, writing modular system tools, using command pipelining, as well as cooperative software development practices.

Distributed Collaborative Software Development

We will be using git to collaborate on the software project, but we will notably not use GitHub. Instead, we will use the git command-line tools and communicate via the class mailing list and Slack to emulate professional or open-source development flows.

Our central, authoritative repository will be cs615asa.netmeister.org:~git/ebs-backup, and each student will have their own forked repository at cs615asa.netmeister.org:~<your-username>/ebs-backup, from which you can issue pull requests.

For this, you will need to submit a public SSH key to me to have your account created and configured:

ssh-keygen -f ~/.ssh/cs615 -P ''
Attach the file ~/.ssh/cs615.pub to an email to jschauma@stevens.edu.

Once your account has been provisioned, you can then follow the workflow below for your code work.

Coding flow

First, add an SSH configuration section to your ~/.ssh/config:

cat >>~/.ssh/config <<EOF
Host cs615asa
        HostName cs615asa.netmeister.org
        User $USER
        IdentityFile ~/.ssh/cs615
EOF

The SSH host keys of the system are available here. You can append these directly to your ~/.ssh/known_hosts via

curl https://stevens.netmeister.org/615/cs615asa-known_hosts >> ~/.ssh/known_hosts
They are, of course, also provided via DNSSEC signed SSHFP records, so you can also add VerifyHostKeyDNS yes to your ~/.ssh/config entry for this host.

Then, clone the authoritative upstream repository:

git clone git@cs615asa:~git/ebs-backup

Next, add your fork as a second remote repository:

git remote add $USER cs615asa:ebs-backup

Now you can work in this directory as usual. You can git commit your changes and

git push $USER main

Submitting a Pull-Request

When you want to submit a pull request, you'd run these commands:

git request-pull -p HEAD $USER@cs615asa:~$USER/ebs-backup main > /tmp/PR
git diff $USER/main origin/main >> /tmp/PR
and submit the resulting file /tmp/PR to the class mailing list or share it with your team mates on e.g., Slack for review.

A pull request needs to be reviewed and approved by at least one of your peers. As such, you may not commit directly into the 'origin' tree on your own behalf, but you may merge somebody else's pull request, but you'll have to be careful not to accidentally commit your own changes.

For example, if user1 had submitted a PR you'd like to merge, you'd then

git remote add user1 cs615asa:~user1/ebs-backup # add their fork as a remote; only necessary the first time you're merging code from user1
git pull user1 <hash>
git push origin main

You may wish to pull the PR into a branch to avoid accidentally merging any outstanding changes you might have before pushing the change into origin.

Work Areas

This project includes a number of work areas, to which each student can contribute to the best of their abilities, divided into the following directories:

  • src -- writing the actual code to implement the functionality
  • doc -- any and all documentation around the project, including how to use the tool, how to install it, how to collaborate, etc.
  • pkg -- anything relating to packaging the software for the supported platforms
  • test -- unit tests for all functionality

Program behaviour and safe assumptions

The program will not require any modification of the environment (i.e., you can assume the user has his/her environment set up for EC2), and exection of the program will be exactly as outlined in the manual page ebs-backup(1). Please read this manual page carefully.

In addition to what the manual page notes, you may further assume the following:

  • the default EC2 security group of the invoking user allows any and all connections to instances placed in that group
  • the user's SSH config allows for connecting to hosts ending in amazonaws.com without explicitly specifying an SSH key or dealing with host keys etc.
  • the default AWS region for the user is 'us-east-1'
  • there are no other AWS defaults configured

Deliverables and Due Date

This is a group project and due at the end of the semester. Your participation throughout the semester will be used as a basis for your grade. On 2021-05-10, I will clone the repository; any commits pushed after that date will not be considered.

Aspects important for a successful project grade:

  • functionality
  • code quality and cleanliness
  • ease of installation on different platforms
  • completeness of the test harness
  • documentation
  • code reviews
  • project discussions


[Course Website]