Page tree
Skip to end of metadata
Go to start of metadata

Sections marked with green color describe output relevant for your learning diary and count for points towards completing the course. 

The paja instructors check your exercises (in the report) on the spot.

Exercise 1

Sharpening your tools

A quick refresher on moving around the command shell is provided by Ubuntu:

EXERCISE 2

Man pages

The manual pages are your best friend not only on this course, but during many other periods of life in general.

Try it by executing the command man followed by the program you want to read the manual of:

man man -> manual of the man command
man mkdir
man cd
man ls

EXERCISE 3

Directory setup

The solutions are meant to be iterative in the sense that a student builds up on the previous program code to achieve more and more difficult tasks.

Open a terminal to your home directory, and create a LinuxFundamentals2015 folder using the terminal. Within that directory create a Week1 folder.

Put in your report, the command lines for creating the directories

EXERCISE 4

Identity shift

The alias command can make life simple by abbreviating very long command line argument lists into shorter commands.

Mask the standard ls command by creating your own alias, using your favourite command line arguments for ls.

Put in your report

  • the alias of your favorite command line for ls
  • an alias cman which uses the Chromium browser to read man pages.

EXERCISE 5

Permanent changes

Making things permanent is necessary as no one wants to recreate their favourite aliases or other tweaks each time they open a terminal.

There are many ways to achieve it. The three main possibilities are the files ~/.bashrc~/.bash_profile, and ~/.bash_aliases. Use .bash_profile for now.

For more information about the different possibilities, try :

man bash

We will soon figure out the difference between login and non-login shells.

Put in your report, a copy of your ~/.bash_profile and make a copy to this week's home folder

EXERCISE 6

Remote shells with SSH

ssh is a very useful utility. One of its uses is to start remote sessions to hosts running a variant of the ssh daemon.

One can use ssh either with passwords, private and public keys, or a combination thereof. The last variant is the most useful.

Look at the list of Ukko computing cluster nodes: http://www.cs.helsinki.fi/en/compfac/high-performance-cluster-ukko

Choose your favourite node from the list and log on to it with ssh.

Put in your report, a copy of the alias command to log on to your favourite Ukko node with ssh

Remote invocations with SSH

Instead of starting a login shell, ssh can be used to just input one or more commands and return the output.

Try invoking ls on your favourite Ukko node without starting a login shell.

Put in your report the command line for invoking ls without starting a login shell

Is the output in the same format as on your own host?

Private and public keys

By now, it should be getting tedious to input your passwords to ssh on every login. Let's set up private and public keys.

Note: on department, ssh logins to for example shell and users (.cs.helsinki.fi) work without asking password via KERBEROS. However, UKKO nodes do ask password, and we'll be using UKKO so let's fix this.

ssh-keygen will create your own set of keys in the ~/.ssh subdirectory. They are typically named id_rsa and id_rsa.pub. You will need to do this only once.

Remember to protect your private key with a passphrase that is at least as strong as your department login's password, because this key is equivalent to your login and password.

The contents of the client computer's ~/.ssh/id_rsa.pub go into the server computer's ~/.ssh/authorized_keysDo not add extra line breaks when inserting the key!

There are many guides on the net which can be read for hints:

After this, you can do ssh -i ~/.ssh/id_rsa ukko-node.hpc.cs.helsinki.fi. If your private key had no password, ssh will not ask for it.

To be honest, the -i ~/.ssh/id_rsa part is redundant, since that is the default name for the private keyfile anyhow. You can omit it in the future.

Hint: be very careful with the permissions of the .ssh directory and id_rsa file. See the FILES section of man ssh for details.

What about the passwords?

Using a password for the private keyfile is crucial. However, a user does not wish to be prompted for the private key's password on every ssh login. This would be exactly as tedious as using passwords for the login in the first place!

The solution is to set up a SSH agent that handles your private keys. It will ask for your password only once, and then remember it for the duration of your session.

If you did not add a key to your ~/.ssh/id_rsa in the last step, you can use ssh-keygen -p -f ~/.ssh/id_rsa to add (or change) the passphrase. You can also check the contents of id_rsa to see if it's encrypted.

ssh-agent must be run to manage your private keys. It's output is a list of shell variables to be set. By default, Ubuntu runs ssh-agent for you, so you don't have to worry about this.

ssh-add must be run to add your private key from ssh-keygen to the store managed by ssh-agent. Hint: if you chose the default file names ~/.ssh/id_rsa and ~/.ssh/id_rsa.pubssh-add will discover them automatically. ssh-add -llists keys managed by ssh-agent.

Now paste your private key's password into... Just kidding! You should never do that. Never, ever, and then some more.

Put in your report, the sequence of commands that need to be entered to get a password-prompt-less login to Ukko

Keyring

Ubuntu handles the ssh-agent key store very nicely. Ubuntu integrates stored ssh keys into it's keyring. Open the GUI program "Passwords and Keys" and find your added ssh key from there.

Log out and back in to the machine you are on. Connect to Ukko. Keyring should (graphically) ask you to unlock the saved key, asking the passphrase for it. If you want, you can change the key to be automatically unlocked on login, which causes it to be never asked again.

EXERCISE 7

The right file, at the right time

It's now time to split your alias invocations into their own file. This file is named ~/.bash_aliases by convention.

Move all lines beginning with alias from your ~/.bashrc and ~/.bash_profile files into ~/.bash_aliases. This file should not begin with a she-bang line, or #!/bin/bash.

What we now want to do is to invoke the alias commands on every login and non-login shell session. For this to happen we must:

  1. Add ~/.bash_aliases into ~/.bashrc for *non-*login shells
  2. Add ~/.bash_aliases into ~/.bash_profile login shells
.bash_aliases snippet
if [ -f ~/.bash_aliases ]; then
  . ~/.bash_aliases
fi

Check your ~/.bashrc and ~/.bash_profile and insert the above snippet in order to call ~/.bash_aliases.

Make sure to update your report with the added lines.

(NON)-logins

The key difference between login and non-login shells can be demonstrated using ssh.

You can try out the difference by removing the snippet invoking ~/.bash_aliases from one of the bash configuration files and then invoking ssh for a login or directly by using the ls command.

Hint: when you call a command like ssh ukko160.hpc.cs.helsinki.fi, the host operating system actually invokes ssh ukko160.hpc.cs.helsinki.fi "bash -" if your default shell is Bash. Check echo $SHELL on the target host to verify this.

Present an example where the output of ls differs depending on how you have invoked ssh.

EXERCISE 8

Shared home directories

By now, you might have noticed that in the Department's Linux system, the home directories are shared. Technically, it's done through the Network File System protocol, or NFS for short. However, Ukko nodes have their own shared directories, accessible from the Department's Linux system at /cs/work/home/$USER/

Go on, try it out. We know you want to. Create a file called hostname.txt in your subdir by doing hostname > ~/LinuxFundamentals2015/Week1/hostname.txt. Copy the file to /cs/work/home/$USER/.  Then login to your favorite Ukko node and use cat to print the contents of that file.

Do this with ssh. Show in your output that ssh does not ask for a password.

Can you both produce and print the contents in one non-login ssh session? Hint: commands are separated with semicolons (;).

This is tremendously helpful, since it makes copying files between hosts mostly redundant. In many cases, things are not this simple.

EXERCISE 9

Enter RSYNC

Between hosts which do not share their home directories, the best way to copy files is through rsync. It is a very powerful utility, specially for backups, since it can detect changes between a source and a destination file, and then send only the changed portions.

In the Dept. network, we need some content to actually see the speed improvement. Let's copy a sizeable chunk of files.

The Exactum-kamera keeps watch of your Dept's home building. It saves one image every hour, on the hour. These archives go back many, many years, and can be browsed if one knows where to look. I'll tell you, but let's keep this our secret:

http://www.cs.helsinki.fi/u/tkt_cam/2015/09/30 is the web URL for Wednesday, September 30th, 2015. You could go there and click on each link to view the pictures, or you could just copy the files over the NFS system. Two alternatives:

  1. cp --archive /home/tktl-csfs/home/tkt_cam/public_html/2015/09/30 ~/LinuxFundamentals/Week1/Wednesday.2015.09.30
  2. rsync --archive /home/tktl-csfs/home/tkt_cam/public_html/2015/09/30/ ~/LinuxFundamentals/Week1/Wednesday.2015.09.30

Please be very thorough with those commands.

Experiment by running rsync on either a previously empty subdir Wednesday.2015.09.30 and one previously populated with cp. Show how rsync's output changes. Explain what's going on. Try adding the --stats argument to rsync.

EXERCISE 10

Time and Date

The commands from the previous exercise are fine, but very static. They are good for that one day only.

Read man date and experiment with its format parameters. For example, try date +%d.%m.%Y

Print today's date in the format equivalent to 'Wednesday.2015.09.30'. Show the command and it's output.

Inserting date

You can use backquotes to evaluate a command into a string

Example:

Echo today's date
#!/bin/bash
echo "Today is `date +%m.%d.%Y`"

 

Create a script rsync-todays-Exactum-cam.sh that uses echo to print a rsync command with the correct paths for the current date. Paste the code into your diary.

  • No labels