INTRO

Your server is now running two services: the sshd (Secure Shell Daemon) service that you use to login; and the Apache2 web server. Both of these services are generating logs as you and others access your server - and these are text files which we can analyse using some simple tools.

Plain text files are a key part of “the Unix way” and there are many small “tools” to allow you to easily edit, sort, search and otherwise manipulate them. Today we’ll use grep, cat, more, less, cut, awk and tail to slice and dice your logs.

The grep command is famous for being extremely powerful and handy, but also because its “nerdy” name is typical of Unix/Linux conventions.

YOUR TASKS TODAY

  • Dump out the complete contents of a file with cat like this: cat /var/log/apache2/access.log
  • Use less to open the same file, like this: less /var/log/apache2/access.log - and move up and down through the file with your arrow keys, then use “q” to quit.
  • Again using less, look at a file, but practice confidently moving around using gg, GG and /, n and N (to go to the top of the file, bottom of the file, to search for something and to hop to the next “hit” or back to the previous one)
  • View recent logins and sudo usage by viewing /var/log/auth.log with less
  • Look at just the tail end of the file with tail /var/log/apache2/access.log (yes, there’s also a head command!)
  • Follow a log in real-time with: tail -f /var/log/apache2/access.log (while accessing your server’s web page in a browser)
  • You can take the output of one command and “pipe” it in as the input to another by using the | (pipe) symbol
  • So, dump out a file with cat, but pipe that output to grep with a search term - like this: cat /var/log/auth.log | grep "authenticating"
  • Simplify this to: grep "authenticating" /var/log/auth.log
  • Piping allows you to narrow your search, e.g. grep "authenticating" /var/log/auth.log | grep "root"
  • Use the cut command to select out most interesting portions of each line by specifying “-d” (delimiter) and “-f” (field) - like: grep "authenticating" /var/log/auth.log| grep "root"| cut -f 10- -d" " (field 10 onwards, where the delimiter between field is the " " character). This approach can be very useful in extracting useful information from log data.
  • Use the -v option to invert the selection and find attempts to login with other users: grep "authenticating" /var/log/auth.log| grep -v "root"| cut -f 10- -d" "

The output of any command can be “redirected” to a file with the “>” operator. The command: ls -ltr > listing.txt wouldn’t list the directory contents to your screen, but instead redirect into the file “listing.txt” (creating that file if it didn’t exist, or overwriting the contents if it did).

WHERE’S MY /VAR/LOG/AUTH.LOG?

If you didn’t find the file /var/log/auth.log you’re probably using a minimal version of Ubuntu (it can be your own local VM or a version in one of the VPS). That minimal image is, well… minimal. It only has the systemd journal available and it didn’t come with the old syslog system by default.

But don’t worry! To get that back, sudo apt install rsyslog and the file will be created. Just give it a few minutes to populate before working on the lesson.

It also be missing a few of the other programs we use in the challenge, but you can always install them.

POSTING YOUR PROGRESS

Re-run the command to list all the IP’s that have unsuccessfully tried to login to your server as root - but this time, use the the “>” operator to redirect it to the file: ~/attackers.txt. You might like to share and compare with others doing the course how heavily you’re “under attack”!

EXTENSION

  • See if you can extend your filtering of auth.log to select just the IP addresses, then pipe this to sort, and then further to uniq to get a list of all those IP addresses that have been “auditing” your server security for you.
  • Investigate the awk and sed commands. When you’re having difficulty figuring out how to do something with grep and cut, then you may need to step up to using these. Googling for “linux sed tricks” or “awk one liners” will get you many examples.
  • Aim to learn at least one simple useful trick with both awk and sed

RESOURCES

TROUBLESHOOT AND MAKE A SAD SERVER HAPPY!

Practice what you’ve learned with some challenges at SadServers.com:

Some rights reserved. Check the license terms here