It’s a fact of human nature that the more space you have, the more stuff you will accumulate to fill it. This is true for our lives as much as it is true for our servers. Given that you have invested in a nice big 1Tb or more hard drive, you may find that after a while it starts to fill up with life-debris. In order to keep your server running efficiently, you will need to monitor where its storage space is being used, and maintain order in your digital life.
The starting point for this is simply knowing how much storage is being used on your server. Linux comes has a few built-in commands which will help you monitor disk space usage.
The first of these is df. df simply reports back disk space usage; I’m also using the –h flag for human-readable values, otherwise the sizes are reported in kilobytes. The report will also return values for some pseudo-file systems, but among them will be the report for your disk drive(s). In the case below, my 1Tb drive, listed as /dev/sda1, is the largest, and it is 52% full. If you have other drives mounted, either additional hard drives or USB drives, they will also be shown here with their respective values.
So now that we know how full our drive is, how to we begin to identify where this storage space is being used up? There are two tools we can use here. The first is du. du (for ‘disk usage’) reports on file/folder space usage. There are many flags and options which can be used with this tool, but I have found myself using du –sh ./* most commonly. This works well when you are within a folder, and want to see the sizes of the files and folders within that folder.
The –s option summarises the sizes of any folders under the one you are in currently. Without this, du scrolls through each sub-folder and gives the sizes of each and every file within them, leading to a rapidly scrolling screen of text.
The –h option specifies human-readable values again, otherwise sizes are reported in Kb.
The ./* is important also. With this, you are specifying that you want to see the summarised sizes of each of the files & folders in the folder you are in. Without this, du will just print the overall size of the folder, without breaking it down.
The only problem with this report is that it is ordered alphabetically and not in order of size. You could try piping the results through the sort command (du –sh ./* | sort), but this does not make a distinction between Kb, Mb and Gb, so 1Gb files will appear before 2Mb ones.
ncdu is a very useful utility which allows you to browse through the directories on your server and visualise how much space the files and folders are using up. It does not come as part of Ubuntu server, so must be installed:
sudo apt install ncdu
When ncdu is run, it scans the current directory you are in and all sub-directories. You can also run it and specify a directory to start from, e.g. ncdu / to scan the root and everything underneath it. It builds a database of the folder structure, and runs surprisingly fast even when scanning the whole server.
I find it useful to run ncdu under screen so I can continue working while it is scanning.
Once it is finished it presents a screen showing directories listed in order of decreasing size. To explore any directory and its contents, simply move down to it, using the arrow keys, and click Enter. At the very top of each sub-directory is a link (..) to allow you to move back up and out of the folder you are in.
To get more information on a file or folder, move down to it and press i. To delete something which is taking up too much space, move down to it and press d. You can also launch a command prompt in the directory which you have moved into using ncdu by pressing b.
Other keyboard commands include:
|a||Toggle between apparent size and disk usage|
|b||Spawn shell in current directory|
|c||Toggle display of child item counts|
|C||Sort by items (ascending/descending)|
|d||Delete selected file or directory|
|e||Show/hide hidden or excluded files|
|g||Show percentage and/or graph|
|i||Show information about selected item|
|n||Sort by name (ascending/descending)|
|r||Recalculate the current directory|
|s||Sort by size (ascending/descending)|
|t||Toggle dirs before files when sorting|