It is done mostly to prevent information leak such as your GET variables or website structure.
If your users have any kind of GET variables in their URL like
examplle.com/login.php?user=me&pass=pass and then clicks on that 'badpeople.com', the apache log of 'badpeople.com' will record the URL that your user came from as well as all the variables that were in URL field.
If you are an admin, with hidden admin panel like:
example.com/super-secret-admin-panel/super_admin.php and click on 'badpeople.com', their log will record the url of your super secret admin panel.
If your user is checking messages with a url of
example.com/user/admin/messages/ and goes to 'badpeople.com', apache log of 'badpeople.com' will have the username of your user. And since most people have same passwords on multiple websites they can essentially match those two and gain access to your account.
Generally speaking, if you are hosting a forum, a private website, or just don't want to deal with any additional headache from other people, disabling referrer tracking is not a bad idea.