There are constant level of high attacks and port scans on Linux servers all the time, while a properly configured firewall and regular security system updates adds a extra layer to keep the system safe, but you should also frequently watch if anyone got in. This will also helps to ensure that your server stays free of any program that aims at disrupting its normal operation.