How to set up Apt caching server on Ubuntu or Debian

If you often create guest VMs or containers on your Linux computer, you will go through package installation many times across different VMs/containers. In that case, your time may be better spent on something more productive than just waiting to finish installing packages. While server provisioning tools like Puppet, Chef or Ansible can automate the whole machine provisioning process, they are an overkill for virtualization on a single laptop or desktop computer. Also parallel downloads have a limited effect when the network is slow.