How to install and configure MariaDB Galera as master to master replication cluster on Ubuntu 16.04 LTS

MariaDB Galera Cluster is an open source and free synchronous multi-master cluster for MariaDB database. It is available on Linux only and only supports the XtraDB/InnoDB storage engines. There is experimental support for MyISAM, but it is not well tested. Starting with MariaDB 10.1, the wsrep API for Galera Cluster is included by default. How do I install and configure MariaDB Galera master to master cluster on Ubuntu Linux 16.04 LTS server to get both read and write scalability?