Getting Started with DebOps
===========================

Welcome to DebOps! You have installed Ansible and DebOps scripts, downloaded
the roles and playbooks, and are wondering where to go next? Here you can read
about creating your first DebOps project, and managing remote hosts.

.. contents::
   :local:

An example environment
----------------------

Ansible and DebOps are installed on your workstation or laptop, a so called
"Ansible Controller" - this machine is used to control Ansible and issue
commands. The machine you will configure using DebOps is known as a "remote
host".

DebOps is designed to manage a host from the ground up. A good base
installation is a Debian Stable netinst, with only SSH server enabled and
configured. Everything else will be installed as needed.

.. note::

   If you are using Debian Jessie or other distributions based on it as the
   base install, by default OpenSSH server configured by the installer will
   disallow password authentication on the ``root`` account. You can either
   enable it manually in the ``/etc/ssh/sshd_config`` file, or configure
   a separate admin account and use that to bootstrap the host.

An important part of the environment is correctly configured DNS. Some of the
DebOps roles expect a configured domain - it doesn't need to be a real, global
domain, but it should be resolvable by the host. A good way to check if
a remote host has a correctly configured domain is to use the ``hostname --fqdn``
command. If the output has at least 1 dot, you should be good to go.

Do not use the domain apex (``example.com``) as the host name - this will
confuse Ansible and leave you with a broken configuration. Instead, create
separate hostnames inside the domain (``server.example.com``), this will be
used by Ansible correctly. It's good to use only one subdomain level per
subnet, this will make wildcard certificates work without issues. If you want,
you can create separate subdomains per subnet.

If your host does not have a domain configured, you will be able to do that
during the bootstrapping process.

In this guide, we will manage an example host called ``server.example.com``.
This host is a virtual machine, and we can connect to it using commands like:

.. code-block:: console

   alice@laptop:~$ ssh root@server.example.com
   alice@laptop:~$ ssh root@server

The ``root`` account requires a password, SSH keys are not installed yet and
there's no administrator account.

Ansible commands are executed on the Ansible Controller from an unprivileged
account ``alice``. This user has an SSH key pair stored in ``~/.ssh/id_rsa`` or
has its SSH key available in the SSH Agent. An administrator account with the
same name will be created on the remote host during the bootstrap process.

Your first project
------------------

Begin by creating a "DebOps project". It's a directory which contains all of
the data related to a given environment - Ansible inventory, passwords and
other secrets, custom playbooks and roles. To do this, use the ``debops-init``
command:

.. code-block:: console

   alice@laptop:~$ debops-init ~/myproject

This will create a new directory called ``myproject`` and populate it with some
example directories and files. You will perform most of the commands from the
main project directory, so let's ``cd`` into it:

.. code-block:: console

   alice@laptop:~$ cd ~/myproject

Ansible uses a ``hosts`` file to identify hosts that are under its control. In
the project directory this file is located in ``ansible/inventory/hosts``. Open
it in your favorite text editor and add the remote host in the main DebOps
host group:

.. code-block:: none

   [debops_all_hosts]
   server    ansible_ssh_host=server.example.com

Using a short inventory name allows you to run Ansible commands without
specifying the fully qualified domain name of the host.

Important inventory variables
-----------------------------

Some of the configuration used by DebOps cannot be auto-detected - examples
include IP addresses or network subnets that can connect to a SSH service
remotely, the administrator e-mail account which should receive important
notifications, and so on. Here you can find a list of the most important
variables which, when set correctly in inventory, can save you a trip to the
data center.

To make sure that these variables apply to all hosts in your environment, you
can include them in ``ansible/inventory/group_vars/all/`` directory. A common
practice is to name the files inside inventory directories after variable
prefixes, separately for each Ansible role. For example, variables related to
``debops.sshd`` role are stored in
``ansible/inventory/group_vars/all/sshd.yml``, variables used by the
``debops.postfix`` role are written in
``ansible/inventory/group_vars/all/postfix.yml``, and so on. The same scheme
can be used in other inventory groups or for separate hosts.

ansible_ssh_user
~~~~~~~~~~~~~~~~

This is an internal Ansible variable which is used to determine what remote
user account will be used to login to the server. If it's not explicitly set,
Ansible depends on SSH defaults which conventionally use the name of the
current user as the remote username. It's customary to specify this variable
directly in the ``hosts`` file, that way it can be unique for each host:

.. code-block:: none

   [debops_all_hosts]
   server    ansible_ssh_host=server.example.com ansible_ssh_user=ansible-admin

In DebOps this variable can be used to change the name of the default
administrator account, it's also used as a primary user account for various
tasks, like database and application administrative accounts.

On a specific platforms you can set this variable to an automatically created
username to make the remote host administration easier:

- Ubuntu-based hosts usually use the ``ubuntu`` username;

- Raspberry Pi / Pi 2 Linux distributions use the ``pi`` user account for this
  purpose;

However, it is advisable to not use the default user accounts, and instead
either create ones based on your own username (the default behavior) or create
completely separate Ansible accounts with administrative access. If you
configure the ``ansible_ssh_user`` variable before bootstrapping the host, the
specified username will be used to create an administrator account.

bootstrap_domain
~~~~~~~~~~~~~~~~

If hosts that you want to manage don't have a DNS domain set, or it's incorrect
(for example your VPS provider's domain instead of your own), the
``debops.bootstrap`` role can be used to easily fix that and configure your own
domain. By setting this variable to, for example:

.. code-block:: yaml

   ---
   bootstrap_domain: 'example.com'

By running the ``debops bootstrap`` command (see further down), your domain
will be configured in the remote hosts' ``/etc/hosts`` file. Additionally, the
hostname will be changed to the one you specified in the Ansible inventory.
After that is done, it's best to reboot the machine to make sure all of the
changed settings are applied and are persistent.

This variable won't have any effect on hosts that are not "bootstrapped", and
are instead configured using Debian preseeding or LXC templates - these hosts
will presumably get the needed information like hostname and domain from your
own DHCP server.

You can check `other debops.bootstrap variables <http://docs.debops.org/en/latest/ansible/roles/ansible-bootstrap/docs/defaults.html>`_ for some more useful configuration, like the name of the administrator account.

sshd__whitelist
~~~~~~~~~~~~~~~

Protection of the SSH service is very important. Hosts configured by DebOps use
a firewall and TCP Wrappers to restrict what hosts can connect to it and
automatically block repeated offenders for certain amount of time.

To not block the Ansible Controller, DebOps tries to detect the IP address
from which the connection is made. For the most part it should work as
expected, but if you still are getting blocked, or to be sure that remote
access won't be interrupted, you can define a list of IP addresses or CIDR
subnets that will be allowed to connect to SSH without restrictions.

To do that, in ``ansible/inventory/group_vars/all/sshd.yml`` add:

.. code-block:: yaml

   ---
   sshd__whitelist: [ '192.0.2.0/24', '2001:db8::/32' ]

This will configure the ``debops.ferm`` and ``debops.tcpwrappers`` roles
to allow connections to the ``ssh`` service from specified networks.

The ``debops.sshd`` role has `many more variables <http://docs.debops.org/en/latest/ansible/roles/ansible-sshd/docs/defaults.html>`_, you can checkout to see the default configuration used by DebOps and what can be changed as needed.

ntp_timezone
~~~~~~~~~~~~

By default, DebOps does not try to change the remote host timezone and tries to
use the detected one in roles that need that information for the configuration.
If you need to change the timezone, you can do it by setting the
``ntp_timezone`` variable like this:

.. code-block:: yaml

   ---
   ntp_timezone: 'America/New_York'

For UTC timezone, use this format:

.. code-block:: yaml

   ---
   ntp_timezone: 'Etc/UTC'

postfix_relayhost
~~~~~~~~~~~~~~~~~

The default SMTP server used by DebOps is Postfix. It has a somewhat secure
configuration by default, with all of the inbound mail being redirected to your
domain's MX gateway (e-mail messages to other domains are sent to them
directly). If you want to instead pass all e-mail messages to a different SMTP
host for further processing, you can specify its FQDN hostname using this
variable:

.. code-block:: yaml

   ---
   postfix_relayhost: 'internal-mx.{{ ansible_domain }}'

Only one relayhost is supported at a time. The specified host should accept
messages from hosts controlled by Ansible for this to work correctly.

Postfix can be configured to a large extent using `the debops.postfix role variables <http://docs.debops.org/en/latest/ansible/roles/ansible-postfix/docs/defaults.html>`_ - you can use it to create an MX gateway for your network, setup a host with local mail, enable archiving, add support for sender authentication, and so on.

apt_default_mirrors_lookup, apt_default_sources_lookup
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

DebOps tries to detect the operating system a given host is using and configure
it accordingly. Currently selected Debian and Ubuntu releases are recognized
and the package sources for these operating systems should be configured
without issues.

The Raspbian operating system is a little difficult to detect, because Ansible
currently classifies it as "Debian", however its package repositories are
completely different. To avoid issues with incompatible package sources on your
Raspberry Pi/Pi2, you should change the default ``debops.apt`` configuration
manually to use the Raspbian repositories. To do that, add these values in
relevant inventory files:

.. code-block:: yaml

   ---
   apt_default_mirrors_lookup: 'raspbian'
   apt_default_sources_lookup: 'raspbian'


Bootstrap a new host
--------------------

.. warning::

  Bootstrapping a host without a configured ``bootstrap_domain`` will result in
  a broken host configuration.

At this point you most likely have to connect to that host using the ``root``
account and specifying a password. To make that easier, you can use a special
"bootstrap" Ansible playbook to prepare a host for easier management. To do
this, execute the command:

.. code-block:: console

   alice@laptop:~/myproject$ debops bootstrap --limit server --user root --ask-pass

Or, for short:

.. code-block:: console

   alice@laptop:~/myproject$ debops bootstrap -l server -u root -k

This command will execute the `debops.bootstrap
<http://docs.debops.org/en/latest/ansible/roles/ansible-bootstrap/docs/index.html>`_
role and use it to install a base set of packages needed by Ansible like
``python`` and ``sudo``, prepare a new administrator account named after your
system user (``alice`` in our example) and allow that account full access to
the ``root`` account using ``sudo``. Your SSH keys will be installed on both
the ``root`` and administrator accounts.

.. note::

   Bootstrapping a host this way is not needed if you already have an
   administrator account that can use ``sudo`` without a password. This
   includes hosts configured using Debian Preseed provided by DebOps as well as
   OpenVZ/LXC containers configured using provided templates.

When the bootstrap playbook has finished and there are no errors, you can check
if you are able to connect to the server on the administrator account without a
password:

.. code-block:: console

   alice@laptop:~/myproject$ ssh server

After logging in, check if you can run commands using ``sudo`` without
a password:

.. code-block:: console

   alice@server:~$ sudo -l

Configure the remote host
-------------------------

When a new remote host has been prepared for Ansible management, you can start
the configuration:

.. code-block:: console

   alice@laptop:~/myproject$ debops -l server

This will start the ``ansible-playbook`` command with the main DebOps playbook. This by default includes the `common playbook <https://github.com/debops/debops-playbooks/blob/master/playbooks/common.yml>`_ with a default set of roles, and any additional playbooks, if they have been enabled.

The initial configuration might take 5-10 minutes on a reasonably fast machine.
There are some steps, like Diffie-Hellman parameter generation, which might
take significantly more time to complete.

When the playbook run has been finished, your remote host should be configured
with:

- a correct set of APT repositories for your operating system release;
- automatic updates of the installed packages with related e-mail messages sent
  to your admin account;
- a set of Diffie-Hellman parameters and SSL certificates ready to use by
  different services (encrypted TLS/SSL connections out of the box);
- configured ``iptables``/``ip6tables`` firewall and TCP Wrappers;
- enabled network time synchronization as needed;
- a set of useful management software installed on the host (``htop``,
  ``mtr-tiny``, ``mc``, ``vim``, among other things);

Example application - DokuWiki
------------------------------

Each host configured by common DebOps playbook should have the same set of base
services. After a host is configured, you can enable additional Ansible roles
to install and configure software and applications of your choice.

We will use `DokuWiki <http://dokuwiki.org/>`_ as an example application. The
role that manages the installation is called ``debops.dokuwiki``, it uses
``debops.nginx`` and ``debops.php5`` roles to configure a webserver and PHP5
environment. The ``debops.nginx`` role calls some additional roles, such as
``debops.ferm`` to configure needed services.

To install DokuWiki on your new remote host, you need to enable the respective
role in Ansible inventory. This is done by creating a new host group,
``[debops_service_dokuwiki]`` in the ``hosts`` file, and adding the desired
hosts to it:

.. code-block:: none

   [debops_all_hosts]
   server    ansible_ssh_host=server.example.com

   [debops_service_dokuwiki]
   server

As you can see, you don't need to copy the wohole host entry, only the short
name is enough.

The ``debops.dokuwiki`` has `many default variables
<http://docs.debops.org/en/latest/ansible/roles/ansible-dokuwiki/docs/defaults.html>`_
you can use to customize the installation. One of the more useful ones is
``dokuwiki_main_domain``; it's a list which specifies what DNS subdomains are
used to access the wiki (each application in the DebOps set of roles is
configured on a separate subdomain). By default DokuWiki will be accessible on
the ``wiki.{{ ansible_domain }}`` subdomain, if you want to change it, you can
do so by creating the ``ansible/inventory/host_vars/server/dokuwiki.yml``
configuration file and specifying the subdomain(s) in it:

.. code-block:: yaml

   ---
   dokuwiki_main_domain: [ 'wiki.{{ ansible_domain }}' ]

Remember that the chosen subdomain (``wiki.`` or your own) needs to be
configured in your DNS server to point to the specified remote host.

When everything is configured, you can execute the ``debops`` script to apply
new configuration on the host:

.. code-block:: console

   alice@laptop:~/myproject$ debops -l server

This will apply the whole playbook with all the configuration on the specified
server. However, to make this process faster, DebOps provides separate "service
playbooks" for each of the roles. To use these playbooks, you can specify them
as the first argument to the ``debops`` command:

.. code-block:: console

   alice@laptop:~/myproject$ debops service/dokuwiki -l server

This will tell the script to look for the playbook in several places:

- ``playbooks/`` and ``ansible/playbooks/`` subdirectories in the project
  directory;
- ``debops-playbooks/playbooks/`` subdirectory of the project directory, if
  DebOps playbooks and roles are installed inside of it;
- ``~/.local/share/debops/debops-playbooks/playbooks/`` directory (default
  install location);

The first one found will be executed. You can use this to your advantage by
adding custom playbooks in ``playbooks/`` or ``ansible/playbooks/``
directories, they need the be named with ``.yml`` extension. Custom roles can
be placed in the ``roles/`` or ``ansible/roles/`` subdirectories located in the
project directory.

After Ansible finishes the configuration, you will need to go to the
``https://wiki.<domain>/install.php`` page to complete the installation
process.

At this time you might find that the web browser you are using does not
recognize the CA certificates served by the host. This happens when server uses
certificates signed by internal DebOps Certificate Authority instead of the
"regular" ones. To fix that, consult the ``debops.pki`` role documentation
(when it's available).

Where to go from here
---------------------

You can add more hosts to the Ansible inventory and configure them in a cluster.
Hosts should automatically trust each other using an internal Certificate
Authority, so encrypted connections between them should work out of the box.

DebOps contains multiple Ansible roles that allow you to install and configure
useful software, like GitLab, phpIPAM, ownCloud and others. You should check
`the documentation <http://docs.debops.org/>`_ of the respective roles to see
some example configurations and useful tips. Note that parts of the
documentation are currently outdated - if a given role has only one page, you
should check the role files directly.

You can check the `DebOps Changelog <https://log.debops.org/>`_ for updates
related to roles and playbooks (there's also an `Atom feed <https://log.debops.org/atom.xml>`_ available for your feed reader).

..
 Local Variables:
 mode: rst
 ispell-local-dictionary: "american"
 End:
