verdnatura/vn-ansible
verdnatura
/
vn-ansible
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Initial commit

This commit is contained in:
Juan Ferrer 2024-10-16 17:53:15 +02:00
commit 4be47afe7d
128 changed files with 3420 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.gitignore vendored Normal file
View File

@ -0,0 +1,6 @@
.vscode/
.vault-pass
.vault.yml
.passbolt.yml
inventories/local
venv

8
.passbolt.tpl.yml Normal file
View File

@ -0,0 +1,8 @@
PASSBOLT_BASE_URL: https://passbolt.domain.local/
PASSBOLT_PASSPHRASE: "S3cr3tP4$$w0rd"
PASSBOLT_PRIVATE_KEY: |
-----BEGIN PGP PRIVATE KEY BLOCK-----
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
-----END PGP PRIVATE KEY BLOCK-----

17
LICENSE Normal file
View File

@ -0,0 +1,17 @@
Copyright (C) 2024 - Verdnatura Levante S.L.
This package is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
On Debian systems, the complete text of the GNU General Public
License can be found in "/usr/share/common-licenses/GPL-3".

111
README.md Normal file
View File

@ -0,0 +1,111 @@
# Verdnatura Ansible playbooks
Collection of Ansible playbooks used in the Verdnatura server farm.
## Setup Ansible
### Debian
Install Ansible package.
```
apt install ansible
```
### Python
Create a Python virtual environment.
```
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip ansible==10.1.0 ansible-builder==3.1.0
```
Before running any Python dependent command, activate the virtual environment.
```
source venv/bin/activate
```
Once you are done, deactivate the virtual environment.
```
deactivate
```
### All platforms
Install dependencies.
```
pip install -r requirements.txt
ansible-galaxy collection install -r collections/requirements.yml
```
## Run playbook
Before merging changes into protected branches, playbooks should be tested
locally to ensure they work properly. The *inventories/local* inventory is not
uploaded to the repository and can be used for local testing. In any case, it
is advisable to use a different repository to store inventories.
Run playbook on inventory host.
```
ansible-playbook -i inventories/local -l <host> [-t tag1,tag2...] playbooks/ping.yml
```
Run playbook on the fly on a host not declared in the inventory.
```
ansible-playbook -i <ip_or_hostname>, playbooks/ping.yml
```
*Note the comma at the end of the hostname or IP.*
## Manage secrets
Secrets can be managed by using Ansible vault or an external keystore, Passbolt
is used in this case. It is recommended to use an external keystore to avoid
publicly exposing the secrets, even if they are encrypted.
When running playbooks that use any of the keystores mentioned above, the
*run-playbook.sh* script can be used, it is an ovelay over the original
*ansible-playbook* command which injects the necessary parameters.
### Passbolt
Add the necessary environment variables to the *.passbolt.yml* file, the
template file *.passbolt.tpl.yml* is included as a reference:
* https://galaxy.ansible.com/ui/repo/published/anatomicjc/passbolt/docs/
### Ansible vault
To manage Ansible vault place the encryption password into *.vault-pass* file.
Manage the vault.
```
ansible-vault {view,edit,create} --vault-pass-file .vault-pass .vault.yml
```
> The files used for the vault must only be used locally and
> under **no** circumstances can they be uploaded to the repository.
## Build execution environment for AWX
Create an image with *ansible-builder* and upload it to registry.
```
ansible-builder build --tag awx-ee:vn1
```
## Common playbooks
* **facts.yml**: Collect and display facts from a host
* **ping.yml**: Check that a host is alive and reachable
* **awx.yml**: Create and configure AWX user
* **debian.yml**: Setup base Debian server
## Documentation
* https://docs.ansible.com/ansible/latest/reference_appendices/config.html
* https://docs.ansible.com/ansible/latest/collections/ansible/builtin/gather_facts_module.html
* https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_vars_facts.html
* https://ansible.readthedocs.io/projects/builder/en/latest/
* https://www.ansible.com/blog/introduction-to-ansible-builder/
* https://github.com/ansible/awx-ee/
* https://www.passbolt.com/blog/managing-secrets-in-ansible-using-passbolt

11
ansible.cfg Normal file
View File

@ -0,0 +1,11 @@
[defaults]
remote_user = root
host_key_checking = False
roles_path = ./roles
inventory = ./inventories/lab
gathering = smart
interpreter_python = auto_silent
deprecation_warnings = False
[privilege_escalation]
become = True

34
collections/README.md Normal file
View File

@ -0,0 +1,34 @@
## Collections
The purpose of collections is get more modules and plugins to use in ansible.
Place your collection dependencies here in *requirements.yml* and AWX will
install them automtatically.
# Install collections
In *awx-operator* container, execute:
```
ansible-galaxy collection install -r requirements.yml
```
Where `requirements.yml` we need to specify the list of collections that we
want to install:
```
collections:
- name: community.general
```
# Collection index
* https://docs.ansible.com/ansible/latest/collections/index.html
# List collections
To list installed collections, run `ansible-galaxy collection list` (inside awx
operator container)
* https://docs.ansible.com/ansible/latest/collections_guide/collections_listing.html
# Install multiple collections
* https://docs.ansible.com/ansible/devel/collections_guide/collections_installing.html#install-multiple-collections-with-a-requirements-file

10
collections/requirements.yml Normal file
View File

@ -0,0 +1,10 @@
collections:
- name: ansible.utils
version: '>=4.1.0'
type: galaxy
- name: ansible.windows
version: '>=2.3.0'
type: galaxy
- name: anatomicjc.passbolt
version: '>=0.0.14'
type: galaxy

96
context/Dockerfile Normal file
View File

@ -0,0 +1,96 @@
ARG EE_BASE_IMAGE="quay.io/centos/centos:stream9"
ARG PYCMD="/usr/bin/python3.12"
ARG PYPKG="python3.12"
ARG PKGMGR_PRESERVE_CACHE=""
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS=""
ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS=""
ARG ANSIBLE_INSTALL_REFS="ansible-core>=2.17.0 ansible-runner==2.4.0"
ARG PKGMGR="/usr/bin/dnf"
# Base build stage
FROM $EE_BASE_IMAGE as base
USER root
ENV PIP_BREAK_SYSTEM_PACKAGES=1
ARG EE_BASE_IMAGE
ARG PYCMD
ARG PYPKG
ARG PKGMGR_PRESERVE_CACHE
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
ARG ANSIBLE_INSTALL_REFS
ARG PKGMGR
COPY _build/scripts/ /output/scripts/
COPY _build/scripts/entrypoint /opt/builder/bin/entrypoint
RUN $PKGMGR install $PYPKG -y ; if [ -z $PKGMGR_PRESERVE_CACHE ]; then $PKGMGR clean all; fi
RUN /output/scripts/pip_install $PYCMD
RUN $PYCMD -m pip install --no-cache-dir $ANSIBLE_INSTALL_REFS
# Galaxy build stage
FROM base as galaxy
ARG EE_BASE_IMAGE
ARG PYCMD
ARG PYPKG
ARG PKGMGR_PRESERVE_CACHE
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
ARG ANSIBLE_INSTALL_REFS
ARG PKGMGR
RUN /output/scripts/check_galaxy
COPY _build /build
WORKDIR /build
RUN mkdir -p /usr/share/ansible
RUN ansible-galaxy role install $ANSIBLE_GALAXY_CLI_ROLE_OPTS -r requirements.yml --roles-path "/usr/share/ansible/roles"
RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"
# Builder build stage
FROM base as builder
ENV PIP_BREAK_SYSTEM_PACKAGES=1
WORKDIR /build
ARG EE_BASE_IMAGE
ARG PYCMD
ARG PYPKG
ARG PKGMGR_PRESERVE_CACHE
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
ARG ANSIBLE_INSTALL_REFS
ARG PKGMGR
RUN $PYCMD -m pip install --no-cache-dir bindep pyyaml packaging
COPY --from=galaxy /usr/share/ansible /usr/share/ansible
COPY _build/requirements.txt requirements.txt
COPY _build/bindep.txt bindep.txt
RUN $PYCMD /output/scripts/introspect.py introspect --user-pip=requirements.txt --user-bindep=bindep.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
RUN /output/scripts/assemble
# Final build stage
FROM base as final
ENV PIP_BREAK_SYSTEM_PACKAGES=1
ARG EE_BASE_IMAGE
ARG PYCMD
ARG PYPKG
ARG PKGMGR_PRESERVE_CACHE
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
ARG ANSIBLE_INSTALL_REFS
ARG PKGMGR
RUN /output/scripts/check_ansible $PYCMD
COPY --from=galaxy /usr/share/ansible /usr/share/ansible
COPY --from=builder /output/ /output/
RUN /output/scripts/install-from-bindep && rm -rf /output/wheels
RUN chmod ug+rw /etc/passwd
RUN mkdir -p /runner && chgrp 0 /runner && chmod -R ug+rwx /runner
WORKDIR /runner
RUN $PYCMD -m pip install --no-cache-dir 'dumb-init==1.2.5'
RUN rm -rf /output
LABEL ansible-execution-environment=true
USER 1000
ENTRYPOINT ["/opt/builder/bin/entrypoint", "dumb-init"]
CMD ["bash"]

18
context/_build/bindep.txt Normal file
View File

@ -0,0 +1,18 @@
git-core [platform:rpm]
python3.11-devel [platform:rpm compile]
libcurl-devel [platform:rpm compile]
krb5-devel [platform:rpm compile]
krb5-workstation [platform:rpm]
subversion [platform:rpm]
subversion [platform:dpkg]
git-lfs [platform:rpm]
sshpass [platform:rpm]
rsync [platform:rpm]
epel-release [platform:rpm]
unzip [platform:rpm]
podman-remote [platform:rpm]
cmake [platform:rpm compile]
gcc [platform:rpm compile]
gcc-c++ [platform:rpm compile]
make [platform:rpm compile]
openssl-devel [platform:rpm compile]

3
context/_build/requirements.txt Normal file
View File

@ -0,0 +1,3 @@
py-passbolt==0.0.18
cryptography==3.3.2
PGPy==0.6.0

10
context/_build/requirements.yml Normal file
View File

@ -0,0 +1,10 @@
collections:
- name: ansible.utils
version: '>=4.1.0'
type: galaxy
- name: ansible.windows
version: '>=2.3.0'
type: galaxy
- name: anatomicjc.passbolt
version: '>=0.0.14'
type: galaxy

169
context/_build/scripts/assemble Executable file
View File

@ -0,0 +1,169 @@
#!/bin/bash
# Copyright (c) 2019 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Make a list of bindep dependencies and a collection of built binary
# wheels for the repo in question as well as its python dependencies.
# Install javascript tools as well to support python that needs javascript
# at build time.
set -ex
RELEASE=$(source /etc/os-release; echo $ID)
# NOTE(pabelanger): Allow users to force either microdnf or dnf as a package
# manager.
PKGMGR="${PKGMGR:-}"
PKGMGR_OPTS="${PKGMGR_OPTS:-}"
PKGMGR_PRESERVE_CACHE="${PKGMGR_PRESERVE_CACHE:-}"
PYCMD="${PYCMD:=/usr/bin/python3}"
PIPCMD="${PIPCMD:=$PYCMD -m pip}"
if [ -z $PKGMGR ]; then
# Expect dnf to be installed, however if we find microdnf default to it.
PKGMGR=/usr/bin/dnf
if [ -f "/usr/bin/microdnf" ]; then
PKGMGR=/usr/bin/microdnf
fi
fi
if [ "$PKGMGR" = "/usr/bin/microdnf" ]
then
if [ -z "${PKGMGR_OPTS}" ]; then
# NOTE(pabelanger): skip install docs and weak dependencies to
# make smaller images. Sadly, setting these in dnf.conf don't
# appear to work.
PKGMGR_OPTS="--nodocs --setopt install_weak_deps=0"
fi
fi
# NOTE(pabelanger): Ensure all the directory we use exists regardless
# of the user first creating them or not.
mkdir -p /output/bindep
mkdir -p /output/wheels
mkdir -p /tmp/src
cd /tmp/src
function install_bindep {
# Protect from the bindep builder image use of the assemble script
# to produce a wheel. Note we append because we want all
# sibling packages in here too
if [ -f bindep.txt ] ; then
bindep -l newline | sort >> /output/bindep/run.txt || true
if [ "$RELEASE" == "centos" ] ; then
bindep -l newline -b epel | sort >> /output/bindep/stage.txt || true
grep -Fxvf /output/bindep/run.txt /output/bindep/stage.txt >> /output/bindep/epel.txt || true
rm -rf /output/bindep/stage.txt
fi
compile_packages=$(bindep -b compile || true)
if [ ! -z "$compile_packages" ] ; then
$PKGMGR install -y $PKGMGR_OPTS ${compile_packages}
fi
fi
}
function install_wheels {
# NOTE(pabelanger): If there are build requirements to install, do so.
# However do not cache them as we do not want them in the final image.
if [ -f /tmp/src/build-requirements.txt ] && [ ! -f /tmp/src/.build-requirements.txt ] ; then
$PIPCMD install $CONSTRAINTS $PIP_OPTS --no-cache -r /tmp/src/build-requirements.txt
touch /tmp/src/.build-requirements.txt
fi
# Build a wheel so that we have an install target.
# pip install . in the container context with the mounted
# source dir gets ... exciting, if setup.py exists.
# We run sdist first to trigger code generation steps such
# as are found in zuul, since the sequencing otherwise
# happens in a way that makes wheel content copying unhappy.
# pip wheel isn't used here because it puts all of the output
# in the output dir and not the wheel cache, so it's not
# possible to tell what is the wheel for the project and
# what is the wheel cache.
if [ -f setup.py ] ; then
$PYCMD setup.py sdist bdist_wheel -d /output/wheels
fi
# Install everything so that the wheel cache is populated with
# transitive depends. If a requirements.txt file exists, install
# it directly so that people can use git url syntax to do things
# like pick up patched but unreleased versions of dependencies.
# Only do this for the main package (i.e. only write requirements
# once).
if [ -f /tmp/src/requirements.txt ] && [ ! -f /output/requirements.txt ] ; then
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels -r /tmp/src/requirements.txt
cp /tmp/src/requirements.txt /output/requirements.txt
fi
# If we didn't build wheels, we can skip trying to install it.
if [ $(ls -1 /output/wheels/*whl 2>/dev/null | wc -l) -gt 0 ]; then
$PIPCMD uninstall -y /output/wheels/*.whl
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels /output/wheels/*whl
fi
}
PACKAGES=$*
PIP_OPTS="${PIP_OPTS-}"
# bindep the main package
install_bindep
# go through ZUUL_SIBLINGS, if any, and build those wheels too
for sibling in ${ZUUL_SIBLINGS:-}; do
pushd .zuul-siblings/${sibling}
install_bindep
popd
done
# Use a clean virtualenv for install steps to prevent things from the
# current environment making us not build a wheel.
# NOTE(pabelanger): We allow users to install distro python packages of
# libraries. This is important for projects that eventually want to produce
# an RPM or offline install.
$PYCMD -m venv /tmp/venv --system-site-packages --without-pip
source /tmp/venv/bin/activate
# If there is an upper-constraints.txt file in the source tree,
# use it in the pip commands.
if [ -f /tmp/src/upper-constraints.txt ] ; then
cp /tmp/src/upper-constraints.txt /output/upper-constraints.txt
CONSTRAINTS="-c /tmp/src/upper-constraints.txt"
fi
# If we got a list of packages, install them, otherwise install the
# main package.
if [[ $PACKAGES ]] ; then
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels $PACKAGES
for package in $PACKAGES ; do
echo "$package" >> /output/packages.txt
done
else
install_wheels
fi
# go through ZUUL_SIBLINGS, if any, and build those wheels too
for sibling in ${ZUUL_SIBLINGS:-}; do
pushd .zuul-siblings/${sibling}
install_wheels
popd
done
if [ -z $PKGMGR_PRESERVE_CACHE ]; then
$PKGMGR clean all
rm -rf /var/cache/{dnf,yum}
fi
rm -rf /var/lib/dnf/history.*
rm -rf /var/log/{dnf.*,hawkey.log}
rm -rf /tmp/venv

110
context/_build/scripts/check_ansible Executable file
View File

@ -0,0 +1,110 @@
#!/bin/bash
# Copyright (c) 2023 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#####################################################################
# Script to validate that Ansible and Ansible Runner are installed.
#
# Usage: check_ansible <PYCMD>
#
# Options:
# PYCMD - The path to the python executable to use.
#####################################################################
set -x
PYCMD=$1
if [ -z "$PYCMD" ]
then
echo "Usage: check_ansible <PYCMD>"
exit 1
fi
if [ ! -x "$PYCMD" ]
then
echo "$PYCMD is not an executable"
exit 1
fi
ansible --version
if [ $? -ne 0 ]
then
cat<<EOF
**********************************************************************
ERROR - Missing Ansible installation
An Ansible installation cannot be found in the final builder image.
Ansible must be installed in the final image. If you are using a
recent enough version of the execution environment file, you may
use the 'dependencies.ansible_core' configuration option to install
Ansible for you, or use 'additional_build_steps' to manually do
this yourself. Alternatively, use a base image with Ansible already
installed.
**********************************************************************
EOF
exit 1
fi
ansible-runner --version
if [ $? -ne 0 ]
then
cat<<EOF
**********************************************************************
ERROR - Missing Ansible Runner installation
An Ansible Runner installation cannot be found in the final builder
image.
Ansible Runner must be installed in the final image. If you are
using a recent enough version of the execution environment file, you
may use the 'dependencies.ansible_runner' configuration option to
install Ansible Runner for you, or use 'additional_build_steps' to
manually do this yourself. Alternatively, use a base image with
Ansible Runner already installed.
**********************************************************************
EOF
exit 1
fi
$PYCMD -c 'import ansible ; import ansible_runner'
if [ $? -ne 0 ]
then
cat<<EOF
**********************************************************************
ERROR - Missing Ansible or Ansible Runner for selected Python
An Ansible and/or Ansible Runner installation cannot be found in
the final builder image using the following Python interpreter:
$PYCMD
Ansible and Ansible Runner must be installed in the final image and
available to the selected Python interpreter. If you are using a
recent enough version of the execution environment file, you may use
the 'dependencies.ansible_core' configuration option to install
Ansible and the 'dependencies.ansible_runner' configuration option
to install Ansible Runner. You can also use 'additional_build_steps'
to manually do this yourself. Alternatively, use a base image with
Ansible and Ansible Runner already installed.
**********************************************************************
EOF
exit 1
fi
exit 0

46
context/_build/scripts/check_galaxy Executable file
View File

@ -0,0 +1,46 @@
#!/bin/bash
# Copyright (c) 2023 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#####################################################################
# Script to validate that Ansible Galaxy is installed on the system.
#####################################################################
set -x
ansible-galaxy --version
if [ $? -ne 0 ]
then
cat<<EOF
**********************************************************************
ERROR - Missing Ansible installation
The 'ansible-galaxy' command is not found in the base image. This
image is used to create the intermediary image that performs the
Galaxy collection and role installation process.
Ansible must be installed in the base image. If you are using a
recent enough version of the execution environment file, you may
use the 'dependencies.ansible_core' configuration option to install
Ansible for you, or use 'additional_build_steps' to manually do
this yourself. Alternatively, use a base image with Ansible already
installed.
**********************************************************************
EOF
exit 1
fi
exit 0

152
context/_build/scripts/entrypoint Executable file
View File

@ -0,0 +1,152 @@
#!/usr/bin/env bash
# Copyright: (c) 2023, Ansible Project
# Apache License, Version 2.0 (see LICENSE.md or https://www.apache.org/licenses/LICENSE-2.0)
# This entrypoint script papers over a number of problems that manifest under different container runtimes when
# using ephemeral UIDs, then chain-execs to the requested init system and/or command. It is an implementation
# detail for the convenience of Ansible execution environments built by ansible-builder.
#
# If we're running as a legit user that has an entry in /etc/passwd and a valid and writeable homedir, we're all good.
#
# If the current uid is not in /etc/passwd, we'll attempt to add it, but /etc/passwd is often not writable by GID 0.
# `ansible-builder` defaults to making /etc/passwd writable by GID0 by default for maximum compatibility, but this is
# not guaranteed. Some runtimes/wrappers (eg podman, cri-o) already create an /etc/passwd entry on the fly as-needed,
# but they may set the homedir to something inaccessible (eg, `/`, WORKDIR).
#
# There are numerous cases where a missing or incorrect homedir in /etc/passwd are fatal. It breaks
# `async` in ansible-core, things like `echo ~someuid`, and numerous other software packages that assume a valid POSIX
# user configuration.
#
# If the homedir listed in /etc/passwd is not writeable by the current user (supposed to be primary GID0), we'll try
# to make it writeable (except `/`), or select another writeable home directory from `$HOME`, `/runner`, or `/tmp` and
# update $HOME (and /etc/passwd if possible) accordingly for the current process chain.
#
# This script is generally silent by default, but some likely-fatal cases will issue a brief warning to stderr. The
# envvars described below can be set before container init to cause faster failures and/or get tracing output.
# options:
# EP_BASH_DEBUG=1 (enable set -x)
# EP_DEBUG_TRACE=1 (enable debug trace to stderr)
# EP_ON_ERROR=ignore/warn/fail (default ignore)
set -eu
if (( "${EP_BASH_DEBUG:=0}" == 1 )); then
set -x
fi
: "${EP_DEBUG_TRACE:=0}"
: "${EP_ON_ERROR:=warn}"
: "${HOME:=}"
CUR_UID=$(id -u)
CUR_USERNAME=$(id -u -n 2> /dev/null || true) # whoami-free way to get current username, falls back to current uid
DEFAULT_HOME="/runner"
DEFAULT_SHELL="/bin/bash"
if (( "$EP_DEBUG_TRACE" == 1 )); then
function log_debug() { echo "EP_DEBUG: $1" 1>&2; }
else
function log_debug() { :; }
fi
log_debug "entrypoint.sh started"
case "$EP_ON_ERROR" in
"fail")
function maybe_fail() { echo "EP_FAIL: $1" 1>&2; exit 1; }
;;
"warn")
function maybe_fail() { echo "EP_WARN: $1" 1>&2; }
;;
*)
function maybe_fail() { log_debug "EP_FAIL (ignored): $1"; }
;;
esac
function is_dir_writable() {
[ -d "$1" ] && [ -w "$1" ] && [ -x "$1" ]
}
function ensure_current_uid_in_passwd() {
log_debug "is current uid ${CUR_UID} in /etc/passwd?"
if ! getent passwd "${CUR_USERNAME}" &> /dev/null ; then
if [ -w "/etc/passwd" ]; then
log_debug "appending missing uid ${CUR_UID} into /etc/passwd"
# use the default homedir; we may have to rewrite it to another value later if it's inaccessible
echo "${CUR_UID}:x:${CUR_UID}:0:container user ${CUR_UID}:${DEFAULT_HOME}:${DEFAULT_SHELL}" >> /etc/passwd
else
maybe_fail "uid ${CUR_UID} is missing from /etc/passwd, which is not writable; this error is likely fatal"
fi
else
log_debug "current uid is already in /etc/passwd"
fi
}
function ensure_writeable_homedir() {
if (is_dir_writable "${CANDIDATE_HOME}") ; then
log_debug "candidate homedir ${CANDIDATE_HOME} is valid and writeable"
else
if [ "${CANDIDATE_HOME}" == "/" ]; then
log_debug "skipping attempt to fix permissions on / as homedir"
return 1
fi
log_debug "candidate homedir ${CANDIDATE_HOME} is missing or not writeable; attempt to fix"
if ! (mkdir -p "${CANDIDATE_HOME}" >& /dev/null && chmod -R ug+rwx "${CANDIDATE_HOME}" >& /dev/null) ; then
log_debug "candidate homedir ${CANDIDATE_HOME} cannot be made writeable"
return 1
else
log_debug "candidate homedir ${CANDIDATE_HOME} was successfully made writeable"
fi
fi
# this might work; export it even if we end up not being able to update /etc/passwd
# this ensures the envvar matches current reality for this session; future sessions should set automatically if /etc/passwd is accurate
export HOME=${CANDIDATE_HOME}
if [ "${CANDIDATE_HOME}" == "${PASSWD_HOME}" ] ; then
log_debug "candidate homedir ${CANDIDATE_HOME} matches /etc/passwd"
return 0
fi
if ! [ -w /etc/passwd ]; then
log_debug "candidate homedir ${CANDIDATE_HOME} is valid for ${CUR_USERNAME}, but /etc/passwd is not writable to update it"
return 1
fi
log_debug "resetting homedir for user ${CUR_USERNAME} to ${CANDIDATE_HOME} in /etc/passwd"
# sed -i wants to create a tempfile next to the original, which won't work with /etc permissions in many cases,
# so just do it in memory and overwrite the existing file if we succeeded
NEWPW=$(sed -r "s;(^${CUR_USERNAME}:(.*:){4})(.*:);\1${CANDIDATE_HOME}:;g" /etc/passwd)
echo "${NEWPW}" > /etc/passwd
}
ensure_current_uid_in_passwd
log_debug "current value of HOME is ${HOME}"
PASSWD_HOME=$(getent passwd "${CUR_USERNAME}" | cut -d: -f6)
log_debug "user ${CUR_USERNAME} homedir from /etc/passwd is ${PASSWD_HOME}"
CANDIDATE_HOMES=("${PASSWD_HOME}" "${HOME}" "${DEFAULT_HOME}" "/tmp")
# we'll set this in the loop as soon as we find a writeable dir
unset HOME
for CANDIDATE_HOME in "${CANDIDATE_HOMES[@]}"; do
if ensure_writeable_homedir ; then
break
fi
done
if ! [ -v HOME ] ; then
maybe_fail "a valid homedir could not be set for ${CUR_USERNAME}; this is likely fatal"
fi
# chain exec whatever we were asked to run (ideally an init system) to keep any envvar state we've set
log_debug "chain exec-ing requested command $*"
exec "${@}"

105
context/_build/scripts/install-from-bindep Executable file
View File

@ -0,0 +1,105 @@
#!/bin/bash
# Copyright (c) 2019 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -ex
# NOTE(pabelanger): Allow users to force either microdnf or dnf as a package
# manager.
PKGMGR="${PKGMGR:-}"
PKGMGR_OPTS="${PKGMGR_OPTS:-}"
PKGMGR_PRESERVE_CACHE="${PKGMGR_PRESERVE_CACHE:-}"
PYCMD="${PYCMD:=/usr/bin/python3}"
PIPCMD="${PIPCMD:=$PYCMD -m pip}"
PIP_OPTS="${PIP_OPTS-}"
if [ -z $PKGMGR ]; then
# Expect dnf to be installed, however if we find microdnf default to it.
PKGMGR=/usr/bin/dnf
if [ -f "/usr/bin/microdnf" ]; then
PKGMGR=/usr/bin/microdnf
fi
fi
if [ "$PKGMGR" = "/usr/bin/microdnf" ]
then
if [ -z "${PKGMGR_OPTS}" ]; then
# NOTE(pabelanger): skip install docs and weak dependencies to
# make smaller images. Sadly, setting these in dnf.conf don't
# appear to work.
PKGMGR_OPTS="--nodocs --setopt install_weak_deps=0"
fi
fi
if [ -f /output/bindep/run.txt ] ; then
PACKAGES=$(cat /output/bindep/run.txt)
if [ ! -z "$PACKAGES" ]; then
$PKGMGR install -y $PKGMGR_OPTS $PACKAGES
fi
fi
if [ -f /output/bindep/epel.txt ] ; then
EPEL_PACKAGES=$(cat /output/bindep/epel.txt)
if [ ! -z "$EPEL_PACKAGES" ]; then
$PKGMGR install -y $PKGMGR_OPTS --enablerepo epel $EPEL_PACKAGES
fi
fi
# If there's a constraints file, use it.
if [ -f /output/upper-constraints.txt ] ; then
CONSTRAINTS="-c /output/upper-constraints.txt"
fi
# If a requirements.txt file exists,
# install it directly so that people can use git url syntax
# to do things like pick up patched but unreleased versions
# of dependencies.
if [ -f /output/requirements.txt ] ; then
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels -r /output/requirements.txt
fi
# Add any requested extras to the list of things to install
EXTRAS=""
for extra in $* ; do
EXTRAS="${EXTRAS} -r /output/$extra/requirements.txt"
done
if [ -f /output/packages.txt ] ; then
# If a package list was passed to assemble, install that in the final
# image.
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels -r /output/packages.txt $EXTRAS
else
# Install the wheels. Uninstall any existing version as siblings maybe
# be built with the same version number as the latest release, but we
# really want the speculatively built wheels installed over any
# automatic dependencies.
# NOTE(pabelanger): It is possible a project may not have a wheel, but does have requirements.txt
if [ $(ls -1 /output/wheels/*whl 2>/dev/null | wc -l) -gt 0 ]; then
$PIPCMD uninstall -y /output/wheels/*.whl
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels /output/wheels/*.whl $EXTRAS
elif [ ! -z "$EXTRAS" ] ; then
$PIPCMD uninstall -y $EXTRAS
$PIPCMD install $CONSTRAINTS $PIP_OPTS --cache-dir=/output/wheels $EXTRAS
fi
fi
# clean up after ourselves, unless requested to keep the cache
if [[ "$PKGMGR_PRESERVE_CACHE" != always ]]; then
$PKGMGR clean all
rm -rf /var/cache/{dnf,yum}
fi
rm -rf /var/lib/dnf/history.*
rm -rf /var/log/{dnf.*,hawkey.log}

507
context/_build/scripts/introspect.py Normal file
View File

@ -0,0 +1,507 @@
from __future__ import annotations
import argparse
import logging
import os
import re
import sys
import yaml
from packaging.requirements import InvalidRequirement, Requirement
BASE_COLLECTIONS_PATH = '/usr/share/ansible/collections'
# regex for a comment at the start of a line, or embedded with leading space(s)
COMMENT_RE = re.compile(r'(?:^|\s+)#.*$')
EXCLUDE_REQUIREMENTS = frozenset((
# obviously already satisfied or unwanted
'ansible', 'ansible-base', 'python', 'ansible-core',
# general python test requirements
'tox', 'pycodestyle', 'yamllint', 'pylint',
'flake8', 'pytest', 'pytest-xdist', 'coverage', 'mock', 'testinfra',
# test requirements highly specific to Ansible testing
'ansible-lint', 'molecule', 'galaxy-importer', 'voluptuous',
# already present in image for py3 environments
'yaml', 'pyyaml', 'json',
))
logger = logging.getLogger(__name__)
class CollectionDefinition:
"""
This class represents the dependency metadata for a collection
should be replaced by logic to hit the Galaxy API if made available
"""
def __init__(self, collection_path):
self.reference_path = collection_path
# NOTE: Filenames should match constants.DEAFULT_EE_BASENAME and constants.YAML_FILENAME_EXTENSIONS.
meta_file_base = os.path.join(collection_path, 'meta', 'execution-environment')
ee_exists = False
for ext in ('yml', 'yaml'):
meta_file = f"{meta_file_base}.{ext}"
if os.path.exists(meta_file):
with open(meta_file, 'r') as f:
self.raw = yaml.safe_load(f)
ee_exists = True
break
if not ee_exists:
self.raw = {'version': 1, 'dependencies': {}}
# Automatically infer requirements for collection
for entry, filename in [('python', 'requirements.txt'), ('system', 'bindep.txt')]:
candidate_file = os.path.join(collection_path, filename)
if has_content(candidate_file):
self.raw['dependencies'][entry] = filename
def target_dir(self):
namespace, name = self.namespace_name()
return os.path.join(
BASE_COLLECTIONS_PATH, 'ansible_collections',
namespace, name
)
def namespace_name(self):
"Returns 2-tuple of namespace and name"
path_parts = [p for p in self.reference_path.split(os.path.sep) if p]
return tuple(path_parts[-2:])
def get_dependency(self, entry):
"""A collection is only allowed to reference a file by a relative path
which is relative to the collection root
"""
req_file = self.raw.get('dependencies', {}).get(entry)
if req_file is None:
return None
if os.path.isabs(req_file):
raise RuntimeError(
'Collections must specify relative paths for requirements files. '
f'The file {req_file} specified by {self.reference_path} violates this.'
)
return req_file
def line_is_empty(line):
return bool((not line.strip()) or line.startswith('#'))
def read_req_file(path):
"""Provide some minimal error and display handling for file reading"""
if not os.path.exists(path):
print(f'Expected requirements file not present at: {os.path.abspath(path)}')
with open(path, 'r') as f:
return f.read()
def pip_file_data(path):
pip_content = read_req_file(path)
pip_lines = []
for line in pip_content.split('\n'):
if line_is_empty(line):
continue
if line.startswith('-r') or line.startswith('--requirement'):
_, new_filename = line.split(None, 1)
new_path = os.path.join(os.path.dirname(path or '.'), new_filename)
pip_lines.extend(pip_file_data(new_path))
else:
pip_lines.append(line)
return pip_lines
def bindep_file_data(path):
sys_content = read_req_file(path)
sys_lines = []
for line in sys_content.split('\n'):
if line_is_empty(line):
continue
sys_lines.append(line)
return sys_lines
def process_collection(path):
"""Return a tuple of (python_dependencies, system_dependencies) for the
collection install path given.
Both items returned are a list of dependencies.
:param str path: root directory of collection (this would contain galaxy.yml file)
"""
col_def = CollectionDefinition(path)
py_file = col_def.get_dependency('python')
pip_lines = []
if py_file:
pip_lines = pip_file_data(os.path.join(path, py_file))
sys_file = col_def.get_dependency('system')
bindep_lines = []
if sys_file:
bindep_lines = bindep_file_data(os.path.join(path, sys_file))
return (pip_lines, bindep_lines)
def process(data_dir=BASE_COLLECTIONS_PATH,
user_pip=None,
user_bindep=None,
exclude_pip=None,
exclude_bindep=None,
exclude_collections=None):
"""
Build a dictionary of Python and system requirements from any collections
installed in data_dir, and any user specified requirements.
Excluded requirements, if any, will be inserted into the return dict.
Example return dict:
{
'python': {
'collection.a': ['abc', 'def'],
'collection.b': ['ghi'],
'user': ['jkl'],
'exclude: ['abc'],
},
'system': {
'collection.a': ['ZYX'],
'user': ['WVU'],
'exclude': ['ZYX'],
},
'excluded_collections': [
'a.b',
]
}
"""
paths = []
path_root = os.path.join(data_dir, 'ansible_collections')
# build a list of all the valid collection paths
if os.path.exists(path_root):
for namespace in sorted(os.listdir(path_root)):
if not os.path.isdir(os.path.join(path_root, namespace)):
continue
for name in sorted(os.listdir(os.path.join(path_root, namespace))):
collection_dir = os.path.join(path_root, namespace, name)
if not os.path.isdir(collection_dir):
continue
files_list = os.listdir(collection_dir)
if 'galaxy.yml' in files_list or 'MANIFEST.json' in files_list:
paths.append(collection_dir)
# populate the requirements content
py_req = {}
sys_req = {}
for path in paths:
col_pip_lines, col_sys_lines = process_collection(path)
col_def = CollectionDefinition(path)
namespace, name = col_def.namespace_name()
key = f'{namespace}.{name}'
if col_pip_lines:
py_req[key] = col_pip_lines
if col_sys_lines:
sys_req[key] = col_sys_lines
# add on entries from user files, if they are given
if user_pip:
col_pip_lines = pip_file_data(user_pip)
if col_pip_lines:
py_req['user'] = col_pip_lines
if exclude_pip:
col_pip_exclude_lines = pip_file_data(exclude_pip)
if col_pip_exclude_lines:
py_req['exclude'] = col_pip_exclude_lines
if user_bindep:
col_sys_lines = bindep_file_data(user_bindep)
if col_sys_lines:
sys_req['user'] = col_sys_lines
if exclude_bindep:
col_sys_exclude_lines = bindep_file_data(exclude_bindep)
if col_sys_exclude_lines:
sys_req['exclude'] = col_sys_exclude_lines
retval = {
'python': py_req,
'system': sys_req,
}
if exclude_collections:
# This file should just be a newline separated list of collection names,
# so reusing bindep_file_data() to read it should work fine.
excluded_collection_list = bindep_file_data(exclude_collections)
if excluded_collection_list:
retval['excluded_collections'] = excluded_collection_list
return retval
def has_content(candidate_file):
"""Beyond checking that the candidate exists, this also assures
that the file has something other than whitespace,
which can cause errors when given to pip.
"""
if not os.path.exists(candidate_file):
return False
with open(candidate_file, 'r') as f:
content = f.read()
return bool(content.strip().strip('\n'))
def strip_comments(reqs: dict[str, list]) -> dict[str, list]:
"""
Filter any comments out of the Python collection requirements input.
:param dict reqs: A dict of Python requirements, keyed by collection name.
:return: Same as the input parameter, except with no comment lines.
"""
result: dict[str, list] = {}
for collection, lines in reqs.items():
for line in lines:
# strip comments
if (base_line := COMMENT_RE.sub('', line.strip())):
result.setdefault(collection, []).append(base_line)
return result
def should_be_excluded(value: str, exclusion_list: list[str]) -> bool:
"""
Test if `value` matches against any value in `exclusion_list`.
The exclusion_list values are either strings to be compared in a case-insensitive
manner against value, OR, they are regular expressions to be tested against the
value. A regular expression will contain '~' as the first character.
:return: True if the value should be excluded, False otherwise.
"""
for exclude_value in exclusion_list:
if exclude_value[0] == "~":
pattern = exclude_value[1:]
if re.fullmatch(pattern.lower(), value.lower()):
return True
elif exclude_value.lower() == value.lower():
return True
return False
def filter_requirements(reqs: dict[str, list],
exclude: list[str] | None = None,
exclude_collections: list[str] | None = None,
is_python: bool = True) -> list[str]:
"""
Given a dictionary of Python requirement lines keyed off collections,
return a list of cleaned up (no source comments) requirements
annotated with comments indicating the sources based off the collection keys.
Currently, non-pep508 compliant Python entries are passed through. We also no
longer attempt to normalize names (replace '_' with '-', etc), other than
lowercasing it for exclusion matching, since we no longer are attempting
to combine similar entries.
:param dict reqs: A dict of either Python or system requirements, keyed by collection name.
:param list exclude: A list of requirements to be excluded from the output.
:param list exclude_collections: A list of collection names from which to exclude all requirements.
:param bool is_python: This should be set to True for Python requirements, as each
will be tested for PEP508 compliance. This should be set to False for system requirements.
:return: A list of filtered and annotated requirements.
"""
exclusions: list[str] = []
collection_ignore_list: list[str] = []
if exclude:
exclusions = exclude.copy()
if exclude_collections:
collection_ignore_list = exclude_collections.copy()
annotated_lines: list[str] = []
uncommented_reqs = strip_comments(reqs)
for collection, lines in uncommented_reqs.items():
# Bypass this collection if we've been told to ignore all requirements from it.
if should_be_excluded(collection, collection_ignore_list):
logger.debug("# Excluding all requirements from collection '%s'", collection)
continue
for line in lines:
# Determine the simple name based on type of requirement
if is_python:
try:
parsed_req = Requirement(line)
name = parsed_req.name
except InvalidRequirement:
logger.warning(
"Passing through non-PEP508 compliant line '%s' from collection '%s'",
line, collection
)
annotated_lines.append(line) # We intentionally won't annotate these lines (multi-line?)
continue
else:
# bindep system requirements have the package name as the first "word" on the line
name = line.split(maxsplit=1)[0]
if collection.lower() not in {'user', 'exclude'}:
lower_name = name.lower()
if lower_name in EXCLUDE_REQUIREMENTS:
logger.debug("# Excluding requirement '%s' from '%s'", name, collection)
continue
if should_be_excluded(lower_name, exclusions):
logger.debug("# Explicitly excluding requirement '%s' from '%s'", name, collection)
continue
annotated_lines.append(f'{line} # from collection {collection}')
return annotated_lines
def parse_args(args=None):
parser = argparse.ArgumentParser(
prog='introspect',
description=(
'ansible-builder introspection; injected and used during execution environment build'
)
)
subparsers = parser.add_subparsers(
help='The command to invoke.',
dest='action',
required=True,
)
create_introspect_parser(subparsers)
return parser.parse_args(args)
def run_introspect(args, log):
data = process(args.folder,
user_pip=args.user_pip,
user_bindep=args.user_bindep,
exclude_pip=args.exclude_pip,
exclude_bindep=args.exclude_bindep,
exclude_collections=args.exclude_collections)
log.info('# Dependency data for %s', args.folder)
excluded_collections = data.pop('excluded_collections', None)
data['python'] = filter_requirements(
data['python'],
exclude=data['python'].pop('exclude', []),
exclude_collections=excluded_collections,
)
data['system'] = filter_requirements(
data['system'],
exclude=data['system'].pop('exclude', []),
exclude_collections=excluded_collections,
is_python=False
)
print('---')
print(yaml.dump(data, default_flow_style=False))
if args.write_pip and data.get('python'):
write_file(args.write_pip, data.get('python') + [''])
if args.write_bindep and data.get('system'):
write_file(args.write_bindep, data.get('system') + [''])
sys.exit(0)
def create_introspect_parser(parser):
introspect_parser = parser.add_parser(
'introspect',
help='Introspects collections in folder.',
description=(
'Loops over collections in folder and returns data about dependencies. '
'This is used internally and exposed here for verification. '
'This is targeted toward collection authors and maintainers.'
)
)
introspect_parser.add_argument('--sanitize', action='store_true',
help=argparse.SUPPRESS)
introspect_parser.add_argument(
'folder', default=BASE_COLLECTIONS_PATH, nargs='?',
help=(
'Ansible collections path(s) to introspect. '
'This should have a folder named ansible_collections inside of it.'
)
)
introspect_parser.add_argument(
'--user-pip', dest='user_pip',
help='An additional file to combine with collection pip requirements.'
)
introspect_parser.add_argument(
'--user-bindep', dest='user_bindep',
help='An additional file to combine with collection bindep requirements.'
)
introspect_parser.add_argument(
'--exclude-bindep-reqs', dest='exclude_bindep',
help='An additional file to exclude specific bindep requirements from collections.'
)
introspect_parser.add_argument(
'--exclude-pip-reqs', dest='exclude_pip',
help='An additional file to exclude specific pip requirements from collections.'
)
introspect_parser.add_argument(
'--exclude-collection-reqs', dest='exclude_collections',
help='An additional file to exclude all requirements from the listed collections.'
)
introspect_parser.add_argument(
'--write-pip', dest='write_pip',
help='Write the combined pip requirements file to this location.'
)
introspect_parser.add_argument(
'--write-bindep', dest='write_bindep',
help='Write the combined bindep requirements file to this location.'
)
return introspect_parser
def write_file(filename: str, lines: list) -> bool:
parent_dir = os.path.dirname(filename)
if parent_dir and not os.path.exists(parent_dir):
logger.warning('Creating parent directory for %s', filename)
os.makedirs(parent_dir)
new_text = '\n'.join(lines)
if os.path.exists(filename):
with open(filename, 'r') as f:
if f.read() == new_text:
logger.debug("File %s is already up-to-date.", filename)
return False
logger.warning('File %s had modifications and will be rewritten', filename)
with open(filename, 'w') as f:
f.write(new_text)
return True
def main():
args = parse_args()
if args.action == 'introspect':
run_introspect(args, logger)
logger.error("An error has occurred.")
sys.exit(1)
if __name__ == '__main__':
main()

56
context/_build/scripts/pip_install Executable file
View File

@ -0,0 +1,56 @@
#!/bin/bash
# Copyright (c) 2024 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#####################################################################
# Script to encapsulate pip installation.
#
# Usage: pip_install <PYCMD>
#
# Options:
# PYCMD - The path to the python executable to use.
#####################################################################
set -x
PYCMD=$1
if [ -z "$PYCMD" ]
then
echo "Usage: pip_install <PYCMD>"
exit 1
fi
if [ ! -x "$PYCMD" ]
then
echo "$PYCMD is not an executable"
exit 1
fi
# This is going to be our default functionality for now. This will likely
# need to change if we add support for non-RHEL distros.
$PYCMD -m ensurepip --root /
if [ $? -ne 0 ]
then
cat<<EOF
**********************************************************************
ERROR - pip installation failed for Python $PYCMD
**********************************************************************
EOF
exit 1
fi
exit 0

33
execution-environment.yml Normal file
View File

@ -0,0 +1,33 @@
version: 3
images:
base_image:
name: quay.io/centos/centos:stream9
dependencies:
python: requirements.txt
galaxy: collections/requirements.yml
python_interpreter:
package_system: python3.12
python_path: /usr/bin/python3.12
ansible_core:
package_pip: ansible-core>=2.17.0
ansible_runner:
package_pip: ansible-runner==2.4.0
system: |
git-core [platform:rpm]
python3.11-devel [platform:rpm compile]
libcurl-devel [platform:rpm compile]
krb5-devel [platform:rpm compile]
krb5-workstation [platform:rpm]
subversion [platform:rpm]
subversion [platform:dpkg]
git-lfs [platform:rpm]
sshpass [platform:rpm]
rsync [platform:rpm]
epel-release [platform:rpm]
unzip [platform:rpm]
podman-remote [platform:rpm]
cmake [platform:rpm compile]
gcc [platform:rpm compile]
gcc-c++ [platform:rpm compile]
make [platform:rpm compile]
openssl-devel [platform:rpm compile]

23
inventories/group_vars/all.yml Normal file
View File

@ -0,0 +1,23 @@
hostname_fqdn: "{{inventory_hostname_short}}.{{host_domain}}"
ansible_host: "{{hostname_fqdn}}"
passbolt: 'anatomicjc.passbolt.passbolt'
passbolt_inventory: 'anatomicjc.passbolt.passbolt_inventory'
sysadmin_mail: sysadmin@domain.local
sysadmin_group: sysadmin
smtp_server: smtp.domain.local
homes_server: homes.domain.local
nagios_server: nagios.domain.local
time_server: time1.domain.local time2.domain.local
main_dns_server: ns1.domain.local
ldap_uri: ldap://ldap.domain.local
ldap_base: dc=domain,dc=local
dc_net: "10.0.0.0/16"
resolvers:
- '8.8.8.8'
- '8.8.4.4'
awx_email: awx@domain.local
awx_pub_key: >
ssh-ed25519
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
awx@domain.local
passbolt_folder: 00000000-0000-0000-0000-000000000000

20
inventories/template Normal file
View File

@ -0,0 +1,20 @@
[all:vars]
host_domain=domain.local
[pve:vars]
host_domain=core.domain.local
[ceph]
ceph[1:3]
[pve]
pve[1:5]
[infra:children]
ceph
pve
[servers]
server1 ansible_host=10.0.0.1
server1 ansible_host=10.0.0.2
server3 ansible_host=10.0.0.3

7
playbooks/awx.yml Normal file
View File

@ -0,0 +1,7 @@
- name: Configure AWX user
hosts: all
gather_facts: no
become: no
tasks:
- import_role:
name: awx

5
playbooks/ceph.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Configure Ceph
hosts: all
tasks:
- import_role:
name: ceph

36
playbooks/clean.yml Normal file
View File

@ -0,0 +1,36 @@
- name: Clean deprecated configuration
hosts: all
gather_facts: no
tasks:
- name: Delete awx-user
user:
name: awx-user
state: absent
remove: yes
tags: awx
- name: Delete awx-user sudoers file
file:
path: /etc/sudoers.d/awx-user
state: absent
tags: awx
- name: Delete old MOTD configuration
file:
path: /etc/profile.d/mymotd.sh
state: absent
tags: motd
- name: Delete old Ansible bashrc configuration
blockinfile:
path: /root/.bashrc
marker_begin: 'BEGIN ANSIBLE MANAGED BLOCK'
marker_end: 'END ANSIBLE MANAGED BLOCK'
marker: "# {mark}"
state: absent
tags: bashrc
- name: Delete old custom bashrc configuration
replace:
path: /root/.bashrc
regexp: '{{ start_delimiter }}\\s\\S*?{{ end_delimiter }}'
replace: ''
vars:
start_delimiter: '### 4Loo'
end_delimiter: 'esac'

5
playbooks/debian-once.yml Normal file
View File

@ -0,0 +1,5 @@
- name: First time host configuration
hosts: all
tasks:
- import_role:
name: debian-once

5
playbooks/debian-upgrade.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Upgrade Debian host
hosts: all
tasks:
- import_role:
name: debian-upgrade

18
playbooks/debian.yml Normal file
View File

@ -0,0 +1,18 @@
- name: Configure base Debian host
hosts: all
tasks:
- name: Configure virtual machine or host (not LXC)
import_role:
name: debian-host
when: ansible_virtualization_role == 'host' or ansible_virtualization_type == 'kvm'
- name: Configure base system (all)
import_role:
name: debian-base
- name: Configure guest
import_role:
name: debian-guest
when: ansible_virtualization_role == 'guest'
- name: Configure virtual machine
import_role:
name: debian-qemu
when: ansible_virtualization_type == 'kvm'

10
playbooks/facts.yml Normal file
View File

@ -0,0 +1,10 @@
- name: Gather facts from host
hosts: all
gather_facts: yes
tasks:
- name: Print all available facts
debug:
var: ansible_facts
- name: Print variable value
debug:
msg: "Variable: {{ ansible_fqdn }}"

5
playbooks/freeradius.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Install and configure Freeradius with TOTP
hosts: all
tasks:
- import_role:
name: freeradius

5
playbooks/kube.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Configure Kubernetes
hosts: all
tasks:
- import_role:
name: kube

5
playbooks/nsupdate.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Configure zone with nsupdate
hosts: ns1
tasks:
- import_role:
name: nsupdate

12
playbooks/passbolt.yml Normal file
View File

@ -0,0 +1,12 @@
- name: Fetch or create passbolt password
hosts: all
gather_facts: no
tasks:
- debug:
msg: "{{ lookup(passbolt, 'test', password=passbolt_password) }}"
vars:
passbolt_password: 'S3cR3tP4$$w0rd'
environment:
PASSBOLT_CREATE_NEW_RESOURCE: true
PASSBOLT_NEW_RESOURCE_PASSWORD_LENGTH: 18
PASSBOLT_NEW_RESOURCE_PASSWORD_SPECIAL_CHARS: false

6
playbooks/ping.yml Normal file
View File

@ -0,0 +1,6 @@
- name: Check whether host is alive and reachable
hosts: all
gather_facts: no
become: no
tasks:
- ping:

5
playbooks/pve.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Configure PVE
hosts: all
tasks:
- import_role:
name: pve

6
playbooks/send-mail.yml Normal file
View File

@ -0,0 +1,6 @@
- name: Send mail
hosts: localhost
become: no
tasks:
- import_role:
name: send-mail

4
playbooks/win-ping.yml Normal file
View File

@ -0,0 +1,4 @@
- name: Ping Windows host
hosts: all
tasks:
- ansible.windows.win_ping:

29
playbooks/win-update.yml Normal file
View File

@ -0,0 +1,29 @@
- name: Update windows host
hosts: all
serial: 1
tasks:
- block:
- name: Check if there are missing updates
win_updates: state=searched
register: update_count
- block:
- name: Install missing updates only if at least one is missing
win_updates:
category_names: '*'
#- Application
#- Connectors
#- DefinitionUpdates
#- DeveloperKits
#- FeaturePacks
#- Guidance
#- ServicePacks
#- Tools
#- UpdateRollups
#- CriticalUpdates
#- SecurityUpdates
log_path: C:\Win_Template_Patch.log
register: update_result
- name: Reboot, if needed.
win_reboot:
when: update_result.reboot_required
when: update_count.found_update_count|int >= 1

3
requirements.txt Normal file
View File

@ -0,0 +1,3 @@
py-passbolt==0.0.18
cryptography==3.3.2
passlib==1.7.4

1
roles/awx/files/sudoers Normal file
View File

@ -0,0 +1 @@
awx ALL=(ALL) NOPASSWD:ALL

23
roles/awx/tasks/main.yaml Normal file
View File

@ -0,0 +1,23 @@
- name: Create awx user
user:
name: awx
password: '*'
shell: /bin/bash
groups: sudo
state: present
comment: ssh user
- name: Add awx SSH public key
authorized_key:
user: awx
key: "{{ awx_pub_key }}"
- name: Install sudo package
apt:
name: sudo
state: present
- name: Add awx user to sudoers
copy:
src: sudoers
dest: /etc/sudoers.d/awx
mode: u=rw,g=r
owner: root
group: root

5
roles/ceph/files/nrpe.cfg Normal file
View File

@ -0,0 +1,5 @@
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 700 -c 1000
command[check_chrony]=/usr/lib/nagios/plugins/check_chrony 1 2
command[check_smartdisk]=/etc/nagios/plugins/check_smartdisk.sh /dev/sda /dev/sdb
command[check_raid]=/usr/lib/nagios/plugins/check_raid
command[check_mon]=/etc/nagios/plugins/check_ceph_mon --id nagios --monid "`hostname`"

4
roles/ceph/handlers/main.yml Normal file
View File

@ -0,0 +1,4 @@
- name: restart-nrpe
service:
name: nagios-nrpe-server
state: restarted

8
roles/ceph/tasks/main.yml Normal file
View File

@ -0,0 +1,8 @@
- name: Set NRPE Ceph configuration
copy:
src: nrpe.cfg
dest: /etc/nagios/nrpe.d/95-ceph.cfg
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart-nrpe

31
roles/debian-base/defaults/main.yaml Normal file
View File

@ -0,0 +1,31 @@
vn_witness: false
default_user: user
root_password: Pa$$w0rd
fail2ban:
email: "{{ sysadmin_mail }}"
bantime: 600
maxretry: 4
ignore: "127.0.0.0/8 {{ dc_net }}"
logpath: "/var/log/auth.log"
fail2ban_base_packages:
- fail2ban
- rsyslog
time_server_spain: ntp.roa.es
nagios_packages:
- nagios-nrpe-server
- nagios-plugins-contrib
- monitoring-plugins-basic
base_packages:
- htop
- psmisc
- bash-completion
- screen
- aptitude
- tree
- btop
- ncdu
- debconf-utils
- net-tools
locales_present:
- en_US.UTF-8
- es_ES.UTF-8

102
roles/debian-base/files/motd Normal file
View File

@ -0,0 +1,102 @@
#!/bin/bash
LABEL="\033[0;32m"
SUBLB="\033[0;36m"
RESET="\033[0m"
BLINK="\033[5m"
. /etc/os-release
# Environment
PRO="\033[1;5;31m"
LAB="\033[0;35m"
VN="\033[0;32m"
UNKNOWN="\033[0;33m"
FQDN=$(hostname --fqdn)
case "$FQDN" in
*.dc.verdnatura.es)
ENVIRONMENT="${PRO}Production${RESET}"
;;
*.lab.verdnatura.es)
ENVIRONMENT="${LAB}Laboratory${RESET}"
;;
*.verdnatura.es)
ENVIRONMENT="${VN}Verdnatura${RESET}"
;;
*)
ENVIRONMENT="${UNKNOWN}Unknown${RESET}"
;;
esac
# Last login
LAST="$(last -n1 | head -1)"
if [ "$LAST" != "" ] ; then
LAST_LOGIN_USER="$(echo $LAST | cut -d' ' -f1)"
LAST_LOGIN_IP="$(echo $LAST | cut -d' ' -f3)"
LAST_LOGIN_TIME="$(echo $LAST | cut -d' ' -f4-)"
LAST_LOGIN="$LAST_LOGIN_USER ($LAST_LOGIN_IP) $LAST_LOGIN_TIME"
fi
# Packages
PACKAGES=$(dpkg-query -W -f='${binary:Package}\n' | wc -l)
SHOW_UPGRADEABLE=0
USER_UID=$(id -u $USER)
USER_GROUPS=$(id -Gn $USER)
if [ ${USER_UID} -eq 0 ] ; then
SHOW_UPGRADEABLE=1
fi
if [[ "${USER_GROUPS}" == *"sysadmin"* ]] ; then
SHOW_UPGRADEABLE=1
fi
if [ $SHOW_UPGRADEABLE -eq 1 ] ; then
UPGRADEABLE="$(apt list --upgradable 2>/dev/null | tail -n +2 | wc -l)"
if [ "$UPGRADEABLE" -gt 0 ]; then
UPGRADEABLE_ALERT="${BLINK}($UPGRADEABLE upgradeable)${RESET}"
fi
fi
# Network and users
NET_IPS=$(ip -o -4 add sh | tail -n +2 | awk '{printf "\033[0;36m%16s >\033[0m %-15s\n",$2,$4}')
CONNECTED_USERS=$(w | tail -n +2 | awk '{printf "\t%15s %12s %8s %s\n",$1,$3,$4, substr( $0, index($0,$8), index($0,$NF)) }')
# Logo
FW="\033[1;37m"
FG="\033[1;32m"
GB="\e[48;5;112m"
GL="\e[48;5;70m"
RS="\e[0m"
echo -e
echo -e " $GL $GB $RS"
echo -e " $GL $GB $GL $RS $GL $GB $RS" " ${FW}__ __ _ ${FG} _ _ _ ${RS}"
echo -e " $GB $GL $RS $GL $GB $GL $RS" " ${FW}\ \ / /__ _ __ __| |${FG}| \ | | __ _| |_ _ _ _ __ __ _ ${RS}"
echo -e " $GL $GB $RS $GB $GL $RS " " ${FW} \ \ / / _ \ '__/ _' |${FG}| \| |/ _' | __| | | | '__/ _' |${RS}"
echo -e " $GL $GB $RS $GB $GL $RS " " ${FW} \ V / __/ | | (_| |${FG}| |\ | (_| | |_| |_| | | | (_| |${RS}"
echo -e " " " ${FW} \_/ \___|_| \__,_|${FG}|_| \_|\__,_|\__|\__,_|_| \__,_|${RS}"
echo -e " $GL $GB $RS $GB $GL $RS"
echo -e " $GL $GB $GL $RS $GL $GB $RS"
echo -e " $GB $GL $RS"
echo -e
# Information
echo -e "${LABEL}Host :${RESET} $FQDN"
echo -e "${LABEL}OS :${RESET} $NAME $(cat /etc/debian_version) ($VERSION_CODENAME)"
echo -e "${LABEL}Kernel :${RESET} $(uname -r)"
echo -e "${LABEL}Shell :${RESET} $SHELL $(echo $BASH_VERSION | cut -d'(' -f1)"
echo -e "${LABEL}Uptime :${RESET} $(uptime -p | tr -d ',')"
echo -e "${LABEL}Packages :${RESET} $PACKAGES $UPGRADEABLE_ALERT"
echo -e "${LABEL}IP :${RESET}"
echo -e "$NET_IPS"
echo -e "${LABEL}Last Login :${RESET} $LAST_LOGIN"
echo -e "${LABEL}Environment :${RESET} $ENVIRONMENT"
echo -e "${LABEL}Connected users :${RESET}"
echo -e "$CONNECTED_USERS"

47
roles/debian-base/files/profile.sh Normal file
View File

@ -0,0 +1,47 @@
#!/bin/bash
# Prompt
FQDN=$(hostname --fqdn)
if [[ $FQDN == *.verdnatura.es ]]; then
SHORT_HOST=${FQDN%.verdnatura.es}
case "$SHORT_HOST" in
*.dc)
ENVIRONMENT="\[\033[01;31m\]PRO\[\033[00m\]"
;;
*.lab)
ENVIRONMENT="\[\033[01;35m\]LAB\[\033[00m\]"
;;
*)
ENVIRONMENT="\[\033[01;32m\]VN\[\033[00m\]"
;;
esac
PS1="\u@$SHORT_HOST[$ENVIRONMENT]:\w"
if [ "$(id -u)" -eq 0 ]; then
PS1="$PS1# "
else
PS1="$PS1\$ "
fi
fi
# History
HISTSIZE=10000
HISTFILESIZE=50000
HISTTIMEFORMAT="%Y-%m-%d %H:%M:%S "
# Security
TMOUT=3600
# Aliases
#export LS_OPTIONS='--color=auto'
#eval "$(dircolors)"
#alias ls='ls $LS_OPTIONS'
#alias ll='ls $LS_OPTIONS -l'
#alias la='ls $LS_OPTIONS -la'

5
roles/debian-base/files/vimrc.local Normal file
View File

@ -0,0 +1,5 @@
syntax on
set showcmd
set showmatch
set ignorecase
set smartcase

26
roles/debian-base/handlers/main.yml Normal file
View File

@ -0,0 +1,26 @@
- name: restart systemd-timesyncd
systemd:
name: systemd-timesyncd
state: restarted
- name: restart-ssh
systemd:
name: ssh
state: restarted
- name: restart fail2ban
systemd:
name: fail2ban
state: restarted
- name: restart-nrpe
systemd:
name: nagios-nrpe-server
state: restarted
- name: restart sshd
systemd:
name: sshd
state: restarted
- name: generate locales
command: /usr/sbin/locale-gen
- name: reconfigure tzdata
command: dpkg-reconfigure -f noninteractive tzdata
- name: update exim configuration
command: /usr/sbin/update-exim4.conf

50
roles/debian-base/tasks/bacula.yml Normal file
View File

@ -0,0 +1,50 @@
- name: Install Bacula FD packages
apt:
name: bacula-fd
state: present
- name: Read content file in base64
slurp:
src: /etc/bacula/common_default_passwords
register: file_content
- name: Going to text plane
set_fact:
file_content_decoded: "{{ file_content.content | b64decode }}"
- name: Extracting passwords
set_fact:
passwords: "{{ file_content_decoded.splitlines() | select('match', '^[^#]') | map('regex_replace', '^([^=]+)=(.+)$', '\\1:\\2') | list }}"
- name: Initialize password dictionary
set_fact:
bacula_passwords: {}
- name: Convert lines to individual variables generating a new dict
set_fact:
bacula_passwords: "{{ bacula_passwords | combine({item.split(':')[0].lower(): item.split(':')[1] | regex_replace('\\n$', '') }) }}"
loop: "{{ passwords }}"
when: "'FDPASSWD' in item or 'FDMPASSWD' in item"
- name: Configure Bacula FD
template:
src: bacula-fd.conf
dest: /etc/bacula/bacula-fd.conf
owner: root
group: bacula
mode: u=rw,g=r,o=
backup: true
register: bacula_config
- name: Configure master cert
copy:
content: "{{ master_cert_content }}"
dest: /etc/bacula/master-cert.pem
owner: root
group: root
mode: u=rw,g=r,o=r
- name: Configure master cert
copy:
content: "{{ lookup(passbolt, 'fd-cert.pem', folder_parent_id=passbolt_folder).description }}"
dest: /etc/bacula/fd-cert.pem
owner: root
group: bacula
mode: u=rw,g=r,o=
- name: Restart Bacula FD service
service:
name: bacula-fd
state: restarted
when: bacula_config.changed

5
roles/debian-base/tasks/defuser.yml Normal file
View File

@ -0,0 +1,5 @@
- name: Delete default user
user:
name: "{{ default_user }}"
state: absent
remove: yes

32
roles/debian-base/tasks/fail2ban.yml Normal file
View File

@ -0,0 +1,32 @@
- name: Install fail2ban and rsyslog packages
apt:
name: "{{ fail2ban_base_packages }}"
state: present
- name: Configure sshd_config settings
copy:
dest: /etc/ssh/sshd_config.d/vn-fail2ban.conf
content: |
# Do not edit this file! Ansible will overwrite it.
SyslogFacility AUTH
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart sshd
- name: Configure fail2ban service
template:
src: jail.local
dest: /etc/fail2ban/jail.local
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart fail2ban
register: jail
- name: Ensure file for auth sshd custom log exists
file:
path: /var/log/auth.log
state: touch
owner: root
group: adm
mode: u=rw,g=r,o=
when: jail.changed

4
roles/debian-base/tasks/install.yml Normal file
View File

@ -0,0 +1,4 @@
- name: Install base packages
apt:
name: "{{ base_packages }}"
state: present

6
roles/debian-base/tasks/locale.yml Normal file
View File

@ -0,0 +1,6 @@
- name: make sure locales in variable are generated
locale_gen:
name: "{{ item }}"
state: present
with_items: "{{ locales_present }}"
notify: generate locales

32
roles/debian-base/tasks/main.yml Normal file
View File

@ -0,0 +1,32 @@
- import_tasks: witness.yml
tags: witness
- import_tasks: resolv.yml
tags: resolv
- import_tasks: timesync.yml
tags: timesync
- import_tasks: ssh.yml
tags: ssh
- import_tasks: defuser.yml
tags: defuser
- import_tasks: install.yml
tags: install
- import_tasks: locale.yml
tags: locale
- import_tasks: tzdata.yml
tags: tzdata
- import_tasks: relayhost.yml
tags: relayhost
- import_tasks: motd.yml
tags: motd
- import_tasks: profile.yml
tags: profile
- import_tasks: vim.yml
tags: vim
- import_tasks: nrpe.yml
tags: nrpe
- import_tasks: fail2ban.yml
tags: fail2ban
- import_tasks: bacula.yml
tags: bacula
- import_tasks: vn-repo.yml
tags: vn-repo

7
roles/debian-base/tasks/motd.yml Normal file
View File

@ -0,0 +1,7 @@
- name: Copy MOTD configuration file
copy:
src: motd
dest: /etc/update-motd.d/90-vn
mode: u=rwx,g=rx,o=rx
owner: root
group: root

22
roles/debian-base/tasks/nrpe.yml Normal file
View File

@ -0,0 +1,22 @@
- name: Install NRPE packages
apt:
name: "{{ nagios_packages }}"
state: present
install_recommends: no
- name: Set NRPE generic configuration
template:
src: nrpe.cfg
dest: /etc/nagios/nrpe.d/90-vn.cfg
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart-nrpe
- name: Create NRPE local configuration file
file:
path: /etc/nagios/nrpe.d/99-local.cfg
state: touch
owner: nagios
group: nagios
mode: u=rw,g=r,o=
modification_time: preserve
access_time: preserve

7
roles/debian-base/tasks/profile.yml Normal file
View File

@ -0,0 +1,7 @@
- name: Copy profile configuration file
copy:
src: profile.sh
dest: /etc/profile.d/vn.sh
mode: u=rw,g=r,o=r
owner: root
group: root

29
roles/debian-base/tasks/relayhost.yml Normal file
View File

@ -0,0 +1,29 @@
- name: Install exim packages
apt:
name: exim4
state: present
- name: Prepare exim configuration
blockinfile:
path: /etc/exim4/update-exim4.conf.conf
marker_begin: '--- BEGIN VN ---'
marker_end: '--- END VN ---'
marker: "# {mark}"
block: |
dc_eximconfig_configtype='satellite'
dc_other_hostnames='{{ ansible_fqdn }}'
dc_local_interfaces='127.0.0.1'
dc_readhost='{{ ansible_fqdn }}'
dc_smarthost='{{ smtp_server }}'
dc_hide_mailname='true'
state: present
create: yes
mode: u=rw,g=r,o=r
notify: update exim configuration
register: exim_config
- name: Force execution of handlers immediately
meta: flush_handlers
- name: Sending mail to verify relay host configuration works
shell: >
sleep 2; echo "If you see this message, relayhost on {{ ansible_fqdn }} has been configured correctly." \
| mailx -s "Relayhost test for {{ ansible_fqdn }}" "{{ sysadmin_mail }}"
when: exim_config.changed

22
roles/debian-base/tasks/resolv.yml Normal file
View File

@ -0,0 +1,22 @@
- name: Check if DNS is already configured
stat:
path: /etc/resolv.conf
register: resolv_conf
- name: Read /etc/resolv.conf
slurp:
path: /etc/resolv.conf
register: resolv_conf_content
when: resolv_conf.stat.exists
- name: Check if DNS servers are already present
set_fact:
dns_configured: "{{ resolv_conf_content['content'] | b64decode | regex_search('^nameserver') is not none }}"
when: resolv_conf.stat.exists
- name: Apply resolv.conf template only if DNS is not configured
template:
src: templates/resolv.conf
dest: /etc/resolv.conf
owner: root
group: root
mode: u=rw,g=r,o=r
backup: true
when: not resolv_conf.stat.exists or not dns_configured

22
roles/debian-base/tasks/ssh.yml Normal file
View File

@ -0,0 +1,22 @@
- name: Generate SSH key pairs
openssh_keypair:
path: "/etc/ssh/ssh_host_{{ item.type }}_key"
type: "{{ item.type }}"
force: yes
when: vn_witness
loop:
- { type: 'rsa' }
- { type: 'ecdsa' }
- { type: 'ed25519' }
notify: restart sshd
- name: Configure sshd_config settings
copy:
dest: /etc/ssh/sshd_config.d/vn-listenipv4.conf
content: |
# Do not edit this file! Ansible will overwrite it.
ListenAddress 0.0.0.0
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart sshd

23
roles/debian-base/tasks/timesync.yml Normal file
View File

@ -0,0 +1,23 @@
- name: Ensure directory for timesyncd custom configuration exists
file:
path: /etc/systemd/timesyncd.conf.d/
state: directory
owner: root
group: root
mode: u=rwx,g=rx,o=rx
- name: Configure NTP settings in /etc/systemd/timesyncd.conf.d/vn-ntp.conf
copy:
dest: /etc/systemd/timesyncd.conf.d/vn-ntp.conf
content: |
[Time]
NTP={{ time_server }}
FallbackNTP={{ time_server_spain }}
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart systemd-timesyncd
- name: Ensure systemd-timesyncd service is enabled and started
service:
name: systemd-timesyncd
enabled: yes
state: started

11
roles/debian-base/tasks/tzdata.yml Normal file
View File

@ -0,0 +1,11 @@
- name: Configure debconf for tzdata
debconf:
name: tzdata
question: "{{ item.question }}"
value: "{{ item.value }}"
vtype: "string"
loop:
- { question: "tzdata/Areas", value: "Europe" }
- { question: "tzdata/Zones/Europe", value: "Madrid" }
- { question: "tzdata/Zones/Etc", value: "UTC" }
notify: reconfigure tzdata

11
roles/debian-base/tasks/vim.yml Normal file
View File

@ -0,0 +1,11 @@
- name: Install vim packages
apt:
name: vim
state: present
- name: Copy vim configuration file
copy:
src: vimrc.local
dest: /etc/vim/
mode: u=rw,g=r,o=r
owner: root
group: root

3
roles/debian-base/tasks/vn-repo.yml Normal file
View File

@ -0,0 +1,3 @@
- name: Install package
apt:
deb: "{{ vn_host_url }}"

12
roles/debian-base/tasks/witness.yml Normal file
View File

@ -0,0 +1,12 @@
- name: Check if witness have been generated
stat:
path: /etc/vn.witness
register: keys_generated_marker
- name: Generate variable if not exists
set_fact:
vn_witness: "{{ not keys_generated_marker.stat.exists }}"
- name: Create marker file to indicate vn happends
file:
path: /etc/vn.witness
state: touch
when: vn_witness

24
roles/debian-base/templates/bacula-fd.conf Normal file
View File

@ -0,0 +1,24 @@
Director {
Name = bacula-dir
Password = "{{ bacula_passwords.fdpasswd }}"
}
Director {
Name = bacula-mon
Password = "{{ bacula_passwords.fdmpasswd }}"
Monitor = yes
}
FileDaemon {
Name = bacula-fd
WorkingDirectory = /var/lib/bacula
Pid Directory = /run/bacula
Maximum Concurrent Jobs = 20
Plugin Directory = /usr/lib/bacula
PKI Signatures = Yes
PKI Encryption = Yes
PKI Keypair = "/etc/bacula/fd-cert.pem"
PKI Master Key = "/etc/bacula/master-cert.pem"
}
Messages {
Name = Standard
director = bacula-dir = all, !skipped, !restored
}

22
roles/debian-base/templates/jail.local Normal file
View File

@ -0,0 +1,22 @@
# Do not edit this file! Ansible will overwrite it.
[DEFAULT]
ignoreip = {{ fail2ban.ignore }}
bantime = {{ fail2ban.bantime }}
findtime = {{ fail2ban.bantime }}
maxretry = {{ fail2ban.maxretry }}
destemail = {{ fail2ban.email }}
sender = root@{{ ansible_fqdn }}
banaction = nftables-multiport
action = %(action_)s
#+++++++++++++++ Jails
[sshd]
ignoreip = 127.0.0.1/8
enabled = true
port = 0:65535
filter = sshd
logpath = {{ fail2ban.logpath }}
action = %(action_mwl)s

13
roles/debian-base/templates/nrpe.cfg Normal file
View File

@ -0,0 +1,13 @@
allowed_hosts={{ nagios_server }}
server_address={{ ansible_default_ipv4.address }}
command[check_disk_root]=/usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /
command[check_disk_var]=/usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /var
command[check_disk_usr]=/usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /usr
command[check_disk_home]=/usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /home
command[check_disk_tmp]=/usr/lib/nagios/plugins/check_disk -w 10% -c 5% -p /tmp
command[check_dummy]=/usr/lib/nagios/plugins/check_dummy 0
command[check_swap]=/usr/lib/nagios/plugins/check_swap -w 40% -c 20% -n OK
command[check_load]=/usr/lib/nagios/plugins/check_load -r -w 3.5,3.25,3 -c 4.5,4.25,4
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 400 -c 500
command[check_memory]=/usr/lib/nagios/plugins/check_memory --available -m -w 20%: -c 10%:

7
roles/debian-base/templates/resolv.conf Normal file
View File

@ -0,0 +1,7 @@
domain {{ host_domain }}
search {{ host_domain }}
{% if resolvers is defined %}
{% for resolver in resolvers %}
nameserver {{resolver}}
{% endfor %}
{% endif %}

4
roles/debian-guest/handlers/main.yml Normal file
View File

@ -0,0 +1,4 @@
- name: restart-nslcd
service:
name: nslcd
state: restarted

25
roles/debian-guest/tasks/auth.yml Normal file
View File

@ -0,0 +1,25 @@
- name: Install packages
apt:
name: nslcd
state: present
- name: Configure NSLCD
template:
src: nslcd.conf
dest: /etc/nslcd.conf
owner: root
group: nslcd
mode: '0640'
notify:
- restart-nslcd
register: nslcd
- name: Configure nsswitch to use NSLCD
lineinfile:
dest: /etc/nsswitch.conf
regexp: "{{item.regexp}}"
line: "{{item.line}}"
state: present
with_items:
- regexp: "^passwd:"
line: "passwd: files systemd ldap"
- regexp: "^group:"
line: "group: files systemd ldap"

4
roles/debian-guest/tasks/main.yml Normal file
View File

@ -0,0 +1,4 @@
- import_tasks: auth.yml
tags: auth
- import_tasks: sudoers.yml
tags: sudoers

11
roles/debian-guest/tasks/sudoers.yml Normal file
View File

@ -0,0 +1,11 @@
- name: Install sudo package
apt:
name: sudo
state: present
- name: Add sysadmin to sudoers
template:
src: sudoers
dest: /etc/sudoers.d/vn
mode: u=rw,g=r,o=
owner: root
group: root

16
roles/debian-guest/templates/nslcd.conf Normal file
View File

@ -0,0 +1,16 @@
# See nslcd.conf(5) for details.
uid nslcd
gid nslcd
uri {{ ldap_uri }}
idle_timelimit 60
base {{ ldap_base }}
binddn cn=nss,ou=admins,{{ ldap_base }}
bindpw {{ lookup(passbolt, 'nslcd', folder_parent_id=passbolt_folder).password }}
pagesize 500
filter group (&(objectClass=posixGroup)(cn={{ sysadmin_group }}))
filter passwd (&(objectClass=posixAccount)(memberOf=cn={{ sysadmin_group }},ou=dnGroups,{{ ldap_base }}))
pam_authz_search (&(objectClass=posixGroup)(cn={{ sysadmin_group }})(memberuid=$username))

1
roles/debian-guest/templates/sudoers Normal file
View File

@ -0,0 +1 @@
%{{ sysadmin_group }} ALL=(ALL) NOPASSWD: ALL

4
roles/debian-host/files/sysctl/30-basic.conf Normal file
View File

@ -0,0 +1,4 @@
vm.swappiness=10
vm.dirty_ratio=30
vm.dirty_background_ratio=5
net.core.somaxconn=65536

7
roles/debian-host/files/sysctl/40-network.conf Normal file
View File

@ -0,0 +1,7 @@
net.core.rmem_max=134217728
net.core.wmem_max=134217728
net.core.netdev_max_backlog=250000
net.ipv4.tcp_rmem=4096 87380 67108864
net.ipv4.tcp_wmem=4096 65536 67108864
net.ipv4.tcp_congestion_control=htcp
net.ipv4.tcp_mtu_probing=1

3
roles/debian-host/files/sysctl/42-noipv6.conf Normal file
View File

@ -0,0 +1,3 @@
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6=1

4
roles/debian-host/handlers/main.yml Normal file
View File

@ -0,0 +1,4 @@
- name: restart-sysctl
systemd:
name: systemd-sysctl
state: restarted

12
roles/debian-host/tasks/apparmor.yml Normal file
View File

@ -0,0 +1,12 @@
- name: Stop AppArmor
systemd:
name: apparmor
state: stopped
- name: Disable AppArmor service
systemd:
name: apparmor
enabled: no
- name: Mask AppArmor service
systemd:
name: apparmor
masked: yes

9
roles/debian-host/tasks/hostname.yml Normal file
View File

@ -0,0 +1,9 @@
- name: Set the hostname
hostname:
name: "{{ inventory_hostname_short }}"
use: debian
- name: Populating hosts file with hostname
lineinfile:
path: /etc/hosts
regexp: '^127\.0\.1\.1'
line: '127.0.1.1 {{ hostname_fqdn }} {{ inventory_hostname_short }}'

6
roles/debian-host/tasks/main.yml Normal file
View File

@ -0,0 +1,6 @@
- import_tasks: hostname.yml
tags: hostname
- import_tasks: sysctl.yml
tags: sysctl
- import_tasks: apparmor.yml
tags: apparmor

8
roles/debian-host/tasks/sysctl.yml Normal file
View File

@ -0,0 +1,8 @@
- name: Set systctl custom vn configuration
copy:
src: sysctl/
dest: /etc/sysctl.d/
owner: root
group: root
mode: u=rw,g=r,o=r
notify: restart-sysctl

1
roles/debian-once/defaults/main.yaml Normal file
View File

@ -0,0 +1 @@
root_password: Pa$$w0rd

2
roles/debian-once/tasks/main.yml Normal file
View File

@ -0,0 +1,2 @@
- import_tasks: root.yml
tags: root

26
roles/debian-once/tasks/root.yml Normal file
View File

@ -0,0 +1,26 @@
- name: Generate a random root password
set_fact:
root_password: "{{ lookup('password', '/dev/null length=18 chars=ascii_letters,digits') }}"
- name: Save root password into Passbolt
set_fact:
msg: >
{{
lookup(passbolt, inventory_hostname_short,
username='root',
password=root_password,
uri='ssh://'+hostname_fqdn
)
}}
environment:
PASSBOLT_CREATE_NEW_RESOURCE: true
- name: Save the root password to file
copy:
content: "{{ root_password }}\n"
dest: /root/root_password.txt
owner: root
group: root
mode: '0600'
- name: Change root password
user:
name: root
password: "{{ root_password | password_hash('sha512') }}"

1
roles/debian-qemu/defaults/main.yml Normal file
View File

@ -0,0 +1 @@
homes_path: /mnt/homes

1
roles/debian-qemu/files/80-hotplug-cpu-mem.rules Normal file
View File

@ -0,0 +1 @@
SUBSYSTEM=="cpu", ACTION=="add", TEST=="online", ATTR{online}=="0", ATTR{online}="1"

1
roles/debian-qemu/files/hotplug.cfg Normal file
View File

@ -0,0 +1 @@
GRUB_CMDLINE_LINUX_DEFAULT="quiet memhp_default_state=online security=none"

8
roles/debian-qemu/handlers/main.yml Normal file
View File

@ -0,0 +1,8 @@
- name: restart-nslcd
service:
name: nslcd
state: restarted
- name: restart-autofs
service:
name: autofs
state: restarted

4
roles/debian-qemu/tasks/agent.yml Normal file
View File

@ -0,0 +1,4 @@
- name: Install QEMU guest agent
apt:
name: qemu-guest-agent
state: present

38
roles/debian-qemu/tasks/autofs.yml Normal file
View File

@ -0,0 +1,38 @@
- name: Install autofs packages
apt:
name: "{{ item }}"
state: present
with_items:
- nfs-common
- autofs
- libnfs-utils
- autofs-ldap
- name: Create homes directory
file:
path: "{{ homes_path }}"
state: directory
mode: '0755'
- name: Configure nsswitch for autofs
lineinfile:
path: /etc/nsswitch.conf
line: "automount: files"
notify: restart-nslcd
- name: Add file homes.autofs configured to autofs
template:
src: homes.autofs
dest: /etc/auto.master.d/homes.autofs
owner: root
group: root
mode: '0644'
- name: Add file /etc/auto.homes configured to the systemd
template:
src: auto.homes
dest: /etc/auto.homes
owner: root
group: root
mode: '0644'
notify: restart-autofs
- name: Service autofs service
service:
name: autofs
enabled: yes

16
roles/debian-qemu/tasks/hotplug.yml Normal file
View File

@ -0,0 +1,16 @@
- name: Configure udev hotplug rules
copy:
src: 80-hotplug-cpu-mem.rules
dest: /usr/lib/udev/rules.d/
mode: u=rw,g=r,o=r
owner: root
group: root
- name: Configure GRUB for hotplug
copy:
src: hotplug.cfg
dest: /etc/default/grub.d/
mode: u=rw,g=r,o=r
owner: root
group: root
- name: Generate GRUB configuration
command: update-grub

6
roles/debian-qemu/tasks/main.yml Normal file
View File

@ -0,0 +1,6 @@
- import_tasks: agent.yml
tags: agent
- import_tasks: hotplug.yml
tags: hotplug
- import_tasks: autofs.yml
tags: autofs

1
roles/debian-qemu/templates/auto.homes Normal file
View File

@ -0,0 +1 @@
* -fstype=nfs4,rw {{ homes_server }}:{{ homes_path }}/&

1
roles/debian-qemu/templates/homes.autofs Normal file
View File

@ -0,0 +1 @@
{{ homes_path }} /etc/auto.homes --timeout=30

17
roles/debian-upgrade/tasks/main.yaml Normal file
View File

@ -0,0 +1,17 @@
- name: Update APT package index
ansible.builtin.apt:
update_cache: true
force_apt_get: true
- name: Update all packages to their latest version
ansible.builtin.apt:
name: "*"
state: latest
force_apt_get: true
- name: Upgrade the OS (apt-get full-upgrade)
ansible.builtin.apt:
upgrade: full
force_apt_get: true
- name: Autoremove unused packages
ansible.builtin.apt:
autoremove: true
force_apt_get: true

60
roles/freeradius/files/ldap Normal file
View File

@ -0,0 +1,60 @@
ldap {
server = 'ldap.verdnatura.es'
identity = 'cn=admin,dc=verdnatura,dc=es'
password = blablabla
base_dn = 'dc=verdnatura,dc=es'
user_dn = "LDAP-UserDn"
update {
control:Password-With-Header += 'userPassword'
control:NT-Password := 'sambaNTPassword'
}
user {
base_dn = "ou=users,${..base_dn}"
filter = "(uid=%{%{Stripped-User-Name}:-%{User-Name}})"
}
group {
base_dn = "ou=groups,${..base_dn}"
name_attribute = 'cn'
membership_attribute = 'memberUid'
membership_filter = "(memberUid=%{%{Stripped-User-Name}:-%{User-Name}})"
filter = '(objectClass=posixGroup)'
cacheable_name = yes
}
accounting {
reference = "%{tolower:type.%{Acct-Status-Type}}"
type {
start {
update {description := "Online at %S"}
}
interim-update {
update {description := "Last seen at %S"}
}
stop {
update {description := "Offline at %S"}
}
}
}
post-auth {
update {description := "Authenticated at %S"}
}
options {
rebind = yes
res_timeout = 10
srv_timelimit = 3
net_timeout = 1
idle = 60
probes = 3
interval = 3
ldap_debug = 0x0028
}
pool {
start = ${thread[pool].start_servers}
min = ${thread[pool].min_spare_servers}
max = ${thread[pool].max_servers}
spare = ${thread[pool].max_spare_servers}
uses = 0
retry_delay = 30
lifetime = 0
idle_timeout = 60
}
}

5
roles/freeradius/handlers/main.yaml Normal file
View File

@ -0,0 +1,5 @@
- name: restart-freeradius
service:
name: freeradius
state: restarted
enabled: yes

97
roles/freeradius/tasks/main.yml Normal file
View File

@ -0,0 +1,97 @@
- name: Install packagesfor freeradiusotp
apt:
name: "{{ item }}"
state: present
with_items:
- freeradius
- freeradius-ldap
- libpam-google-authenticator
- python3-qrcode
- zip
- mutt
- name: Create a symbolic link
ansible.builtin.file:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: freerad
group: freerad
state: link
force: yes
loop:
- { src: '{{ freeradius_mods_available_folder }}ldap', dest: '{{ freeradius_mods_enabled_folder }}ldap' }
- { src: '{{ freeradius_mods_available_folder }}pam', dest: '{{ freeradius_mods_enabled_folder }}pam' }
- name: config default file
ansible.builtin.template:
src: default.j2
dest: "{{ freeradius_default_config }}"
owner: freerad
group: freerad
mode: '0640'
backup: yes
- name: Copy LDAP file
copy:
src: ldap
dest: "{{ freeradius_mod_ldap }}"
owner: freerad
group: freerad
mode: '0640'
backup: yes
- name: Add password to LDAP file
lineinfile:
dest: "{{ freeradius_mod_ldap }}"
regexp: "{{item.regexp}}"
line: "{{item.line}}"
state: present
with_items:
- regexp: "^ password ="
line: " password = {{ radius_ldap_password }}"
- name: Config dictionary file
ansible.builtin.template:
src: dictionary.j2
dest: "{{ freeradius_dictionary_config }}"
owner: freerad
group: freerad
mode: '0640'
backup: yes
- name: Config clients.conf file
ansible.builtin.template:
src: clients.j2
dest: "{{ freeradius_clients_config }}"
owner: freerad
group: freerad
mode: '0640'
backup: yes
- name: Config filter file
ansible.builtin.template:
src: filter.j2
dest: "{{ freeradius_filter_config }}"
owner: freerad
group: freerad
mode: '0640'
backup: yes
- name: Config radius.conf file
ansible.builtin.template:
src: radiusd.j2
dest: "{{ freeradius_base_config }}"
owner: freerad
group: freerad
mode: '0640'
backup: yes
notify: restart freeradius
- name: Config pam radiusd file
ansible.builtin.template:
src: radiusdpam.j2
dest: "{{ freeradius_pam_config }}"
owner: root
group: root
mode: '0644'
- name: Config freeradius systemd service file
ansible.builtin.template:
src: freeradiusservice.j2
dest: "{{ freeradius_service_config }}"
owner: root
group: root
mode: '0644'
- name: Just force systemd to reread configs (2.4 and above)
ansible.builtin.systemd_service:
daemon_reload: true

Some files were not shown because too many files have changed in this diff Show More