ONLY FOR SELF STUDY, NO COMMERCIAL USAGE!!!
Chapter 5. Deploying Files to Managed Hosts
Modifying and Copying Files to Hosts
Describing File Modules
Most of the commonly used modules related to Linux file management are provided with ansible-core in the ansible.builtin collection. They perform tasks such as creating, copying, editing, and modifying permissions and other attributes of files. The following table provides a list of frequently used file management modules:
Table 5.1. Commonly Used File Modules in ansible.builtin
| Module name | Module description |
|---|---|
blockinfile | Insert, update, or remove a block of multiline text surrounded by customizable marker lines. |
copy | Copy a file from the local or remote machine to a location on a managed host. Similar to the file module, the copy module can also set file attributes, including SELinux context. |
fetch | This module works like the copy module, but in reverse. This module is used for fetching files from remote machines to the control node and storing them in a file tree, organized by host name. |
file | Set attributes such as permissions, ownership, SELinux contexts, and time stamps of regular files, symlinks, hard links, and directories. This module can also create or remove regular files, symlinks, hard links, and directories. A number of other file-related modules support the same options to set attributes as the file module, including the copy module. |
lineinfile | Ensure that a particular line is in a file, or replace an existing line using a back-reference regular expression. This module is primarily useful when you want to change a single line in a file. |
stat | Retrieve status information for a file, similar to the Linux stat command. |
In addition, the ansible.posix collection, which is included in the default automation execution environment, provides some additional modules that are useful for file management:
Table 5.2. Commonly Used File Modules in ansible.posix
| Module name | Module description |
|---|---|
patch | Apply patches to files by using GNU patch. |
synchronize | A wrapper around the rsync command to simplify common tasks. The synchronize module is not intended to provide access to the full power of the rsync command, but does make the most common invocations easier to implement. You might still need to call the rsync command directly via the command module depending on your use case. |
Automation Examples with Files Modules
The following examples show ways that you can use these modules to automate common file management tasks.
Ensuring a File Exists on Managed Hosts
Use the ansible.builtin.file module to touch a file on managed hosts. This works like the touch command, creating an empty file if it does not exist, and updating its modification time if it does exist. In this example, in addition to touching the file, Ansible ensures that the owning user, group, and permissions of the file are set to specific values.
- name: Touch a file and set permissionsansible.builtin.file:path: /path/to/fileowner: user1group: group1mode: 0640state: touch
Example outcome:
[user@host ~]$ ls -l file
-rw-r-----. user1 group1 0 Nov 25 08:00 file
Modifying File Attributes
You can use the ansible.builtin.file module to ensure that a new or existing file has the correct permissions or SELinux type as well.
For example, the following file has retained the default SELinux context relative to a user’s home directory, which is not the desired context.
[user@host ~]$ ls -Z samba_file
-rw-r--r--. owner group unconfined_u:object_r:user_home_t:s0 samba_file
The following task ensures that the SELinux context type attribute of the samba_file file is the desired samba_share_t type. This behavior is similar to the Linux chcon command.
- name: SELinux type is set to samba_share_tansible.builtin.file:path: /path/to/samba_filesetype: samba_share_t
Example outcome:
[user@host ~]$ ls -Z samba_file
-rw-r--r--. owner group unconfined_u:object_r:samba_share_t:s0 samba_file
File attribute parameters are available in multiple file management modules. Use the ansible-navigator doc command for additional information, providing the ansible.builtin.file or ansible.builtin.copy module as an argument.
Note
To set SELinux file contexts persistently in the policy, some options include:
- If you know how to use Ansible roles, you can use the supported
redhat.rhel_system_roles.selinuxrole. That is covered in Chapter 7 of the Red Hat Enterprise Linux Automation with Ansible (RH294) training course. - You can use the module
community.general.sefcontextin the community-supportedcommunity.generalAnsible Content Collection.
Copying and Editing Files on Managed Hosts
In this example, the ansible.builtin.copy module is used to copy a file located in the Ansible working directory on the control node to selected managed hosts.
By default, this module assumes that force: true is set. That forces the module to overwrite the remote file if it exists but has different contents to the file being copied. If force: false is set, then it only copies the file to the managed host if it does not already exist.
- name: Copy a file to managed hostsansible.builtin.copy:src: filedest: /path/to/file
To retrieve files from managed hosts use the ansible.builtin.fetch module. This could be used to retrieve a file such as an SSH public key from a reference system before distributing it to other managed hosts.
- name: Retrieve SSH key from reference hostansible.builtin.fetch:src: "/home/{{ user }}/.ssh/id_rsa.pubdest: "files/keys/{{ user }}.pub"
To ensure a specific single line of text exists in an existing file, use the lineinfile module:
- name: Add a line of text to a fileansible.builtin.lineinfile:path: /path/to/fileline: 'Add this line to the file'state: present
To add a block of text to an existing file, use the ansible.builtin.blockinfile module:
- name: Add additional lines to a fileansible.builtin.blockinfile:path: /path/to/fileblock: |First line in the additional block of textSecond line in the additional block of textstate: present
Note
When using the ansible.builtin.blockinfile module, commented block markers are inserted at the beginning and end of the block to ensure idempotency.
# BEGIN ANSIBLE MANAGED BLOCK
First line in the additional block of text
Second line in the additional block of text
# END ANSIBLE MANAGED BLOCK
You can use the marker parameter to the module to help ensure that the right comment character or text is being used for the file in question.
Removing a File from Managed Hosts
A basic example to remove a file from managed hosts is to use the ansible.builtin.file module with the state: absent parameter. The state parameter is optional to many modules. You should always make your intentions clear whether you want state: present or state: absent for several reasons. Some modules support other options as well. It is possible that the default could change at some point, but perhaps most importantly, it makes it easier to understand the state the system should be in based on your task.
- name: Make sure a file does not exist on managed hostsansible.builtin.file:dest: /path/to/filestate: absent
Retrieving the Status of a File on Managed Hosts
The ansible.builtin.stat module retrieves facts for a file, similar to the Linux stat command. Parameters provide the functionality to retrieve file attributes, determine the checksum of a file, and more.
The ansible.builtin.stat module returns a dictionary of values containing the file status data, which allows you to refer to individual pieces of information using separate variables.
The following example registers the results of a ansible.builtin.stat module task and then prints the MD5 checksum of the file that it checked. (The more modern SHA256 algorithm is also available; MD5 is being used here for legibility.)
- name: Verify the checksum of a fileansible.builtin.stat:path: /path/to/filechecksum_algorithm: md5register: result- ansible.builtin.debugmsg: "The checksum of the file is {{ result.stat.checksum }}"
The outcome should be similar to the following:
TASK [Get md5 checksum of a file] *****************************************
ok: [hostname]TASK [debug] **************************************************************
ok: [hostname] => {"msg": "The checksum of the file is 5f76590425303022e933c43a7f2092a3"
}
Information about the values returned by the ansible.builtin.stat module are documented in ansible-navigator doc ansible.builtin.stat, or you can register a variable and display its contents to see what is available:
- name: Examine all stat output of /etc/passwdhosts: workstationtasks:- name: stat /etc/passwdansible.builtin.stat:path: /etc/passwdregister: results- name: Display stat resultsdebug:var: results
Synchronizing Files Between the Control Node and Managed Hosts
The ansible.posix.synchronize module is a wrapper around the rsync tool, which simplifies common file management tasks in your playbooks. The rsync tool must be installed on both the local and remote host. By default, when using the ansible.posix.synchronize module, the “local host” is the host that the ansible.posix.synchronize task originates on (usually the control node), and the “destination host” is the host that ansible.posix.synchronize connects to.
The following example synchronizes a file located in the Ansible working directory to the managed hosts:
- name: synchronize local file to remote filesansible.posix.synchronize:src: filedest: /path/to/file
You can use the ansible.posix.synchronize module and its many parameters in many different ways, including synchronizing directories. Run the ansible-navigator doc ansible.posix.synchronize command for additional parameters and playbook examples.
References
chmod(1), chown(1), rsync(1), stat(1) and touch(1) man pages
ansible-navigator doc command
Ansible documentation — Index of all Modules - ansible.builtin
Example
---
- name: File management testhosts: serverstasks:- name: Retrieve secure logsansible.builtin.fetch: # get files from remote hostsrc: /var/log/securedest: secure-backups- name: Create files dir and set SE linux contextansible.builtin.file:path: /home/devops/filesstate: directory # create the directory 'files'owner: devopsgroup: devopsmode: 0755# setype: samba_share_t # change the SE contextsetype: _default- name: adding message to textansible.builtin.lineinfile:path: /home/devops/files/users.txtline: "This line was addes by the lineinfile module"state: presentcreate: true # create the file if not existsowner: devopsgroup: devopsmode: 0664 - name: Copying files from here to remoteansible.builtin.copy:src: systemdest: /home/devops/files/mode: 0664owner: devopsgroup: devops- name: Adding message with blockinfileansible.builtin.blockinfile:path: /home/devops/files/users.txtblock: |this block of text consists of two lines.They have been added by the blockinfile module.output:
[student@workstation secure-backups]$ ssh devops@servera 'ls -Z'
unconfined_u:object_r:samba_share_t:s0 files[student@workstation ~]$ [student@workstation file-manage]$ tree -F secure-backups/
secure-backups/
├── servera.lab.example.com/
│ └── var/
│ └── log/
│ └── secure
└── serverb.lab.example.com/└── var/└── log/└── secure[student@workstation file-manage]$ ssh devops@servera 'cat files/users.txt'
This line was addes by the lineinfile module
# BEGIN ANSIBLE MANAGED BLOCK
this block of text consists of two lines.
They have been added by the blockinfile module.
# END ANSIBLE MANAGED BLOCK
Deploying Custom Files with Jinja2 Templates
Templating Files
The ansible.builtin Ansible Content Collection provides a number of modules that can be used to modify existing files. These include lineinfile and blockinfile, among others. However, they are not always easy to use effectively and correctly.
A much more powerful way to manage files is to template them. With this method, you can write a template configuration file that is automatically customized for the managed host when the file is deployed, using Ansible variables and facts. This can be easier to control and is less error-prone.
Introduction to Jinja2
Ansible uses the Jinja2 templating system for template files. Ansible also uses Jinja2 syntax to reference variables in playbooks, so you already know a little about how to use it.
Using Delimiters
Variables and logic expressions are placed between tags, or delimiters. When a Jinja2 template is evaluated, the expression {{ *EXPR* }} is replaced with the results of that expression or variable. Jinja2 templates can also use {% *EXPR* %} for special control structures or logic that loops over Jinja2 code or perform tests. You can use the {# *COMMENT* #} syntax to enclose comments that should not appear in the final file.
In the following example of a Jinja2 template file, the first line includes a comment that is not included in the final file. The variable references in the second line are replaced with the values of the system facts being referenced.
{# /etc/hosts line #}
{{ ansible_facts['default_ipv4']['address'] }} {{ ansible_facts['hostname'] }}
Building a Jinja2 Template
A Jinja2 template is composed of multiple elements: data, variables, and expressions. Those variables and expressions are replaced with their values when the Jinja2 template is rendered. The variables used in the template can be specified in the vars section of the playbook. It is possible to use the managed hosts’ facts as variables in a template.
Template files are most commonly kept in the templates directory of the project for your playbook, and typically are assigned a .j2 file extension to make it clear that they are Jinja2 template files.
Note:
A file containing a Jinja2 template does not need to have any specific file extension (for example, .j2). However, providing such a file extension might make it easier for you to remember that it is a template file.
The following example shows how to create a template for /etc/ssh/sshd_config with variables and facts retrieved by Ansible from managed hosts. When the template is deployed by a play, any facts are replaced by their values for the managed host being configured.
# {{ ansible_managed }}
# DO NOT MAKE LOCAL MODIFICATIONS TO THIS FILE BECAUSE THEY WILL BE LOSTPort {{ ssh_port }}
ListenAddress {{ ansible_facts['default_ipv4']['address'] }}HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_keySyslogFacility AUTHPRIVPermitRootLogin {{ root_allowed }}
AllowGroups {{ groups_allowed }}AuthorizedKeysFile /etc/.rht_authorized_keys .ssh/authorized_keysPasswordAuthentication {{ passwords_allowed }}ChallengeResponseAuthentication noGSSAPIAuthentication yes
GSSAPICleanupCredentials noUsePAM yesX11Forwarding yes
UsePrivilegeSeparation sandboxAcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERSSubsystem sftp /usr/libexec/openssh/sftp-server
Deploying Jinja2 Templates
Jinja2 templates are a powerful tool that you can use to customize configuration files to be deployed on managed hosts. When the Jinja2 template for a configuration file has been created, it can be deployed to managed hosts by using the ansible.builtin.template module, which supports the transfer of a local file on the control node to the managed hosts.
To use the ansible.builtin.template module, use the following syntax. The value associated with the src key specifies the source Jinja2 template, and the value associated with the dest key specifies the file to be created on the destination hosts.
tasks:- name: template renderansible.builtin.template:src: /tmp/j2-template.j2dest: /tmp/dest-config-file.txt
Note:
The ansible.builtin.template module also allows you to specify the owner (the user that owns the file), group, permissions, and SELinux context of the deployed file, just like the ansible.builtin.file module. It can also take a validate option to run an arbitrary command (such as visudo -c) to check the syntax of a file for correctness before templating it into place.
For more details, see ansible-navigator doc ansible.builtin.template.
Managing Templated Files
To avoid having other system administrators modify files that are managed by Ansible, it is a good practice to include a comment at the top of the template to indicate that the file should not be manually edited.
One way to do this is to use the “Ansible managed” string set by the ansible_managed directive. This is not a normal variable but can be used as one in a template. You can set the value for ansible_managed in an ansible.cfg file:
ansible_managed = Ansible managed
To include the ansible_managed string inside a Jinja2 template, use the following syntax:
{{ ansible_managed }}
Control Structures
You can use Jinja2 control structures in template files to reduce repetitive typing, to enter entries for each host in a play dynamically, or conditionally insert text into a file.
Using Loops
Jinja2 uses the for statement to provide looping functionality. In the following example, the users variable has a list of values. The user variable is replaced with all the values in the users variable, one value per line.
{% for user in users %}{{ user }}
{% endfor %}
The following example template uses a for statement and a conditional to run through all the values in the users variable, replacing myuser with each value, unless the value is root.
{# for statement #}
{% for myuser in users if not myuser == "root" %}
User number {{ loop.index }} - {{ myuser }}
{% endfor %}
The loop.index variable expands to the index number that the loop is currently on. It has a value of 1 the first time the loop executes, and it increments by 1 through each iteration.
As another example, this template also uses a for statement. It assumes a myhosts variable that contains a list of hosts to be managed has been defined by the inventory being used. If you put the following for statement in a Jinja2 template, all hosts in the myhosts group from the inventory would be listed in the resulting file.
{% for myhost in groups['myhosts'] %}
{{ myhost }}
{% endfor %}
For a more practical example, you can use this example to generate an /etc/hosts file from host facts dynamically. Assume that you have the following playbook:
- name: /etc/hosts is up to datehosts: allgather_facts: truetasks:- name: Deploy /etc/hostsansible.builtin.template:src: templates/hosts.j2dest: /etc/hosts
The following three-line templates/hosts.j2 template constructs the file from all hosts in the group all. (The middle line is extremely long in the template due to the length of the variable names.) It iterates over each host in the group to get three facts for the /etc/hosts file.
{% for host in groups['all'] %}
{{ hostvars[host]['ansible_facts']['default_ipv4']['address'] }} {{ hostvars[host]['ansible_facts']['fqdn'] }} {{ hostvars[host]['ansible_facts']['hostname'] }}
{% endfor %}
Using Conditionals
Jinja2 uses the if statement to provide conditional control. This allows you to put a line in a deployed file if certain conditions are met.
In the following example, the value of the result variable is placed in the deployed file only if the value of the finished variable is True.
{% if finished %}
{{ result }}
{% endif %}
Variable Filters
Jinja2 provides filters which change the output format for template expressions, essentially converting the data in a variable to some other format in the file that results from the template.
For example, filters are available for languages such as YAML and JSON. The to_json filter formats the expression output using JSON, and the to_yaml filter formats the expression output using YAML.
{{ output | to_json }}
{{ output | to_yaml }}
Additional filters are available, such as the to_nice_json and to_nice_yaml filters, which format the expression output in either JSON or YAML human-readable format.
{{ output | to_nice_json }}
{{ output | to_nice_yaml }}
Both the from_json and from_yaml filters expect strings in either JSON or YAML format, respectively.
{{ output | from_json }}
{{ output | from_yaml }}
For more information you can also review “Using filters to manipulate data” in the Ansible User Guide.
References
ansible.builtin.template module - Template a file out to a target host — Ansible Documentation
Using filters to manipulate data — Ansible Documentation
Example
# Creating a motd.j2 template file
This is the system {{ ansible_facts['fqdn'] }}This is a {{ ansible_facts['distribution'] }} version {{ ansible_facts['distribution_version'] }}{% if ansible_facts['fqdn'] in groups['workstations'] %}
As a workstation user, you need to submit a ticket to receive help with any issues.
{% elif ansible_facts['fqdn'] in groups['webservers'] %}
Please report issues to: {{ system_owner }}.
{% endif %}
---
- name: Managing Files temaplate testhosts: allvars: - system_owner: peanutfish@abc.comtasks:- name: moving motd filesansible.builtin.template:src: motd.j2dest: /etc/motdowner: rootgroup: rootmode: 0644
output:
[student@workstation file-template]$ ssh devops@servera 'cat /etc/motd'
This is the system servera.lab.example.comThis is a RedHat version 9.0Please report issues to: peanutfish@abc.com.[student@workstation file-template]$ ssh devops@workstation 'cat /etc/motd'
This is the system workstation.lab.example.comThis is a RedHat version 9.0As a workstation user, you need to submit a ticket to receive help with any issues.
Chapter 5 TEST
In the /home/student/file-review directory, create a playbook file called motd.yml that contains a new play that runs on all hosts in the inventory. It must log in using the devops user on the remote host, and use become to enable privilege escalation for the whole play.
The play must have a task that uses the ansible.builtin.template module to deploy the motd.j2 Jinja2 template file to the file /etc/motd on the managed hosts. The resulting file must have the root user as its owning user and group, and its permissions must be 0644.
Add an additional task that uses the ansible.builtin.stat module to verify that /etc/motd exists on the managed hosts and registers its results in a variable. That task must be followed by a task that uses ansible.builtin.debug to display the information in that registered variable.
Add a task that uses the ansible.builtin.copy module to place files/issue into the /etc/ directory on the managed host, use the same ownership and permissions as /etc/motd.
Finally, add a task that uses the ansible.builtin.file module to ensure that /etc/issue.net is a symbolic link to /etc/issue on the managed host.
# Create a template file here, folder ./templates/motd.j2Welcome to the END of THE DAY!This system {{ansible_facts['fqdn']}} provided {{ ansible_facts['memtotal_mb'] }} mb memory and {{ ansible_facts['processor_count'] }} CPU core.
---
- name: Starting file review session hosts: allremote_user: devopsbecome: truebecome_user: roottasks:- name: Check system total memory and number of processorsansible.builtin.debug:msg: >The amount of system momory is {{ ansible_facts['memtotal_mb'] }}mb andthe number of processors is {{ ansible_facts['processor_count'] }}.# var: ansible_facts- name: Copying motd fileansible.builtin.template:src: templates/motd.j2dest: /etc/motdowner: rootgroup: rootmode: 0644- name: Check if motd existsansible.builtin.stat:path: /etc/motdregister: result- name: show debug infoansible.builtin.debug:var: result.stat- name: Copying filesansible.builtin.copy:src: files/issuedest: /etc/owner: rootgroup: rootmode: 0644- name: Verify the symbolic linkansible.builtin.file:src: /etc/issuedest: /etc/issue.netstate: linkowner: rootgroup: rootforce: true # Force the creation of the symlinks in two cases: the source file does not exist (but will appear later); the destination exists and is a file (so, we need to unlink the path file and create a symlink to the src file in place of it).
TO BE CONTINUED…