Wednesday, January 15, 2020

Ansible Download or Fetch remote logs from nodes

I really like the Ansible connection management and inventory system (I use a dynamic inventory generated from my Terraform state files).  Playbook organization gives me fits sometimes, I tend to cut things up into small plays with just a few very closely related tasks and then roll those up into larger playbooks using imports so they cover an entire systems.  Then I'll have a good collection of fairly complex but decoupled playbooks that are mostly just imports of smaller plays.  I'll then import the fewer larger playsbooks into site.yaml so it reads like an overview.

I've tried roles, and tags - but they never seem to be satisfying.  IIRC, when I looked at the timeline the import statement was added to ansible *after* roles were introduced - so I assume that means my requirements or conceptualizations that lead me to want to develop, organize and compose things this way is not unprecedented.

I wish I was better at using modules adhoc from the command line.  I'm pretty good at grabbing:

ansible nodes -m shell -a "some bash"

... but I've also written more than a few plays that I only use once and throw away.  For example something like fetch_logs.yaml

- hosts: nodes
  become: true
  tasks:
    - name: fetch logs
      fetch:
        src=/var/log/swift/all.log
        dest=.scratch/logs/
        validate_checksum=false

And while it's entirely reasonable to have a single task play checked into version control in case you need it again later - I'm also finding it getting a little easier to start to convert these things to adhoc commands.

To use ansible to download remote logs from remote nodes using the fetch module adhoc on the commnad line, try something like this:

ansible nodes -b -m fetch -a "src=/var/log/swift/all.log dest=.scratch/logs/ validate_checksum=false"

In this command "nodes" is just an ansible group, to specify hosts, it could just as well be "all".  The "-b" option is for "--become", "-m" specifics the module and the space separated list of arguments is given with "-a". Hopefully I'll find this next time I forget.