Ansible Download or Fetch remote logs from nodes

I really like the Ansible connection management and inventory system (I use a dynamic inventory generated from my Terraform state files).  Playbook organization gives me fits sometimes, I tend to cut things up into small plays with just a few very closely related tasks and then roll those up into larger playbooks using imports so they cover an entire systems.  Then I'll have a good collection of fairly complex but decoupled playbooks that are mostly just imports of smaller plays.  I'll then import the fewer larger playsbooks into site.yaml so it reads like an overview.

I've tried roles, and tags - but they never seem to be satisfying.  IIRC, when I looked at the timeline the import statement was added to ansible *after* roles were introduced - so I assume that means my requirements or conceptualizations that lead me to want to develop, organize and compose things this way is not unprecedented.

I wish I was better at using modules adhoc from the command line.  I'm pretty good at grabbing:

ansible nodes -m shell -a "some bash"

... but I've also written more than a few plays that I only use once and throw away.  For example something like fetch_logs.yaml

- hosts: nodes
  become: true
  tasks:
    - name: fetch logs
      fetch:
        src=/var/log/swift/all.log
        dest=.scratch/logs/
        validate_checksum=false

And while it's entirely reasonable to have a single task play checked into version control in case you need it again later - I'm also finding it getting a little easier to start to convert these things to adhoc commands.

To use ansible to download remote logs from remote nodes using the fetch module adhoc on the commnad line, try something like this:

ansible nodes -b -m fetch -a "src=/var/log/swift/all.log dest=.scratch/logs/ validate_checksum=false"

In this command "nodes" is just an ansible group, to specify hosts, it could just as well be "all".  The "-b" option is for "--become", "-m" specifics the module and the space separated list of arguments is given with "-a". Hopefully I'll find this next time I forget.

Today is a good day.

Walking home from my run this morning, the sun rising above the hills to my back, casting off the cool mist at the break of day in the Sunset district of San Francisco, I began to climb, tired, up a hill behind an older Asian man, maybe two dozen years my senior, moving slowly and smoking a sweet smelling joint.  As I overtook him he turned to me, startled, and said not quite too loudly "Good day!" Assuming he would know, I felt compelled to echo his declaration and assured him, yes indeed it was, a "Good day."

As I walked home I thought I 'ought let y'all know; I have it on reputable authority...

Today is a good day.

My mental model for software

OpenStack for Operators

A mission is a strongly felt aim, ambition, or calling.

The OpenStack mission:

to produce the ubiquitous Open Source Cloud Computing platform that
enables building interoperable public and private clouds regardless of
size, by being simple to implement and massively scalable while serving
the cloud users’ needs

This is what I most strongly identify with OpenStack.

A value is one's judgment of what is important in life.

OpenStack states some values; which I think are also good:

Open Source - License; because IANAL
Open Design - Summits are fun; if you have a spec
Open Development - Patches welcome
Open Community - Let's talk; we're nice!

... the four opens are pretty great - but it's not what I feel called to
do - it's not what I'm about.  I just think that in order to build out
that ubiquitous platform; where the open source cloud is found
everywhere ... no one company can do that?  So we better have a good
strategy for a community if we're gunna do this thing.

And that's what I come to when I think about "what is OpenStack" - I
don't think about the software that OpenStack produces.  To me OpenStack
is a phenomenon - because I literally do not understand the cause or
explanation for why this thing took off like it did?

Maybe I think there's just a lot of hackers out there that all feel like
I do - that building this thing is important (to society?) - and that
they can make a meaningful contribution.

OpenStack is a system to organize the people committed to the mission.
The result is OpenStack software - not all of which is good - some of it
might not even be useful - but if it was done with a passion for the
mission and according to the values - we should be happy to call it
OpenStack.

Or maybe it's the money.

Maybe it's the system to organize the profit-generating-organisms that
feed the hackers committed to the mission, the astroturf, the latch your
wagon^Wdriver-plugin onto it while it's hot, the cushy arm-chair
architect pontificate on the grand unified theory of exception logging,
the ego of the -2, the paycheck, the passport to hang with friends
around a whiteboard.

Meanwhile, in the real world, the market is happily shipping AWS so much
money Google and Microsoft can't mount a reasonable defense - Rackspace
is going to tap out and HP already gave up.

If we're still committed to the mission - we have got to focus on our
deployments.

Nothing tells you more about what's important than working with real
deployments.  The people operating OpenStack software are the best
resource we have to cut through the junk and deal with what matters.

If you're not at least partially engaged with the delivery, deployment
and operation of OpenStack software I strongly recommend you have a talk
with your manager - or seek a different employer.

Listen to the operators.  Ignore everything else.  Ignore internal
organizational boundaries.  Ignore your direct manager.  Ignore the TC.
Ignore me.  Listen to the operators.

If they continue to deploy OpenStack's software - then OpenStack matters
- and we have to keep after the mission - if they do not continue to
deploy OpenStack software - then none of this matters.

#NoBillNoBreak #StopTheStunt

See the problem is, the frequency of high profile mass shootings is too high - it's making people that would prefer not to carry weapons feel uncomfortable.  That and all the murder.

There is in no clear answer.  Smart legislation may help in some cases.  No Fly No Buy would not be perfect.  I think we should vote on it!?

Why is there always so much name calling on social media?  Wag of the finger to both sides!  Just stop doing that!?  Phew, solved that problem.

But, ugh, the FUD.

• No one thinks smart regulations for firearms will solve murder.
• I don't think a radical redefinition of the 2nd amendment is possible in my lifetime - Bernie didn't either.
• Any broad support to legislate confiscation of legally obtained firearms would meet riots on a national level well before congress had a chance to shoot it down - I don't really see how people can honestly claim they think the government is going to "take their guns"?

And yet, every time guns enter the national stage - more Americans arm themselves?

We need to stop talking past each other.  Is there really no common ground?  No compromise?

Congress, do your job.  Get in there and debate it out - I know it's hard - people say stupid shit - seemingly refuse to listen to common sense.  But you're a professional.  You can show us how it's done!  Negotiate.  Find the best solution that you can get some agreement on and pass it!  Demonstrate progress!  Don't think about what sound byte will play best in the next news cycle - think about what is best for your constituents - not what is most likely to get you re-elected.  Then come November, show us someone who is good at their job.

Openstack SDK attach cloud network

I don't think the current OpenStack SDK supports the nova api extension for virtual interfaces.

I'm not a nova guy.

I don't understand the difference between the OpenStack compute v2 os-interfaces API - and the Openstack compute os-virtual-interfacesv2 extension API.

But rackspace seems to have had to write an extension for the novaclient CLI to make it able to attach a new virtual interfaces for a cloud network to a running instance.

I was thinking I might have to write a new resource to support the api extension (probably just crib from the existing ServerInterface) - but it turned out there's already support in OpenStackSDK to boot instances with extra networks.

Here's what I'm using:

#!/usr/bin/env python
import os
import requests
import sys


# default instance and network attributes
IMAGE = '09de0a66-3156-48b4-90a5-1cf25a905207'
FLAVOR = 'general1-4'
REGION = 'DFW'
NETWORK = 'my-test'
CIDR = '192.168.5.0/24'

auth_url = os.environ.get(
    'OS_AUTHURL', 'https://identity.api.rackspacecloud.com/v2.0/')
# username and *password* must be set in the envrion
username = os.environ['OS_USERNAME']
password = os.environ['OS_PASSWORD']


# something about the rackspace cert makes requests mad
requests.packages.urllib3.disable_warnings()

from openstack import connection
from openstack.compute.v2._proxy import _server


# the openstacksdk doesn't support admin_pass
# https://bugs.launchpad.net/python-openstacksdk/+bug/1528949
class PatchedServerResource(_server.Server):

    def create_by_id(self, *args, **kwargs):
        resp = super(PatchedServerResource, self).create_by_id(*args, **kwargs)
        self._attrs['adminPass'] = resp.get('adminPass')
        return resp

_server.Server = PatchedServerResource

conn = connection.Connection(auth_url=auth_url,
                             username=username,
                             password=password)

conn.profile.set_region(conn.profile.ALL, REGION)


def _get_or_create_network(name):
    # this doesn't seem to work
    # network = conn.network.find_network(name)
    # https://feedback.rackspace.com/forums/298158-network/suggestions/11188446-500-error-response-in-cloud-network-api

    # so we just search by hand
    for n in conn.network.networks():
        if n.name == name:
            return n
    network = conn.network.create_network(name=name)
    conn.network.create_subnet(
        network_id=network.id, ip_version='4', cidr=CIDR)
    return network


def main():
    try:
        server_name = sys.argv[1]
    except IndexError:
        return 'USAGE: %s <server-name>' % sys.argv[0]

    image = conn.compute.find_image(IMAGE)
    flavor = conn.compute.find_flavor(FLAVOR)
    network = _get_or_create_network(NETWORK)
    network_ids = (
        network.id,
        # must explicitly include Public & Service networks
        '11111111-1111-1111-1111-111111111111',
        '00000000-0000-0000-0000-000000000000',
    )
    server = conn.compute.create_server(
        name=server_name, image=image, flavor=flavor,
        networks=[{'uuid': id_} for id_ in network_ids]
    )
    print 'pass', server['adminPass']
    server = conn.compute.wait_for_server(server)
    print 'ip', server.access_ipv4


if __name__ == "__main__":
    sys.exit(main())

So... that's not terrible.  You can see there's a few hacks in there - some defaults are globals for simplicity instead of bothering with argparse - credentials from environ for the same reason - I made the conn instance a global because it's easier to get ahold of in a repl just by importing - there's that weird cert error - some ugly monkey patching to get at admin_pass - another function for finding a network by name to work around something I hit in the Rackspace neutron API.

Having to add the awkward default static uuid's for the Public and Service networks sorta threw me on my first go 'round.

But all in all this works for creating new servers with a private cloud network on the Rackspace cloud.  Hooray!

Openstack SDK with Rackspace

I used to use rackspace-novaclient, but that fell apart on me.

If you search for the official python SDK for Rackspace Cloud - you're likely to find pyrax.

But I guess by summer '15 they were full on the Openstack-SDK ride!

Openstack-SDK seems like a decent attempt to organize things.  Docs are decent.  But my first attempt to do something 401'd

In [68]: conn.authorize()
---------------------------------------------------------------------------
HttpException                             Traceback (most recent call last)
 in ()
----> 1 conn.authorize()

/private/tmp/test-os-sdk/lib/python2.7/site-packages/openstack/connection.pyc in authorize(self)
    264             raise exceptions.HttpException("Unknown exception",
    265                                            six.text_type(ex),
--> 266                                            status_code=500)
    267 
    268         return headers.get('X-Auth-Token') if headers else None

HttpException: HttpException: Unknown exception, Unauthorized (HTTP 401)

Turns out, the openstack.connection.Connection class doesn't work quite like novaclient, a few params are different.  Instead of using your API Key - you need to use your password:

from openstack import connection

conn = connection.Connection(
    auth_url='https://identity.api.rackspacecloud.com/v2.0/',
    username='username',
    password='password')  # <- NOT API KEY!!!

After that the only tricky part was setting the region:

conn.profile.set_region(conn.profile.ALL, 'IAD')