Wednesday, December 23, 2015

Openstack SDK attach cloud network

I don't think the current OpenStack SDK supports the nova api extension for virtual interfaces.

I'm not a nova guy.

I don't understand the difference between the OpenStack compute v2 os-interfaces API - and the Openstack compute os-virtual-interfacesv2 extension API.

But rackspace seems to have had to write an extension for the novaclient CLI to make it able to attach a new virtual interfaces for a cloud network to a running instance.

I was thinking I might have to write a new resource to support the api extension (probably just crib from the existing ServerInterface) - but it turned out there's already support in OpenStackSDK to boot instances with extra networks.

Here's what I'm using:
#!/usr/bin/env python
import os
import requests
import sys

# default instance and network attributes
IMAGE = '09de0a66-3156-48b4-90a5-1cf25a905207'
FLAVOR = 'general1-4'
NETWORK = 'my-test'
CIDR = ''

auth_url = os.environ.get(
    'OS_AUTHURL', '')
# username and *password* must be set in the envrion
username = os.environ['OS_USERNAME']
password = os.environ['OS_PASSWORD']

# something about the rackspace cert makes requests mad

from openstack import connection
from openstack.compute.v2._proxy import _server

# the openstacksdk doesn't support admin_pass
class PatchedServerResource(_server.Server):

    def create_by_id(self, *args, **kwargs):
        resp = super(PatchedServerResource, self).create_by_id(*args, **kwargs)
        self._attrs['adminPass'] = resp.get('adminPass')
        return resp

_server.Server = PatchedServerResource

conn = connection.Connection(auth_url=auth_url,

conn.profile.set_region(conn.profile.ALL, REGION)

def _get_or_create_network(name):
    # this doesn't seem to work
    # network =

    # so we just search by hand
    for n in
        if == name:
            return n
    network =, ip_version='4', cidr=CIDR)
    return network

def main():
        server_name = sys.argv[1]
    except IndexError:
        return 'USAGE: %s <server-name>' % sys.argv[0]

    image = conn.compute.find_image(IMAGE)
    flavor = conn.compute.find_flavor(FLAVOR)
    network = _get_or_create_network(NETWORK)
    network_ids = (,
        # must explicitly include Public & Service networks
    server = conn.compute.create_server(
        name=server_name, image=image, flavor=flavor,
        networks=[{'uuid': id_} for id_ in network_ids]
    print 'pass', server['adminPass']
    server = conn.compute.wait_for_server(server)
    print 'ip', server.access_ipv4

if __name__ == "__main__":

So... that's not terrible.  You can see there's a few hacks in there - some defaults are globals for simplicity instead of bothering with argparse - credentials from environ for the same reason - I made the conn instance a global because it's easier to get ahold of in a repl just by importing - there's that weird cert error - some ugly monkey patching to get at admin_pass - another function for finding a network by name to work around something I hit in the Rackspace neutron API.

Having to add the awkward default static uuid's for the Public and Service networks sorta threw me on my first go 'round.

But all in all this works for creating new servers with a private cloud network on the Rackspace cloud.  Hooray!

Tuesday, December 22, 2015

Openstack SDK with Rackspace

I used to use rackspace-novaclient, but that fell apart on me.

If you search for the official python SDK for Rackspace Cloud - you're likely to find pyrax.

But I guess by summer '15 they were full on the Openstack-SDK ride!

Openstack-SDK seems like a decent attempt to organize things.  Docs are decent.  But my first attempt to do something 401'd

In [68]: conn.authorize()
HttpException                             Traceback (most recent call last)
 in ()
----> 1 conn.authorize()

/private/tmp/test-os-sdk/lib/python2.7/site-packages/openstack/connection.pyc in authorize(self)
    264             raise exceptions.HttpException("Unknown exception",
    265                                            six.text_type(ex),
--> 266                                            status_code=500)
    268         return headers.get('X-Auth-Token') if headers else None

HttpException: HttpException: Unknown exception, Unauthorized (HTTP 401)

Turns out, the openstack.connection.Connection class doesn't work quite like novaclient, a few params are different.  Instead of using your API Key - you need to use your password:

from openstack import connection

conn = connection.Connection(
    password='password')  # <- NOT API KEY!!!

After that the only tricky part was setting the region:

conn.profile.set_region(conn.profile.ALL, 'IAD')

Wednesday, December 16, 2015

Eulogy for Thomas Clay Gerrard

Thank you for joining us today to celebrate and honor the memory of my Father.

Thomas Clay Gerrard
Uncle Tom
or more recently Paw Paw
although a lot of us probably called him mostly, just “Yes, Sir”

I’d imagine many of you can share an experience where my Dad held a fatherly like role in your life - protective, supportive.  But in case any of you ever tried to imagine what it might have been like to have Tom as your actual father - well, I’d like to set the record straight.

It was amazing.

Yeah, better than you could imagine.

I was so blessed to grow up in home centered and supported by a good man of strong character, a countryman, a good husband and a great father.  He was a great example, and he is my ideal role model.

Dad was a man of integrity, honesty and authenticity.  He never acted for the benefit of others perception - his actions were guided by his own principles - he would tell you he “could not care less what those people people thought” - but he was not disrespectful of others.  He spoke with sincerity and frankness - without pretense - although he might temper an obvious truth with wit and sarcasm.  “And how did that work out for you?”  

He taught me how to show grace and strength in the face of adversity - how to respectfully disagree…

But I also learned, that if you have passion about something (and Dad had passion for everything he did — “If it’s worth doing; it’s worth doing right”) … I learned that if you have passion for something - it’s ok to show that - it’s ok to get mad when someone messes up - people will respect you more if they know where you stand.  Everyone knew that Dad had high expectations, for himself and for those with him.  You can either suck it up and do your best - or you can get out of the way - there’s no place to half-ass it with Tom.  But he’d never hold a grudge.  He was willing to get mad, but he was far quicker to forgive, and if you told him up front that you had made a mistake, he respected that a great deal, and there was barely another word about it.  He was very fair.
We’ve all been gushing this week about what an amazing man my father was, and on occasion someone would try to help find some perspective by remembering “Yeah but he was strict too” - and that’s true.  I have often described my father as setting very high expectations.  He absolutely expected that the rules were obeyed.  But he was always was up front about the ground rules, and there were always well defined consequences.
So honestly, even as a younger man, I didn’t recall him as strict.  As I recall, as a boy, he had taught me discipline.  And so after that, when I was learning to be a young man and he told me “you KNOW better” - he was right.
My father vigorously supported and loved his country.  His grit and natural talent served our country well - throughout two terms during the Vietnam War with a Naval Construction Battalion - the Fighting SeaBees.   He remained, throughout his life willing and ready to defend this country.  He carried his sense of duty, loyalty and self sacrifice forward onto his family.  I believe he felt personally responsible to ensure our freedom and safety.  And I always knew we were in good strong hands.

My father was wise.  He had the experience, he had the knowledge, and he had good judgment - which made him an exceptional teacher.
I recall a passing moment, as a grown man, at my Aunt Linda’s house.  Dad had heard Aunt Linda mention that she and Fellow had collected the remains of a number of wooden decks - and that they wanted to assemble them together into a deck of sorts around the back of their house.  Dad immediately set to organizing the affair.  We had a lot of help.  Some folks I didn’t even know.  There was a young man, he must have been someone’s boyfriend or a neighbor, Dad had set him to running his screw gun.  A few screws in Dad stopped him - and I overheard “Hey.  Slow down.  Anyone can screw some boards down - it’s not a race.  You need to line up each screw, space them evenly, and run them down straight and even.  That’s craftsmanship.  That’s what it means.  When someone steps onto this deck - they’ll know that the person who built it - did it with care - and that they did a good job”  It felt like an out of body experience, watching that young boy receive that wisdom that my father had given me years ago - and he nodded and he understood.  He did a good job.
My dad taught me so much.  I still had so much to learn.  I know that in time the shock I’m feeling now will give way to grief, and in time after that I’ll learn to carry that sorrow with dignity.  But I don’t think I’ll ever be able to deny the big hole in my life, where I knew I could always turn to him for answers, and support.

So uncle Bill, next time when I’m asking you something about my water heater or whatever it is - and I just flat out loose it - you cut me some slack a’ight.

Dad had an uncanny gift for service.  Service, i’ve read, is one of the “love languages” - and that is how he was best able to show us just how MUCH he loved us.  And it suited him - “actions speak louder than words”.  As you remember how much my father has done for you - try not to get caught feeling a great debt for his service - he was HAPPY to do it.  It was his gift.  He loves you very much.

Monday, September 2, 2013

Twistd upload FTP server

How to fix "Failed to retrieve directory listing"

I was trying to get a quick ftp server up and running, and it seems liked `twistd` is getting to be installed just about everywhere these days so it seemed simple enough:

twistd -n ftp -r .

Got a twistd up and serving ftp out of the current working directly for anonymous download.

But I actually wanted the server to let me upload (side note: people hate windows because it's terrible, what a wasteland, it's a terrible chore to get even the most basic things done without tools like scp?).

So I started with the simple example.

But nothing is ever "simple" in Twisted.  It turns out the FTPRealm blah blah blah - no body cares.

I was getting an error doing the first directory listing when I connected to the server as an authenticated user, anonymous listing worked fine.  Here was the not helpful at all traceback:

2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,] DTPFactory.setTimeout set to 10 seconds
2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,] DTPFactory starting on 62816
2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,] Starting factory
2013-09-02 13:03:54-0700 [twisted.protocols.ftp.DTPFactory] DTPFactory.buildProtocol
2013-09-02 13:03:54-0700 [twisted.protocols.ftp.DTPFactory] cancelling DTP timeout

Oh of course!  The DTPFactory/ProtcolWrapper blah blah - no body cares!

Silly thing was trying to read `/home/` - which while probably useful in on a Linux machine wasn't so helpful on my MacBook.

Here's what I ended up with:

"simple" FTP server on a Mac

#! /usr/bin/env python
import os

from twisted.internet import reactor
from twisted.protocols.ftp  import FTPFactory
from twisted.protocols.ftp  import FTPRealm 
from twisted.cred.portal    import Portal
from twisted.cred.checkers  import AllowAnonymousAccess
from twisted.cred.checkers import InMemoryUsernamePasswordDatabaseDontUse as \


users = {
    os.environ['USER']: PASSWORD

p = Portal(FTPRealm('./', userHome='/Users'), 
    (   AllowAnonymousAccess(),

f = FTPFactory(p)

reactor.listenTCP(2121, f)

^ login with your username and blank password; it's not secure, duh.

Wednesday, March 13, 2013

Short commit SHA in Jenkins jobs' build name

I always forget this, and spend way too much time looking it up.  This is the part I'm looking for:
Hi future me.

If you're not me, you probably already have the GitHub Plugin installed, which pulls in the base Git Plugin.  And if you started reading on the wiki page for the git plugin you may have noticed a reference to a environment variable available as GIT_COMMIT.  After some bit of searching you bump into the Build Name Setter Plugin, and try adding some stuff in the "Build Name" field on the job configuration maybe like:
Which doesn't work, so you go back to more important stuff.

Or maybe you get lucky and realize the Token Macro Plugin is involved and stumble across the example there.  Which is hidden in-between some Java you can barely look at, is the small blurb at the end of a paragraph which mentions "${GIT_REVISION,length=8}".

What was interesting there for me was that the build name setter relies completely on the token macro plugin for the string expansion, and pulling strings out of the ENV is not it's primary purpose.  There's a special syntax even:
Which if you discovered, probably frustrated you, since can't seem to use the ENV vars you export during the build script, or even do basic bash variable manipulation like ${GIT_COMMIT:0:8}.

At this point you're realizing the fact that the token macro expansion only looks like bash variable expansion to confuse you.

Jenkins plugins can and do internally in their java guts define and export sometimes parameterized "tokens" which are exported for the specific purpose of being available to the token macro plugin which the build name setter users.

I honestly had no idea how anyone was supposed to know how this stuff worked together short of trolling through the git plugin's source like I did and reading GitRevisionTokenMacro.  But when I looked at the commit, my eyes were opened!

Jenkin's plugin system allows authors to write these html snippets which will be inlined into the job configuration page!?  This is why I can never seem to find "the docs" for a plugin - they're just added in all over the place whenever you install them - wherever it seems relevant to the author who wrote them!

So my new plan is to "always click the ? first" - why wasn't that my old plan?

Friday, August 17, 2012

Default Devstack Cirros Image "root" password

If you forget to send in an ssh key for injection when you boot the devstack default "cirros" image you can still log-in with password auth.

username: cirros
password: cubswin:)

that's "cubswin" followed by a "colon" and "close-paren" (smily face)


clayg@devstack:~$ ssh cirros@
Warning: Permanently added '' (RSA) to the list of known hosts.
cirros@'s password: cubswin:)

Just as an aside, the dropbear ssh server on the cirros image allows root login by default, but the root user's password on the image is basically disabled:


... that leading "!" in the encrypted password field for "root" in /etc/shadow makes all the difference.  If you remove it, you can see it's the same hash as the cirros user and the "cubswin:)" password works for root too.

If you're just having trouble getting into ssh/port 22, a bit of security groups should square you away:

nova secgroup-add-rule default tcp 22 22

Monday, August 2, 2010

Commits to Redisco

So this is kinda cool, my first pull request on github was accepted:
test, fix

Redisco, is a neat project:

Redisco allows you to store objects in Redis. It is inspired by the Ruby library Ohm and its design and code are loosely based on Ohm and the Django ORM. It is built on top of redis-py. It includes container classes that allow easier access to Redis sets, lists, and sorted sets.
I'm excited about using redisco in a project I'm working on for work that has a django/jQuery front end and celery/redis-server back end.  Redisco itself is a pretty new project (initial commit May 13, 2010 ) but I think it serves and important niche, allowing a lower barrier of entry to projects that want to move away from a RDMS data-store and get their first taste of NoSQL.  Plus, the author was quick to help me fix problems that were effecting ME.

In fact, you can check it out right now - my bug fixes are already available on PyPI:
$ pip install redisco
Downloading/unpacking redisco
  Downloading redisco-0.1.1.tar.gz
  Running egg_info for package redisco
Installing collected packages: redisco
  Running install for redisco
Successfully installed redisco