Ansible Download or Fetch remote logs from nodes

I really like the Ansible connection management and inventory system (I use a dynamic inventory generated from my Terraform state files).  Playbook organization gives me fits sometimes, I tend to cut things up into small plays with just a few very closely related tasks and then roll those up into larger playbooks using imports so they cover an entire systems.  Then I'll have a good collection of fairly complex but decoupled playbooks that are mostly just imports of smaller plays.  I'll then import the fewer larger playsbooks into site.yaml so it reads like an overview.

I've tried roles, and tags - but they never seem to be satisfying.  IIRC, when I looked at the timeline the import statement was added to ansible *after* roles were introduced - so I assume that means my requirements or conceptualizations that lead me to want to develop, organize and compose things this way is not unprecedented.

I wish I was better at using modules adhoc from the command line.  I'm pretty good at grabbing:

ansible nodes -m shell -a "some bash"

... but I've also written more than a few plays that I only use once and throw away.  For example something like fetch_logs.yaml

- hosts: nodes
  become: true
  tasks:
    - name: fetch logs
      fetch:
        src=/var/log/swift/all.log
        dest=.scratch/logs/
        validate_checksum=false

And while it's entirely reasonable to have a single task play checked into version control in case you need it again later - I'm also finding it getting a little easier to start to convert these things to adhoc commands.

To use ansible to download remote logs from remote nodes using the fetch module adhoc on the commnad line, try something like this:

ansible nodes -b -m fetch -a "src=/var/log/swift/all.log dest=.scratch/logs/ validate_checksum=false"

In this command "nodes" is just an ansible group, to specify hosts, it could just as well be "all".  The "-b" option is for "--become", "-m" specifics the module and the space separated list of arguments is given with "-a". Hopefully I'll find this next time I forget.

Today is a good day.

Walking home from my run this morning, the sun rising above the hills to my back, casting off the cool mist at the break of day in the Sunset district of San Francisco, I began to climb, tired, up a hill behind an older Asian man, maybe two dozen years my senior, moving slowly and smoking a sweet smelling joint.  As I overtook him he turned to me, startled, and said not quite too loudly "Good day!" Assuming he would know, I felt compelled to echo his declaration and assured him, yes indeed it was, a "Good day."

As I walked home I thought I 'ought let y'all know; I have it on reputable authority...

Today is a good day.

My mental model for software

OpenStack for Operators

A mission is a strongly felt aim, ambition, or calling.

The OpenStack mission:

to produce the ubiquitous Open Source Cloud Computing platform that
enables building interoperable public and private clouds regardless of
size, by being simple to implement and massively scalable while serving
the cloud users’ needs

This is what I most strongly identify with OpenStack.

A value is one's judgment of what is important in life.

OpenStack states some values; which I think are also good:

Open Source - License; because IANAL
Open Design - Summits are fun; if you have a spec
Open Development - Patches welcome
Open Community - Let's talk; we're nice!

... the four opens are pretty great - but it's not what I feel called to
do - it's not what I'm about.  I just think that in order to build out
that ubiquitous platform; where the open source cloud is found
everywhere ... no one company can do that?  So we better have a good
strategy for a community if we're gunna do this thing.

And that's what I come to when I think about "what is OpenStack" - I
don't think about the software that OpenStack produces.  To me OpenStack
is a phenomenon - because I literally do not understand the cause or
explanation for why this thing took off like it did?

Maybe I think there's just a lot of hackers out there that all feel like
I do - that building this thing is important (to society?) - and that
they can make a meaningful contribution.

OpenStack is a system to organize the people committed to the mission.
The result is OpenStack software - not all of which is good - some of it
might not even be useful - but if it was done with a passion for the
mission and according to the values - we should be happy to call it
OpenStack.

Or maybe it's the money.

Maybe it's the system to organize the profit-generating-organisms that
feed the hackers committed to the mission, the astroturf, the latch your
wagon^Wdriver-plugin onto it while it's hot, the cushy arm-chair
architect pontificate on the grand unified theory of exception logging,
the ego of the -2, the paycheck, the passport to hang with friends
around a whiteboard.

Meanwhile, in the real world, the market is happily shipping AWS so much
money Google and Microsoft can't mount a reasonable defense - Rackspace
is going to tap out and HP already gave up.

If we're still committed to the mission - we have got to focus on our
deployments.

Nothing tells you more about what's important than working with real
deployments.  The people operating OpenStack software are the best
resource we have to cut through the junk and deal with what matters.

If you're not at least partially engaged with the delivery, deployment
and operation of OpenStack software I strongly recommend you have a talk
with your manager - or seek a different employer.

Listen to the operators.  Ignore everything else.  Ignore internal
organizational boundaries.  Ignore your direct manager.  Ignore the TC.
Ignore me.  Listen to the operators.

If they continue to deploy OpenStack's software - then OpenStack matters
- and we have to keep after the mission - if they do not continue to
deploy OpenStack software - then none of this matters.

#NoBillNoBreak #StopTheStunt

See the problem is, the frequency of high profile mass shootings is too high - it's making people that would prefer not to carry weapons feel uncomfortable.  That and all the murder.

There is in no clear answer.  Smart legislation may help in some cases.  No Fly No Buy would not be perfect.  I think we should vote on it!?

Why is there always so much name calling on social media?  Wag of the finger to both sides!  Just stop doing that!?  Phew, solved that problem.

But, ugh, the FUD.

• No one thinks smart regulations for firearms will solve murder.
• I don't think a radical redefinition of the 2nd amendment is possible in my lifetime - Bernie didn't either.
• Any broad support to legislate confiscation of legally obtained firearms would meet riots on a national level well before congress had a chance to shoot it down - I don't really see how people can honestly claim they think the government is going to "take their guns"?

And yet, every time guns enter the national stage - more Americans arm themselves?

We need to stop talking past each other.  Is there really no common ground?  No compromise?

Congress, do your job.  Get in there and debate it out - I know it's hard - people say stupid shit - seemingly refuse to listen to common sense.  But you're a professional.  You can show us how it's done!  Negotiate.  Find the best solution that you can get some agreement on and pass it!  Demonstrate progress!  Don't think about what sound byte will play best in the next news cycle - think about what is best for your constituents - not what is most likely to get you re-elected.  Then come November, show us someone who is good at their job.

Openstack SDK attach cloud network

I don't think the current OpenStack SDK supports the nova api extension for virtual interfaces.

I'm not a nova guy.

I don't understand the difference between the OpenStack compute v2 os-interfaces API - and the Openstack compute os-virtual-interfacesv2 extension API.

But rackspace seems to have had to write an extension for the novaclient CLI to make it able to attach a new virtual interfaces for a cloud network to a running instance.

I was thinking I might have to write a new resource to support the api extension (probably just crib from the existing ServerInterface) - but it turned out there's already support in OpenStackSDK to boot instances with extra networks.

Here's what I'm using:

#!/usr/bin/env python
import os
import requests
import sys


# default instance and network attributes
IMAGE = '09de0a66-3156-48b4-90a5-1cf25a905207'
FLAVOR = 'general1-4'
REGION = 'DFW'
NETWORK = 'my-test'
CIDR = '192.168.5.0/24'

auth_url = os.environ.get(
    'OS_AUTHURL', 'https://identity.api.rackspacecloud.com/v2.0/')
# username and *password* must be set in the envrion
username = os.environ['OS_USERNAME']
password = os.environ['OS_PASSWORD']


# something about the rackspace cert makes requests mad
requests.packages.urllib3.disable_warnings()

from openstack import connection
from openstack.compute.v2._proxy import _server


# the openstacksdk doesn't support admin_pass
# https://bugs.launchpad.net/python-openstacksdk/+bug/1528949
class PatchedServerResource(_server.Server):

    def create_by_id(self, *args, **kwargs):
        resp = super(PatchedServerResource, self).create_by_id(*args, **kwargs)
        self._attrs['adminPass'] = resp.get('adminPass')
        return resp

_server.Server = PatchedServerResource

conn = connection.Connection(auth_url=auth_url,
                             username=username,
                             password=password)

conn.profile.set_region(conn.profile.ALL, REGION)


def _get_or_create_network(name):
    # this doesn't seem to work
    # network = conn.network.find_network(name)
    # https://feedback.rackspace.com/forums/298158-network/suggestions/11188446-500-error-response-in-cloud-network-api

    # so we just search by hand
    for n in conn.network.networks():
        if n.name == name:
            return n
    network = conn.network.create_network(name=name)
    conn.network.create_subnet(
        network_id=network.id, ip_version='4', cidr=CIDR)
    return network


def main():
    try:
        server_name = sys.argv[1]
    except IndexError:
        return 'USAGE: %s <server-name>' % sys.argv[0]

    image = conn.compute.find_image(IMAGE)
    flavor = conn.compute.find_flavor(FLAVOR)
    network = _get_or_create_network(NETWORK)
    network_ids = (
        network.id,
        # must explicitly include Public & Service networks
        '11111111-1111-1111-1111-111111111111',
        '00000000-0000-0000-0000-000000000000',
    )
    server = conn.compute.create_server(
        name=server_name, image=image, flavor=flavor,
        networks=[{'uuid': id_} for id_ in network_ids]
    )
    print 'pass', server['adminPass']
    server = conn.compute.wait_for_server(server)
    print 'ip', server.access_ipv4


if __name__ == "__main__":
    sys.exit(main())

So... that's not terrible.  You can see there's a few hacks in there - some defaults are globals for simplicity instead of bothering with argparse - credentials from environ for the same reason - I made the conn instance a global because it's easier to get ahold of in a repl just by importing - there's that weird cert error - some ugly monkey patching to get at admin_pass - another function for finding a network by name to work around something I hit in the Rackspace neutron API.

Having to add the awkward default static uuid's for the Public and Service networks sorta threw me on my first go 'round.

But all in all this works for creating new servers with a private cloud network on the Rackspace cloud.  Hooray!

Openstack SDK with Rackspace

I used to use rackspace-novaclient, but that fell apart on me.

If you search for the official python SDK for Rackspace Cloud - you're likely to find pyrax.

But I guess by summer '15 they were full on the Openstack-SDK ride!

Openstack-SDK seems like a decent attempt to organize things.  Docs are decent.  But my first attempt to do something 401'd

In [68]: conn.authorize()
---------------------------------------------------------------------------
HttpException                             Traceback (most recent call last)
 in ()
----> 1 conn.authorize()

/private/tmp/test-os-sdk/lib/python2.7/site-packages/openstack/connection.pyc in authorize(self)
    264             raise exceptions.HttpException("Unknown exception",
    265                                            six.text_type(ex),
--> 266                                            status_code=500)
    267 
    268         return headers.get('X-Auth-Token') if headers else None

HttpException: HttpException: Unknown exception, Unauthorized (HTTP 401)

Turns out, the openstack.connection.Connection class doesn't work quite like novaclient, a few params are different.  Instead of using your API Key - you need to use your password:

from openstack import connection

conn = connection.Connection(
    auth_url='https://identity.api.rackspacecloud.com/v2.0/',
    username='username',
    password='password')  # <- NOT API KEY!!!

After that the only tricky part was setting the region:

conn.profile.set_region(conn.profile.ALL, 'IAD')

Eulogy for Thomas Clay Gerrard

Thank you for joining us today to celebrate and honor the memory of my Father.

Thomas Clay Gerrard

Uncle Tom

Dad

or more recently Paw Paw

although a lot of us probably called him mostly, just “Yes, Sir”

I’d imagine many of you can share an experience where my Dad held a fatherly like role in your life - protective, supportive.  But in case any of you ever tried to imagine what it might have been like to have Tom as your actual father - well, I’d like to set the record straight.

It was amazing.

Yeah, better than you could imagine.

I was so blessed to grow up in home centered and supported by a good man of strong character, a countryman, a good husband and a great father.  He was a great example, and he is my ideal role model.

Dad was a man of integrity, honesty and authenticity.  He never acted for the benefit of others perception - his actions were guided by his own principles - he would tell you he “could not care less what those people thought” - but he was not disrespectful of others.  He spoke with sincerity and frankness - without pretense - although he might temper an obvious truth with wit and sarcasm.  “And how did that work out for you?”  

He taught me how to show grace and strength in the face of adversity - how to respectfully disagree…

But I also learned, that if you have passion about something (and Dad had passion for everything he did — “If it’s worth doing; it’s worth doing right”) … I learned that if you have passion for something - it’s ok to show that - it’s ok to get mad when someone messes up - people will respect you more if they know where you stand.  Everyone knew that Dad had high expectations, for himself and for those with him.  You can either suck it up and do your best - or you can get out of the way - there’s no place to half-ass it with Tom.  But he’d never hold a grudge.  He was willing to get mad, but he was far quicker to forgive, and if you told him up front that you had made a mistake, he respected that a great deal, and there was barely another word about it.  He was very fair.

We’ve all been gushing this week about what an amazing man my father was, and on occasion someone would try to help find some perspective by remembering “Yeah but he was strict too” - and that’s true.  I have often described my father as setting very high expectations.  He absolutely expected that the rules were obeyed.  But he was always was up front about the ground rules, and there were always well defined consequences.
So honestly, even as a younger man, I didn’t recall him as strict.  As I recall, as a boy, he had taught me discipline.  And so after that, when I was learning to be a young man and he told me “you KNOW better” - he was right.

My father vigorously supported and loved his country.  His grit and natural talent served our country well - throughout two terms during the Vietnam War with a Naval Construction Battalion - the Fighting SeaBees.   He remained, throughout his life willing and ready to defend this country.  He carried his sense of duty, loyalty and self sacrifice forward onto his family.  I believe he felt personally responsible to ensure our freedom and safety.  And I always knew we were in good strong hands.

My father was wise.  He had the experience, he had the knowledge, and he had good judgment - which made him an exceptional teacher.

I recall a passing moment, as a grown man, at my Aunt Linda’s house.  Dad had heard Aunt Linda mention that she and Fellow had collected the remains of a number of wooden decks - and that they wanted to assemble them together into a deck of sorts around the back of their house.  Dad immediately set to organizing the affair.  We had a lot of help.  Some folks I didn’t even know.  There was a young man, he must have been someone’s boyfriend or a neighbor, Dad had set him to running his screw gun.  A few screws in Dad stopped him - and I overheard “Hey.  Slow down.  Anyone can screw some boards down - it’s not a race.  You need to line up each screw, space them evenly, and run them down straight and even.  That’s craftsmanship.  That’s what it means.  When someone steps onto this deck - they’ll know that the person who built it - did it with care - and that they did a good job”  It felt like an out of body experience, watching that young boy receive that wisdom that my father had given me years ago - and he nodded and he understood.  He did a good job.

My dad taught me so much.  I still had so much to learn.  I know that in time the shock I’m feeling now will give way to grief, and in time after that I’ll learn to carry that sorrow with dignity.  But I don’t think I’ll ever be able to deny the big hole in my life, where I knew I could always turn to him for answers, and support.

So uncle Bill, next time when I’m asking you something about my water heater or whatever it is - and I just flat out loose it - you cut me some slack a’ight.

Dad had an uncanny gift for service.  Service, i’ve read, is one of the “love languages” - and that is how he was best able to show us just how MUCH he loved us.  And it suited him - “actions speak louder than words”.  As you remember how much my father has done for you - try not to get caught feeling a great debt for his service - he was HAPPY to do it.  It was his gift.  He loves you very much.

Twistd upload FTP server

How to fix "Failed to retrieve directory listing"

I was trying to get a quick ftp server up and running, and it seems liked `twistd` is getting to be installed just about everywhere these days so it seemed simple enough:

twistd -n ftp -r .

Got a twistd up and serving ftp out of the current working directly for anonymous download.

But I actually wanted the server to let me upload (side note: people hate windows because it's terrible, what a wasteland, it's a terrible chore to get even the most basic things done without tools like scp?).

So I started with the simple ftpserver.py example.

But nothing is ever "simple" in Twisted.  It turns out the FTPRealm blah blah blah - no body cares.

I was getting an error doing the first directory listing when I connected to the server as an authenticated user, anonymous listing worked fine.  Here was the not helpful at all traceback:

2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,192.168.42.68] DTPFactory.setTimeout set to 10 seconds

2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,192.168.42.68] DTPFactory starting on 62816

2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,192.168.42.68] Starting factory

2013-09-02 13:03:54-0700 [twisted.protocols.ftp.DTPFactory] DTPFactory.buildProtocol

2013-09-02 13:03:54-0700 [twisted.protocols.ftp.DTPFactory] cancelling DTP timeout

Oh of course!  The DTPFactory/ProtcolWrapper blah blah - no body cares!

Silly thing was trying to read `/home/` - which while probably useful in on a Linux machine wasn't so helpful on my MacBook.

Here's what I ended up with:

"simple" FTP server on a Mac

#! /usr/bin/env python
import os

from twisted.internet import reactor
from twisted.protocols.ftp  import FTPFactory
from twisted.protocols.ftp  import FTPRealm 
from twisted.cred.portal    import Portal
from twisted.cred.checkers  import AllowAnonymousAccess
from twisted.cred.checkers import InMemoryUsernamePasswordDatabaseDontUse as \
        InMemoryDB

PASSWORD = ''

users = {
    os.environ['USER']: PASSWORD
}

p = Portal(FTPRealm('./', userHome='/Users'), 
    (   AllowAnonymousAccess(),
        InMemoryDB(**users),)
    )

f = FTPFactory(p)

reactor.listenTCP(2121, f)
reactor.run()

^ login with your username and blank password; it's not secure, duh.

Short commit SHA in Jenkins jobs' build name

I always forget this, and spend way too much time looking it up.  This is the part I'm looking for:

${GIT_REVISION,length=8}

Hi future me.

If you're not me, you probably already have the GitHub Plugin installed, which pulls in the base Git Plugin.  And if you started reading on the wiki page for the git plugin you may have noticed a reference to a environment variable available as GIT_COMMIT.  After some bit of searching you bump into the Build Name Setter Plugin, and try adding some stuff in the "Build Name" field on the job configuration maybe like:

${GIT_COMMIT}-${BUILD_NUMBER}

Which doesn't work, so you go back to more important stuff.

Or maybe you get lucky and realize the Token Macro Plugin is involved and stumble across the example there.  Which is hidden in-between some Java you can barely look at, is the small blurb at the end of a paragraph which mentions "${GIT_REVISION,length=8}".

What was interesting there for me was that the build name setter relies completely on the token macro plugin for the string expansion, and pulling strings out of the ENV is not it's primary purpose.  There's a special syntax even:

${ENV,var="GIT_COMMIT"}

Which if you discovered, probably frustrated you, since can't seem to use the ENV vars you export during the build script, or even do basic bash variable manipulation like ${GIT_COMMIT:0:8}.

At this point you're realizing the fact that the token macro expansion only looks like bash variable expansion to confuse you.

Jenkins plugins can and do internally in their java guts define and export sometimes parameterized "tokens" which are exported for the specific purpose of being available to the token macro plugin which the build name setter users.

I honestly had no idea how anyone was supposed to know how this stuff worked together short of trolling through the git plugin's source like I did and reading GitRevisionTokenMacro.  But when I looked at the commit, my eyes were opened!

Jenkin's plugin system allows authors to write these html snippets which will be inlined into the job configuration page!?  This is why I can never seem to find "the docs" for a plugin - they're just added in all over the place whenever you install them - wherever it seems relevant to the author who wrote them!

So my new plan is to "always click the ? first" - why wasn't that my old plan?

Default Devstack Cirros Image "root" password

If you forget to send in an ssh key for injection when you boot the devstack default "cirros" image you can still log-in with password auth.

username: cirros
password: cubswin:)

that's "cubswin" followed by a "colon" and "close-paren" (smily face)

e.g.

clayg@devstack:~$ ssh cirros@10.0.0.2
Warning: Permanently added '10.0.0.2' (RSA) to the list of known hosts.
cirros@10.0.0.2's password: cubswin:)

Just as an aside, the dropbear ssh server on the cirros image allows root login by default, but the root user's password on the image is basically disabled:

 root:!$1$LJwQnqlv$DK6oKqcTq9Rf2ClC.kMa3/:10933:0:99999:7:::
cirros:$1$LJwQnqlv$DK6oKqcTq9Rf2ClC.kMa3/:10933:0:99999:7:::

... that leading "!" in the encrypted password field for "root" in /etc/shadow makes all the difference.  If you remove it, you can see it's the same hash as the cirros user and the "cubswin:)" password works for root too.

If you're just having trouble getting into ssh/port 22, a bit of security groups should square you away:

nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

Commits to Redisco

So this is kinda cool, my first pull request on github was accepted:
test, fix

Redisco, is a neat project:

Redisco allows you to store objects in Redis. It is inspired by the Ruby library Ohm and its design and code are loosely based on Ohm and the Django ORM. It is built on top of redis-py. It includes container classes that allow easier access to Redis sets, lists, and sorted sets.

I'm excited about using redisco in a project I'm working on for work that has a django/jQuery front end and celery/redis-server back end.  Redisco itself is a pretty new project (initial commit May 13, 2010 ) but I think it serves and important niche, allowing a lower barrier of entry to projects that want to move away from a RDMS data-store and get their first taste of NoSQL.  Plus, the author was quick to help me fix problems that were effecting ME.

In fact, you can check it out right now - my bug fixes are already available on PyPI:

$ pip install redisco
Downloading/unpacking redisco
  Downloading redisco-0.1.1.tar.gz
  Running setup.py egg_info for package redisco
Installing collected packages: redisco
  Running setup.py install for redisco
Successfully installed redisco  

Py3K - PEP 355

So, we still don't get object oriented path manipulation in Py3K

$ python3
Python 3.1.1+ (r311:74480, Nov  2 2009, 14:49:22) 
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import path
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named path
>>> 

Is there still hope?

OK. Pronouncement: PEP 355 is dead. The authors (or the PEP editor)
can update the PEP.

I'm looking forward to a new PEP.

--Guido

PEP 355 died out years ago, but the reference implementation still exists, and IMHO is still garbage.  Will there ever be another attempt at a stdlib PathClass?

Happy Daddy!

Monozygotic twin boys!

How vain am I?

This vain:
http://code.djangoproject.com/ticket/5786

changed by jacob 

resolution set to fixed

Thanks to alextreme, lbruno, and clayg.



Yeah, that's MY bug fix in your django 1.2 (currently in beta, get your official copy March 9th!)

The really sad part is, it's not even really MY code change, not REALLY.  I just re-compiled some other guys patch as a top level diff and updated the tests... but I'm still TOTALLY jazzed about being the last submitter before the final commit.  Wait, is that vain?  Or just totally lame?

Python over-riding options with ConfigParser and OptionParser

It looks like it may be woth looking into argparse (non-standard library)

"""
The basic idea is that config information can be pulled out of

* Global config file
* User local config file
* environment variables
* command line

Higher priority comes later in the list. 
"""

But here's an implementation of cascading config file/command line options with ConfigParser and OptionParser - without subclassing.

Basically you just...

• Set global defaults in at the top of the module in the constants section
• Use those to set defaults in the config object
• Read in the config file possibly over-riding defaults
• Read out the values in the config object to a dictionary
• Use THOSE values to set the command line defaults

Then you can get any option you need out of the command line options argument

It's not very flexible, but it uses standard libraries and should be fairly  obvious.

#! /usr/bin/env python
__doc__ = """
example of cascading options with ConfigParser and OptionParser
"""

# imports
import sys
from ConfigParser import ConfigParser
from optparse import OptionParser
# constants
default_option = 'default_value'
default_toggle = True

# classes
# internal functions

def main():
    # create a config parser
    # config parser objects store all options as strings
    config = ConfigParser({'option':str(default_option),
                           'toggle':str(default_toggle),
                          })
    config.read('example.cfg')
    
    # create command line option parser
    parser = OptionParser("%prog [options]" + __doc__.rstrip())
    
    # configure command line options
    parser.add_option("-o", "--option", action="store", dest="option", help="set option")
    parser.add_option("-t", "--toggleOff", action="store_false", dest="toggle", help="set toggle off")
    parser.add_option("-T", "--toggleOn", action="store_true", dest="toggle", help="set toggle on")
    
    # read config objects defaults section into a dictionary
    config_options = config.defaults()
    # config_options is dictionary of strings, over-ride toggle to bool
    config_options['toggle'] = config.getboolean('DEFAULT', 'toggle')
    # feed dictionary of defaults into parser object
    parser.set_defaults(**config_options)

    # parse command line options
    (options, args) = parser.parse_args()

    print "option: %s" % options.option
    if options.toggle:
        print "toggle is ON"
    else:
        print "toggle is OFF"

    return 0;

if __name__ == '__main__':
    status = main()
    sys.exit(status)

vuze torrents get more peers

When I only had one or two torrents going at a time they seemed to download slower than my connection should have been able to support.  I was looking for ways to maximize my torrent download speed when I found the setting for "maximum number of connections" under options in Vuze.

Please note all the details here have only been tested to work to improve download speeds of freely distributable copyleft materials such as Ubuntu full DVD images.

If you're like me, you've already covered the basics (I had "green smiles" and "open nat") but if not, you may want to start with that.


My problem was that I was only connecting to 20-30 peers out of thounds of seeders and my down speed for a single torrent capped out around 150-170 kB/s.  I discovered that the max number of connections was set to 40, I don't remember changing it - hitting reset seemed to indicate that by default Vuze limits the max number of connections to 50:

 

This number maps directly to the number of peers in your swarm.  And mine was set much lower than the recommend "Good Settings" - I went ahead and just disabled it for now:

And I was pretty happy with the results:

post code snippets in google blogger

Howdy folks, here's some code:

def myfunc(*args, **kwargs):
    for arg in args:
        print arg
    for k,v in kwargs.items():
        print k,v 

First thing you need to do is make sure you're useing the Google Blogger's "Updated Editor".

1. Log into Blogger, and from the Dashboard goto Settings.
2. On the Settings tab, down towards the bottom under "Global Settings"
3. "Select post editor" - "Updated editor"

The updated editor actually solved a lot of the "problems" I was having with pre formatted text getting messed up when I went back and forth between compose and edit html.

But to post some really good looking source code you have to use SyntaxHighlighter.

SyntaxHighlighter is written in JavaScript, and there some css styling as well. To use SyntaxHighlighter on your google blogger page, you have to edit the template.

Under the Layout tab, go to "Edit HTML"
In the "Edit Template" box, scroll down toward the bottom.
You want to add the .js/.css references just before the last closing head tag (</head>):

<link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='Stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css' rel='Stylesheet' type='text/css'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPlain.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCpp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJScript.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJava.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPhp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPython.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushRuby.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushSql.js ' type='text/javascript'/>

SyntaxHighlighter makes it easy to copy the above snippet - just hover over the code, and click the "copy-to-clipboard" icon:

Then at the very bottom, you also want to set some global parameters just before the last closing body tag (</body>):

<script language='javascript'>
SyntaxHighlighter.config.bloggerMode = true;
SyntaxHighlighter.config.clipboardSwf = 'http://alexgorbatchev.com/pub/sh/current/scripts/clipboard.swf';
SyntaxHighlighter.all();
</script>

 You can then make highlighted code by wrapping your code snippet's in a
<pre> tag with a specially formatted "class" attribute, like this:

<pre class="brush:js">
alert("Hello world");
</pre>

Result:

alert("Hello world");

You can add brushes for different languages by adding more JavaScript files to your Template. The author hosts the most recent version here:
http://alexgorbatchev.com/pub/sh/current/

N.B. Just because you put html inside a  <pre> tag doesn't magically make html parsers know you're try to post literal HTML code - you still have to escape valid HTML using &lt; and the like or it won't be rendered correctly.  Try something like this.

I cobbled together the following steps from these sources:
SyntaxHighlighter Homepage
Using SyntaxHighlighter on BLOGGER
Publish Source code in Blogger
Syntax Highlighter NOT working in my googleblog

Newegg Shipping Halo: ODST early

I pre-ordered from Newegg, and it arrived Monday Sept 14th.

Play'd it through the first night on Heroic (6 hours).
It was fun.
No dual wielding.
No battle rifle.
You move surprisingly fast with a turret.
No match-making for firefight (yet?).

Halo: Mythic is a separate disk, with:

• Matchmaking
• Custom Games
• Theater
• Forge

...and ALL the multilayer maps.

Heretic is Midship.
Cathedral is like Epitaph, but VERY symmetrical.
Longshore seems pretty cool.

You play them off Mythic, you don't download them to your HD. Your Halo3 saved content is available from with-in Mythic automatically.

TypeError: dict objects are unhashable

My mom told me to update my blog. Hi mom.

I've been wanting to write this one for awhile anyway.

In retrospect it was rather naive - but I wonder who hasn't at one time tried to create a set of dictionaries:

|>>> parts = [
|... {'id':1, 'desc':'widget', 'detail':'rear widget'},
|... {'id':1, 'desc':'widget', 'detail':'front widget'},
|... {'id':2, 'desc':'gear', 'size':4},
|... {'id':3, 'desc':'cog', 'type':'green'},
|... ]
|>>> myset = set(parts)
|Traceback (most recent call last):
| File "", line 1, in ?
|TypeError: dict objects are unhashable


What did I expect it to do? I think the first time I tried this it seemed more reasonable. I think my list of dictionaries actually contained exactly the same keys, with some exact duplicates and I needed to uniquify the list. I acctually ended up doing something like the example at the bottom...

Dict objects are not hashable, read about hash tables and sets if it isn't obvious why it's important that objects in a set support a __hash__ method.

A hash function is really simple idea:
two equal objects MUST return the same hash
two un-equal objects should RARELY return the same key

But there's no really obvious reasonable way to implement hash on a dictionary of arbitrary keys and values.

Here's a couple dumb ideas for adding a hash function to a dict:


|def __hash__(self):
| key = 0

|def __hash__(self):
| id = 0
| for pair in sorted(self.items()):
| id += hash(pair)
| return hash(id)

And they "work" - at least in the sense that they remove the error:

|>>> class Part(dict):
|... def __hash__(self):
|... return self['id']
|...
|>>> myset = set([Part(x) for x in parts])
|>>> for part in myset:
|... print part
|...
|{'desc': 'widget', 'detail': 'rear widget', 'id': 1}
|{'desc': 'gear', 'id': 2, 'size': 4}
|{'type': 'green', 'id': 3, 'desc': 'cog'}
|{'desc': 'widget', 'detail': 'front widget', 'id': 1}

And that's great, if my parts list contained EXACTLY equal dictionaries they would be removed - I could turn it back into a list an continue on with everything uniquified! It might be worth nothing that the two parts with 'id' = 1 were not considered equal just because they returned the same hash. When there is a hash collision, the inhereited __eq__() method recognized that 'rear widget' != 'left widget' and that the two parts were distinct.

But what's really interesting what I've done by making a mutable object hashable...

It can be used as a dictionary key with surprisingly bad results:

|>>> part.__class__ # part is an instance of my Part class
|&ltclass '__main__.Part'>
|>>> part # a mutable object with a hash function
|{'desc': 'widget', 'detail': 'front widget', 'id': 1}
|>>> mydict = {} # mydict is a plain dictionary
|>>> mydict[part] = 1 # i can use part as a key!
|>>> part['id'] = 2 # I then modify the key
|>>> mydict[part] # there is no value assigned to this new "modified" key
|Traceback (most recent call last):
|File "", line 1, in ?
| KeyError: {'desc': 'widget', 'detail': 'front widget', 'id': 2}
|>>> mydict # or is there?
|{{'desc': 'widget', 'detail': 'front widget', 'id': 2}: 1}

Here's a fairly reasonable attempt at making a custom class that is mostly mutable dictionary, but has a safe and reasonable hash function. I'll also over-ridden __eq__ to ignore minor differences in the 'detail' between objects.


|>>> class Part(dict):
|... def __init__(self, part_dict):
|... if 'id' not in part_dict:
|... raise TypeError("Parts must have an id")
|... dict.__init__(self, part_dict)
|... def __setitem__(self, key, value):
|... if key == 'id':
|... raise ValueError("Part id's can't change - create a new part")
|... return dict.__setitem__(self, key, value)
|... def __hash__(self):
|... return self['id']
|... def __eq__(self, other):
|... a = self.copy()
|... del a['detail']
|... b = other.copy()
|... del b['detail']
|... return a == b
|...
|>>> for part in set([Part(x) for x in parts]):
|... print part
|...
|{'desc': 'widget', 'detail': 'rear widget', 'id': 1}
|{'desc': 'gear', 'id': 2, 'size': 4}
|{'type': 'green', 'id': 3, 'desc': 'cog'}

So if parts is a list of build materials, and you wanted to know how many distinct parts it takes to build this thing... the above Part class might be the right track. Aside from the ambiguity of added by over-riding "==" like that... dose anyone see any other problems with this?