Monday, September 2, 2013

Twistd upload FTP server

How to fix "Failed to retrieve directory listing"


I was trying to get a quick ftp server up and running, and it seems liked `twistd` is getting to be installed just about everywhere these days so it seemed simple enough:

twistd -n ftp -r .

Got a twistd up and serving ftp out of the current working directly for anonymous download.

But I actually wanted the server to let me upload (side note: people hate windows because it's terrible, what a wasteland, it's a terrible chore to get even the most basic things done without tools like scp?).

So I started with the simple ftpserver.py example.

But nothing is ever "simple" in Twisted.  It turns out the FTPRealm blah blah blah - no body cares.

I was getting an error doing the first directory listing when I connected to the server as an authenticated user, anonymous listing worked fine.  Here was the not helpful at all traceback:

2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,192.168.42.68] DTPFactory.setTimeout set to 10 seconds
2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,192.168.42.68] DTPFactory starting on 62816
2013-09-02 13:03:54-0700 [FTP (ProtocolWrapper),0,192.168.42.68] Starting factory
2013-09-02 13:03:54-0700 [twisted.protocols.ftp.DTPFactory] DTPFactory.buildProtocol
2013-09-02 13:03:54-0700 [twisted.protocols.ftp.DTPFactory] cancelling DTP timeout

Oh of course!  The DTPFactory/ProtcolWrapper blah blah - no body cares!

Silly thing was trying to read `/home/` - which while probably useful in on a Linux machine wasn't so helpful on my MacBook.

Here's what I ended up with:

"simple" FTP server on a Mac

#! /usr/bin/env python
import os

from twisted.internet import reactor
from twisted.protocols.ftp  import FTPFactory
from twisted.protocols.ftp  import FTPRealm 
from twisted.cred.portal    import Portal
from twisted.cred.checkers  import AllowAnonymousAccess
from twisted.cred.checkers import InMemoryUsernamePasswordDatabaseDontUse as \
        InMemoryDB

PASSWORD = ''

users = {
    os.environ['USER']: PASSWORD
}

p = Portal(FTPRealm('./', userHome='/Users'), 
    (   AllowAnonymousAccess(),
        InMemoryDB(**users),)
    )

f = FTPFactory(p)

reactor.listenTCP(2121, f)
reactor.run()


^ login with your username and blank password; it's not secure, duh.

Wednesday, March 13, 2013

Short commit SHA in Jenkins jobs' build name

I always forget this, and spend way too much time looking it up.  This is the part I'm looking for:
${GIT_REVISION,length=8}
Hi future me.

If you're not me, you probably already have the GitHub Plugin installed, which pulls in the base Git Plugin.  And if you started reading on the wiki page for the git plugin you may have noticed a reference to a environment variable available as GIT_COMMIT.  After some bit of searching you bump into the Build Name Setter Plugin, and try adding some stuff in the "Build Name" field on the job configuration maybe like:
${GIT_COMMIT}-${BUILD_NUMBER}
Which doesn't work, so you go back to more important stuff.

Or maybe you get lucky and realize the Token Macro Plugin is involved and stumble across the example there.  Which is hidden in-between some Java you can barely look at, is the small blurb at the end of a paragraph which mentions "${GIT_REVISION,length=8}".

What was interesting there for me was that the build name setter relies completely on the token macro plugin for the string expansion, and pulling strings out of the ENV is not it's primary purpose.  There's a special syntax even:
${ENV,var="GIT_COMMIT"}
Which if you discovered, probably frustrated you, since can't seem to use the ENV vars you export during the build script, or even do basic bash variable manipulation like ${GIT_COMMIT:0:8}.

At this point you're realizing the fact that the token macro expansion only looks like bash variable expansion to confuse you.

Jenkins plugins can and do internally in their java guts define and export sometimes parameterized "tokens" which are exported for the specific purpose of being available to the token macro plugin which the build name setter users.

I honestly had no idea how anyone was supposed to know how this stuff worked together short of trolling through the git plugin's source like I did and reading GitRevisionTokenMacro.  But when I looked at the commit, my eyes were opened!


Jenkin's plugin system allows authors to write these html snippets which will be inlined into the job configuration page!?  This is why I can never seem to find "the docs" for a plugin - they're just added in all over the place whenever you install them - wherever it seems relevant to the author who wrote them!

So my new plan is to "always click the ? first" - why wasn't that my old plan?

Friday, August 17, 2012

Default Devstack Cirros Image "root" password

If you forget to send in an ssh key for injection when you boot the devstack default "cirros" image you can still log-in with password auth.

username: cirros
password: cubswin:)

that's "cubswin" followed by a "colon" and "close-paren" (smily face)

e.g.

clayg@devstack:~$ ssh cirros@10.0.0.2
Warning: Permanently added '10.0.0.2' (RSA) to the list of known hosts.
cirros@10.0.0.2's password: cubswin:)

Just as an aside, the dropbear ssh server on the cirros image allows root login by default, but the root user's password on the image is basically disabled:

 root:!$1$LJwQnqlv$DK6oKqcTq9Rf2ClC.kMa3/:10933:0:99999:7:::
cirros:$1$LJwQnqlv$DK6oKqcTq9Rf2ClC.kMa3/:10933:0:99999:7:::

... that leading "!" in the encrypted password field for "root" in /etc/shadow makes all the difference.  If you remove it, you can see it's the same hash as the cirros user and the "cubswin:)" password works for root too.

If you're just having trouble getting into ssh/port 22, a bit of security groups should square you away:

nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

Monday, August 2, 2010

Commits to Redisco

So this is kinda cool, my first pull request on github was accepted:
test, fix

Redisco, is a neat project:


Redisco allows you to store objects in Redis. It is inspired by the Ruby library Ohm and its design and code are loosely based on Ohm and the Django ORM. It is built on top of redis-py. It includes container classes that allow easier access to Redis sets, lists, and sorted sets.
I'm excited about using redisco in a project I'm working on for work that has a django/jQuery front end and celery/redis-server back end.  Redisco itself is a pretty new project (initial commit May 13, 2010 ) but I think it serves and important niche, allowing a lower barrier of entry to projects that want to move away from a RDMS data-store and get their first taste of NoSQL.  Plus, the author was quick to help me fix problems that were effecting ME.

In fact, you can check it out right now - my bug fixes are already available on PyPI:
$ pip install redisco
Downloading/unpacking redisco
  Downloading redisco-0.1.1.tar.gz
  Running setup.py egg_info for package redisco
Installing collected packages: redisco
  Running setup.py install for redisco
Successfully installed redisco  

Thursday, July 1, 2010

Py3K - PEP 355

So, we still don't get object oriented path manipulation in Py3K

$ python3
Python 3.1.1+ (r311:74480, Nov  2 2009, 14:49:22) 
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import path
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named path
>>> 

Is there still hope?
OK. Pronouncement: PEP 355 is dead. The authors (or the PEP editor)
can update the PEP.

I'm looking forward to a new PEP.

--Guido
PEP 355 died out years ago, but the reference implementation still exists, and IMHO is still garbage.  Will there ever be another attempt at a stdlib PathClass?

Monday, May 24, 2010

Monday, March 1, 2010

How vain am I?

This vain:
http://code.djangoproject.com/ticket/5786

changed by jacob 

resolution set to fixed

Thanks to alextreme, lbruno, and clayg.



Yeah, that's MY bug fix in your django 1.2 (currently in beta, get your official copy March 9th!)

The really sad part is, it's not even really MY code change, not REALLY.  I just re-compiled some other guys patch as a top level diff and updated the tests... but I'm still TOTALLY jazzed about being the last submitter before the final commit.  Wait, is that vain?  Or just totally lame?