VBScript, PowerShell, Python, IronPython, Jython, or... ?

In my job, more and more I find that we end up building distributed architectures, i.e. multi-server installations where services are routinely spread around 20+ Windows boxes, plus the occasional Unix. These services have specific startup and shutdown sequences and dependencies, so they can't just be set to Automatic startup; we usually provide batch files to manage them, but this is quickly becoming ugly and unreliable as the number of machines go up every year. It's also tricky to test actual service availability -- some are binary-based, some are HTTP based, some are Weblogic services, etc etc etc... So I'm investigating alternatives.

The first, natural choice for me was obviously Python: it does everything I need, it's flexible etc. However, distributing scripts to customers is a bitch; either you compile everything with cx_freeze (crashy) or py2exe (no python 3! no 64bit bundling! party like it was 2004! have fun tracking down which un-redistributable DLL or manifest you need on each release...), or you drop an entire environment and teach the customer what python is -- not ideal.

The traditional "native" approach to these problems in the Windows world is VBScript. It's fairly flexible, doesn't need to be deployed on Windows Win2003+ (yes, we still deal with loads of Win2003 servers), documentation is extensive and there are plenty of resources out there.
The problem with VBScript, apart from the ugly syntax and pseudo-OOP quirkiness, is that it's clearly seen as legacy by Microsoft. Year after year, running scripts becomes more cumbersome, security checks increase and new technologies don't expose the necessary interfaces. Does it make sense, in 2011, to invest time and effort building solutions that are, de-facto, already obsolete?

So we come to PowerShell, Microsoft's "future of scripting", which must be the less intuitive shell I've ever had the pleasure to deal with. I simply can't get my head around the way it deals with input and output; it doesn't seem to have reference assignment, so you have to retrieve an object on every line before you can use it; the syntax seems to combine the worst of Perl and Bash, and it quickly becomes unreadable. Also, deploying it on anything older than Vista has to be done manually and has to be justified to customers.

I honestly can't see a good solution here. I keep looking at IronPython, but its infrastructure baffles me and I wouldn't know where to start redistributing programs (I don't use Visual Studio). It's clearly a second-class citizen in the .Net world, with all that it entails.

Maybe Jython? After all, the products we install will drop JREs absolutely everywhere, so I could leverage that. I'd like to avoid going full-Java if I can, I hate the whole byzantine toolchain and I'm not really up to speed with post-1.4 changes; plus, there's always a degree of customization required in each environment, so I'd like to keep stuff as easily-editable as possible.

Please feel free to drop any suggestions in comments, I could really do with them!

On The Burden Of Legacy

I recently thought it would be good to write a few utilities for one of the products I support. This product was built on DCOM (don't ask me why). Obviously, being a pythonaro, I just had to use Python. And so my tribulations began.

Python supports COM objects thanks to the lovely Win32 package by Mark Hammond. When you have a DLL that you want to use in python, run "makepy.py yourlib.dll" (or just makepy.py, and browse through the registered libraries to choose one) and lo, it's available as a first-class python module. So I do that, and start poking around with various stuff, and eventually I call a method that returns an object whose definition is in another DLL, so it only comes up as a PyIUnknown object. Ah well. I run makepy.py on the second DLL, to no avail.

After a bit of googling and fiddling, I finally stumbled on the right incantation.

myobj = win32com.client.Dispatch(myobj.QueryInterface(pythoncom.IID_IDispatch))
myobj = win32com.client.CastTo(myobj,'MyIClassName')

Basically, I have to find the right interface ID for the object, dispatch it, and then recast the object with a different type which is specified by a string. Man, isn't that ugly. Not because of having to recast (which I can understand) but because the recasting needs the class name as a string, you can't somehow derive it from the object. If you don't know the "secret" class name, you can't use its methods even if you know it's of the right type. From a modern OOP point of view, this is quite absurd... but there is an explanation.

COM was built when reflection was not yet mainstream, and types were static and immutable. Programmers were used to declare types for each variable, and if the type wasn't right, the compiler would cry. Polimorphism, reflection and dynamic interpreters were slowly reaching mainstream acceptance, but Microsoft had to retrofit them on top of existing frameworks, one feature at a time. The result was the incomplete hybrid I now have to deal with.

Well, I thought, this open-source CPython stuff in Redmond is considered akin to communism, no wonder it doesn't play well with old MS frameworks. Let's try to move further down "the Microsoft Way" and see if things improve: I shall use IronPython. After all, it's just a thin layer on top of the .Net CLR, right? Certainly it will be able to guess types a bit better...

So, a few installs later (you need at minimum the "Windows SDK 6.0" for good interop tools for .Net 2.0, plus the IronPython installer), there I was, trying to do the same thing.
The .Net version of makepy.py is called "tlbimp.exe", and it will basically build a new DLL to wrap the old one. You then import the new DLL in IronPython and lo, you can use the objects. But what happens when a function returns a reference to an object not defined in the DLL? Well, more or less the same thing as for win32com -- you get an unknown interface. So again,you have to wrap all the other DLLs, and explicitly instance these objects with the right type. Slightly better but not really that much different (so I'll stick to CPython, thank you).

As I said, this sort of explicit casting was perfectly acceptable 15 or even 10 years ago. But honestly, don't we live so much better without ? Why win32com and (more damning) IronPython cannot yet look up this sort of thing automatically? I understand that I'm working with legacy interop, but still... this sort of "programmer usability improvement" would help a lot in real-world situation where The Big Rewrite is simply not an option.

Liveblogging Pycon UK 2007 - first notes

Lots of people, fantastic atmosphere, lots of kudos to the Birmingham user group. "Msdn magazine" in the goodie bag sounds like MS really wants to invest in Python, that's probably why Sun decided to go on Ruby instead.

First talk on SQLALchemy by Paul Johnston, very very interesting, too bad it was a bit squeezed, another 15 minutes would have helped. Must investigate the reflection vs autoupdate stuff & migrate (django doesn't manage db changes very well at the moment).

Second talk: Mr. Voidspace (Michael Foorde) on Silverlight & IronPython -- lots of possibilities there, but still very very early. 1Mb of local space is very little for serious usage, there are accessibility issues (but possibly less than Flash) and limitations (1 Canvas only). But having an embedded mini-CLR/DLR in every browser (currently IE/FF/Safari, with Opera in the works) gives me a feeling of "ActiveX done right", something Java should have done 5 years ago.

Made contact with other Manchester pythonistas, it's really true that pythons hide under rocks! Looking forward to build a community in Manc when we go back, people really wants to invest serious time on Python apparently. Oh, and Resolver is hiring, but it's London-based and they do Xtreme (pair) programming, so no telecommuting, but if you are in the area and you fancy "coding the way Guido indented it" give it a go, they seem very nice guys.

Time for Django stuff with Simon Willison!

UPDATE: Simon rocks. Fast as lightning and to the point, lots of goodies for serious django usage, I hope the session was filmed because it was really worth it.

UPDATE: I met Phil Thompson, the creator of PyQt! Jeez, I probably sounded like a fanboy (that I am). And I also met a guy not just from around Manchester -- from Stockport! Astonishing, the world is so small these days. Am now on the PyQt tutorial from Mark Summerfield (who has a book finally coming out on PyQt! fantastic), Trolltech provided some very nice freebies. I feel in geek heaven.

UPDATE: Mark was great, but 2 hours straight are a bit much, so in the end the class was clearly a bit tired. Will definitely go back to his presentation very soon. Break now, then on to the lightning talks -- the list looks endless, might not do them all. Organizers expected about 100 people, got more than 200...

UPDATE: the first lightning talks: Open Spaces (weird, not sure I got it), a lovely chap trying to convert the Hansard (which is getting XML already) in RDF, Jeff Tupholme on putting javascript inside python (crazy) within LiveConnector.

UPDATE: pydoctor statically analyses code to generate docs and then uses a pseudo-wiki interface to correct typos and generate diffs (sounds nice); a lexer parsing thing which went over my head; a freakily-dressed guy from ACCU on how to pitch Python to C(++) shops (use "high-level"!); Software Freedom Day next week (eek! I'm in Oslo)!