There’s No Hope For IT
October 29th, 2007 | Published in dynamic languages, enterprise, REST, services | 26 Comments | Bookmark on Pinboard.in
When you read stuff like this, you can’t help but feel that IT is, without a doubt, doomed.
The gist of the posting is
- REST is too hard for the average developer
- Dynamic languages are too hard for the average developer
Along these same lines, a couple of folks told me in person that my recent blog entries about REST, dynamic languages, and ESBs were misguided because today’s enterprises are interested only in approaches, frameworks and platforms that allow average developers to produce quality systems.
This all strikes me as nothing but wrong-headed thinking.
On the dynamic language front, the worst code I have seen in my career has always, always, always been in compiled imperative languages, most often Java and C++. I would much rather let an average developer loose with a dynamic language, because the surface area is smaller, there’s a lot less rope available for self-hanging, and if they’re going to fail, they’ll fail way faster and thus allow much more time for recovery. The fact that dynamic language programs are usually smaller than their compiled counterparts means that they’re easier to read and review, and statistically, they’re likely to have fewer bugs. Furthermore, counting on the static language compiler to save you is simply wishful thinking. To paraphrase Tim Ewald from a conversation he and I had during lunch a week or so ago, compilation really amounts to just another unit test.
On the REST front, if you’re claiming that it’s harder than the alternatives, to me that’s just a sign that you don’t understand it. Is REST simple? No, but neither is SOA. However, unlike SOA, which is fairly wishy-washy, noncommittal, and loose, REST’s constraints provide real, actual guidance for developers, and those same constraints also provide opportunities for significant flexibility, extensibility, performance, scalability, and serendipity. SOA’s contracts come with no rules or constraints, and thus can easily result in a system that’s extremely brittle, tightly-coupled, and virtually impossible to upgrade. SOA itself isn’t inherently bad, as it’s certainly a step above the “every application for itself” mode of development that’s so widely practiced. Unlike REST, though, SOA doesn’t go nearly far enough to provide real, useful guidance to the poor developer who has to actually write the stuff, make it work, and keep it running.
And finally, regarding the overall notion that enterprises cater only to average developers, I’m not sure I agree. In my former life I met countless enterprise developers who were extremely sharp. While I have no doubt that there are numerous bean-counting CIOs and middle IT managers out there who think they can build high-quality IT systems with low-quality developers, at the end of the day, businesses generally know better than to think they can get something for nothing. Or to put it another way, they know they get what they pay for, and if they pay only for average developers or worse, they’ll get only average software and average systems, or worse. That’s a no-brainer.
If you’re in a position of technical leadership or project management and you’re asked to come in ahead of schedule and under budget, my advice is that you’re generally more likely to succeed with REST and dynamic languages than with the alternatives because their inherent constraints allow for better focus. Also, if you find yourself in such a position, you owe it to yourself and your team to continually lobby your superiors to help them understand the very real costs of their budgetary stinginess.
October 30th, 2007 at 5:46 pm (#)
“REST is too hard for the average developer”
REST is not any more difficult than WebServices!! Building distributed system is hard, whether you use a RESTful approach or SOA or some other technology. But as u said a set of constraints can better guide an average developer than in WS-* where there are no such constraints. But this doesn’t mean that REST is the way to go and WS-* is not. My point is that I don’t accept that REST is harder (or easy) than the alternatives.
However I find that most people don’t know or understand these constraints. What they don’t understand, they perceive as hard.
“Dynamic languages are too hard for the average developer”
You’re right when you said that there is less rope to hang yourself with dynamic languages. Forget about dynamic languages. When Java took care of pointers and garbage collection, it enabled average programmers to code in such way that wouldn’t have been possible with c/c++. So how come dynamic languages are harder for average folks? The following phrase summarizes it all.
“The fact that dynamic language programs are usually smaller than their compiled counterparts means that they’re easier to read and review, and statistically, they’re likely to have fewer bugs”
Some folks think that dynamic languages are for sissies and java/c++ is for the real boys. I strongly believe that dynamic languages are going to play a larger role in enterprise software in the future. The power, the simplicity the compactness is unbelievable. The Apache Qpid project has python and ruby clients for a reason.
“today’s enterprises are interested only in approaches, frameworks and platforms that allow average developers to produce quality systems.”
Steve, I think this misconception is promoted by vendors out there. Why??? They want to tell the managers,CIOs,CTOs and CEOs that they can hire average developers and keep their budget low, by using their product/framework/platform that can magically fill in the gaps. Isn’t that what the whole EJB circus was about? Average developers can’t handle remoting,persistence,transactions,scalability ..etc so we have this wonderful app server that makes their life easy. We all know what followed.
As I said before, at the end of the day “You cannot offset your lack of architectural vision by using some framework, product or vendor.”
October 31st, 2007 at 1:33 am (#)
[…] you like REST interfaces like me, check out this very nice article from Steve […]
October 31st, 2007 at 1:50 am (#)
Most of REST is embodied in an architecture that uses urls for identification, PUT and GET for communication, and exchanging appropriate document types. Doing REST right in legacy languages common today isn’t nearly as easy as it should be, either over HTTP or via another transport. It isn’t an architecture problem, it’s a API problem. Too much or too little abstraction is often applied. We also have a problem of a lack of good document types in many areas.
I know we can do better, and I know that the natural complexity of a REST application is much less than for the null architectural style.
Architects do need to be involved to ensure the software under their control stays on track. I believe that if the foundations are laid properly, even average developers will perform better under REST than under null. REST provides good foundations for architects to base their decisions on, and to set the the level of control they need to exercise.
Benjamin.
October 31st, 2007 at 2:33 am (#)
He has a point about static vs dynamic. If you assume that the languages are similarily difficult. But the point is that with a dynamic language you can make is simpler.
In the choice between Java and, say, Perl, I’d also go for a static language. :-)
October 31st, 2007 at 10:19 am (#)
A good friend of mine has no technical background whatsoever, in stark contrast to me with 31 years of writing software under my increasingly wide belt.
He’s now officially “had enough” and needs to get started on the programming path.
My thoughts are that Java is really the best, most flexible way to start, but your static/dynamic comment does give me pause. Should I start him on something like Python instead?
Thoughts?
October 31st, 2007 at 10:52 am (#)
Hi Mad William, Python or Ruby would be a great way to start, I think. With Java I think your friend might get lost in all the noise.
October 31st, 2007 at 11:17 am (#)
[…] dynamic languages, I found myself mostly siding with the dynamic language crowd in the form of Steve Vinoski and Patrick Logan. This time around the discussion was all about what to do with “average […]
October 31st, 2007 at 11:19 am (#)
Another great post. As usual, you have me thinking on an interesting tangent on how to use dynamic languages on top of REST to maximize the goodness of serendipity:
http://smoothspan.wordpress.com/2007/10/31/serendipity-is-the-key-to-code-reuse/
October 31st, 2007 at 2:58 pm (#)
What’s difficult about REST is understanding why it is better than WS-* in most scenarios.
I like the idea of dynamically typed languages, but I don’t like the idea of languages, like Ruby, that are not type-safe. At work, I continuously find myself having to weed through, understand, and refactor large blocks of code not written by myself. I just can’t imagine doing this in a language that is not type-safe and which doesn’t give the hints to my IDE that I’m used to with Java.
Seriously, why can’t we have it all? A type-safe, dynamically typed, zero-turnaround language that has metadata/annotations?
Finally Steve, are you sure you like Ruby, et al. because its dynamically typed or because of its zeroturnaround features? I think if most Ruby noobs were honest with themselves, they’d answer the latter.
October 31st, 2007 at 3:22 pm (#)
Hi Bill, yes, I am completely sure I like dynamically typed languages because of the dynamic typing. Often, types simply get in the way and you have to play games to appease the compiler. I like a language that lets me say what I need to say. Zero-turnaround is an added bonus.
I don’t use IDEs and so don’t depend on them. Emacs and its embedded Lisp provide all the IDE I’ve ever needed. But perhaps those large blocks of code you have to refactor wouldn’t be so large if they were written in a tighter language to begin with. I haven’t had any trouble at all refactoring other people’s bad perl, for example, without an IDE, and I’ve seen plenty of bad perl over the years.
November 1st, 2007 at 12:57 am (#)
I consider types to be an absolute minimum acceptable level of documentation. So why not let them compiler infer your documentation, then prove that it’s accurate and consistent?
If the type system gets in your way, you should be campaigning for a better type system in your language, not compromising your documentation!
November 1st, 2007 at 11:02 am (#)
Steve, you know I learned everything I know from you…, but,
Still not using an IDE? I started a few years ago, now I can’t live without it. The refactoring and introspection support they provide you are just too invaluable. Seriously, tackle a large Java project and you’ll see what I mean. I just can’t see myself being as remotely productive as I am now going to a language that doesn’t have type-safety backed up with a strong IDE that can do refactoring.
As far as “playing games to appease the compiler”, Greg M is right, you need to move to a different language with a better type system. I just don’t see the types of problems you’re talking in Java that I saw continuously when I worked for you at Iona with our C++ project.
Finally, beyond IDEs, Java annotations have been the biggest productivity booster for me in the past few years. Yet another language feature I just can’t live without now and that doesn’t exist in Ruby, Erlang, Pythong, etc…
What I really want is a type safe, dynamically typed language that make it easier for me to add behavior to annotations and is zero-turnaround. This is why I think Groovy has so much promise.
November 1st, 2007 at 11:05 am (#)
Also, correct me if I’m wrong, but dynamic typing != type safety. With a type safe language, there’s no reason you can’t have dynamic typing that allows you to add mixins, methods, and properties to an existing type.
November 1st, 2007 at 4:59 pm (#)
Hi Bill: note that dynamic typing != typelessness. See this definition.
Regarding annotations: dynamic languages typically have facilities or capabilities that can do everything that Java annotations can do, and more.
Regarding IDEs: to each his own. I’ve used them, but they just never grew on me. Even so, AFAICS, my productivity is still reasonably high.
November 4th, 2007 at 5:34 am (#)
All this hype of SOAP vs REST reminds me of WiFi vs Bluetooth. My take is that the larger the number of alternatives I have to develop an IT architecture the better for me, for the business, for IT. Tying yourself to a single paradigm is like trying to make a choice between chocolate and vanilla icecream: a no-brainer.
On whether SOAP is easier that REST, it is up to you to complicate your life in every single way you can. But again, I like things simple (as long as I can keep them simple.) Chances are you can complicate your architecture as much using SOAP as using REST.
I am no taliban of dynamics languages but having worked with at least 10 different programming languages in my previous lives, I would start again with Python from scratch, should I be given the chance to. This would have saved me a few disappointments and bad habits :)
My 2 cents!!
November 4th, 2007 at 11:23 am (#)
“REST is too hard for the average developer”
Distributed computing is too hard for the average developer. But I think you’re talking about the lack of tools.
“Still not using an IDE? I started a few years ago, now I can’t live without it. The refactoring and introspection support they provide you are just too invaluable. Seriously, tackle a large Java project and you’ll see what I mean. I just can’t see myself being as remotely productive as I am now going to a language that doesn’t have type-safety backed up with a strong IDE that can do refactoring.”
Bill, this is Cedric’s core argument about Java. Logically it seems to hold up, but it’s contingent on the “large” part, and probably self-circular. Java programs are necessarily large, because the language itself is verbose. It’s verbose because of the type declarations, lack of easy access to certain language features, and some redundant syntax. Because it’s verbose we break it up into chunks that fit better on the screen. Becuase we have so many chunks it’s harder to navigate. Becuase it’s hard to navigate we need IDEs to browse it. Because the code is part of a living system, we need to keep changing those chunks around. Becuase we need to chnage those chunks around safely, we use refactoring techniques. The type system lets us do that with syntactic safety. Thank goodness for the type system and automated refactoring!
Remember: refactoring has nothing to do with the type system, but some automated refactoring does.
And let’s be realistic – Java devs always end up running the code through a debugger anyway (and the mass refactorings come later); that’s no better or worse than how it’s done in Python or Ruby.
It’s also a point in time argument – it doesn’t explain how large Java programs were written prior to 2002 or so, when all we really had was autocomplete (which is available to dynamic languages today), and the compile step. Or are we saying that no large Java programs existed before then? Or could not exist then without automated refactoring support? I don’t buy that. The IDEs/refactoring argument also drives you to do everything in the IDE out of neccessity – this alone explains the existence of technology like GWT. At some level I suspect it means you don’t have to care about program size as much, hence programs do get bigger as we are time and resource bound in our work.
I’d encourage anyone to work on a large dynamic codebase for a few years, investing as much time into its upkeep as they would a C# or Java system.
I’d also love to hear a justification that configuration/resource/xml hell of large Java systems is not strongly related to Java’s static typing, making it difficult to use the language for configuration (I suspect it is).
And yet. Despite the pros given to Java by its tools and type system, the untold millions of dollars invested in it, developing web applications is still ridiculously cumbersome on Java. This is a huge problem for the platform’s future viability. It should be clear by now that web frameworks are the new middleware, because the Web is the best service integration platform available to us. When people say Java is the new COBOL, that’s what they’re talking about. As frameworks, Rails, Zope3 and Django wipe the floor with anything written in Java.
“Java annotations have been the biggest productivity booster for me in the past few years. ”
Python has annotations; they’re clean, you’d probably like them. I’m not sure, but I suspect Erlang doesn’t need annotations.
So, I understand the reasoning but it seems to me that static typing can be just as much a problem with Java as a feature and the jury’s out on the overall cost/benefit. Personally I think langauge choice is not a key factor for maintainance with one exception I’ll mention below.
I have the advantage of having done a lot of work in both Python and Java. I use IntelliJ, Eclipse and Emacs a lot. What I haven’t seen is the sky falling in as predicted because of the lack of type declarations in dynamic languages. I have seen Java used where it was not suited, and that was because of its type system. I have seen oversized, always-shipping-late teams who just don’t like dynamic languages and use the type/scale/architecture arguments as rationales. At my most cynical I’d say CV management and making safe bets was more important in some cases than building great systems. In terms of impact, I’ve seen way more NPEs/CNFEs/CCEs in production Java code than attribute/field errors in Python.
I’ve also seen far more impenetrable mind-numbing logic and total lack of cohesion in Java – the kind of code that makes you want to rewrite from scratch.
The real issue is that there aren’t enough Python/Ruby/Erlang/JavaScript/* developers to go around. It has nothing to do with type safety. If there’s a maintanance problem with not using Java, it’s the job pool.
November 4th, 2007 at 8:07 pm (#)
“It’s also a point in time argument – it doesn’t explain how large Java programs were written prior to 2002 or so, when all we really had was autocomplete (which is available to dynamic languages today),”
Funny, that’s when I started using an IDE, post 2002. Even funnier, 2000-2002, I was coding a lot in Python.
“I have the advantage of having done a lot of work in both Python and Java. I use IntelliJ, Eclipse and Emacs a lot. What I haven’t seen is the sky falling in as predicted because of the lack of type declarations in dynamic languages.”
Who said the sky was falling? For me it is entirely about productivity. I was productive, like Steve, prior to 2002 when I primarily used Emacs and coded in both Java, Python, C++, C, Fortran, etc… The thing is, I *know* I’m more productive with Intellij and Java. More so than I could be with Python, which I’m pretty familiar with, and Ruby, that I’m not so familiar with. Mainly, it stems from that I’ve been working on the same project for 6+ years and have had to do major refactorings of the same codebase over that time as well as having to learn and refactor a lot of code from MIA contributors to the open source projects I work on. Yes, I could have been successful if our projects were Python or Ruby based, but as productive? No way.
“developing web applications is still ridiculously cumbersome on Java.”
True. Java webflow sucks ass. That’s why I’ve avoided UI web programming like the plague. It was a *HUGE* backward step in productivity compared to the VB, PB, and even V C++ days. But back end code? Now that we finally have some stable frameworks that take advantage of annotations, its crazy easy and its only going to get better. As projects like JBoss, Seam, Guice, and Spring mature along with the JDK itself, my bet is that we get closer and closer to the zeroturnaround development.
“Remember: refactoring has nothing to do with the type system, but some automated refactoring does.”
NO, but a type system sure makes it easier on you and your IDE to do the refactorings. As an emacs guy a type system has even more importance as to find all the usages of a type, your only option is grep.
“And let’s be realistic – Java devs always end up running the code through a debugger anyway ”
debuggers are for pussies
November 9th, 2007 at 5:14 am (#)
REST+XML has been held back for a number of years due to the lousy state of XML support in languages. New development in javascript and C# will lower the barrier that there is no real reason to use RPC as an interface.
November 13th, 2007 at 11:58 am (#)
There’s no advocate like a convert. :-)
November 25th, 2007 at 10:18 pm (#)
i just laugh about the guys that still need to use static languages (note, i do so too, right now with c#)
my real seller however was ruby. I accept python too, as a friend to bring down both perl and php and keep languages that also look promising like haskell at bay (i think ruby and python are, despite HUGE different philosophies, quite similar to each other)
But on the other hand, WHAT MATTERS MOST
are KILLER APPS
these can be UGLY and HORRIBLE but without any other project to BEAT THESE APPS down, they will continue to be here…
November 26th, 2007 at 3:06 pm (#)
“On the dynamic language front, the worst code I have seen in my career has always, always, always been in compiled imperative languages, most often Java and C++.”
The same could be said for any language. The quality of code is not measured in the language, but by the creator.
“I would much rather let an average developer loose with a dynamic language”
Why would you hire an average developer?
November 26th, 2007 at 3:17 pm (#)
Mathgladiator:
Yes, quality depends somewhat on the creator, but it depends on the language as well. If you’re driving a normal Ford Pinto, you’re unlikely to win the Indy 500 against other drivers using normal race cars, even if you’re the best driver in the world. My point was that on average, people seem more prone to error in those languages.
Now that’s a great question! I personally would try to avoid such hiring, but I’ve been told by some real live software development managers (with whom I do not work, thankfully) that they cut costs by all means necessary, including hiring average or even below-average developers, which they then hope to compensate for by restricting development to easy “mainstream” languages surrounded by lots of tools. Personally, I seriously doubt anything good could come of that approach.
November 27th, 2007 at 9:58 am (#)
@Bill Burke: “True. Java webflow sucks ass. That’s why I’ve avoided UI web programming like the plague.”
Isn’t this a vicious circle? If people that HATE writing web applications are the ones writing frameworks, won’t they keep sucking?
November 27th, 2007 at 10:18 am (#)
[…] have no recollection of how I ended up here: On the REST front, if you’re claiming that it’s harder than the alternatives, to me that’s […]
December 5th, 2007 at 2:46 pm (#)
[…] You can never eliminate the programmer, you can only change the language. Instead of trying to automate programming, maybe you should empower it. […]
February 23rd, 2008 at 3:47 am (#)
Who the hell wants to be average and who the hell wants average on there team anyway unless its to make the tea. Average just spends all its life complaining that the world expects to much from it. If average cant do the job then it should go and be a graphic designer or something. im royally sick of average.