Service/Relational Mapper

Need a little help here...

I've received a few client requests for what I call, "service/relational mappers". In essence it is the same as object relational mapping but with web services. Here are the typical requirements:
1. Be able to both send and receive WS-I compliant web service messages
2. Ability to visually map an in-coming or out-going web service messages to a relational database schema (insert, update, delete, select)
3. "service oriented cron" - ability to wake up every so often, check the database and turn the new records into SOAP messages for outbound delivery.
4. Maintain a log of the activity
5. Send appropriate exceptions / errors

The key here is simple service to relational database mapping (DBMS vendor neutral).
If you've used a product or products that you are happy with, please send me a note:
jschneider@momentumsi.com

He's got Google-sized balls!

Holy ding-a-lings!

Adam decided to ring-a-ling his jinglers on Christmas day!

Encapsulation, Polymorphism, Inheritance

Greg Vaughn, weighs-in on the "four tenets":

"I mean no offense to those who are trying to nail down the fundamentals of what makes an architecture “service oriented”. For some it is a very handy way to organize their own knowledge. And there’s certainly a need to have laws for any developers on a project who have not reached the required maturity and experience levels, but in my opinion these should be more project specific rather than broad architectural category laws."

Here is the problem. Greg is one of the smartest developers I've ever met. For him, the concepts of SOA are obvious. He's a top 1% guy. The problem is the other 99%.

Remember, "Encapsulation, Polymorphism, Inheritance"? These were the basic ideas that the OO guys were trying to teach. They may not have hit the nail on the head but they did provide guidelines for developers.

Laws or rules aren't childish. They are the nourishment that leads us out of child-like naiveness.

Sun Launches the first pure “service oriented operating system”

Sun Launches the first pure “service oriented operating system”

PRESS RELEASE
On January 1st of 2006, Sun Microsystems is scheduled to release the world’s first service oriented operating system called SunStorm. The operating systems will be released under an open sourced license agreement, free to the public.

SunStorm is based around a new model of computing known as “web services”. Early versions of operating systems were primarily developed in structured programming languages like C and were often exposed to applications using ‘objects’. According to Sun, the internals of the operating system will continue to use these highly efficient techniques for things like disk operations and memory management, but the top layer of the system will now be completely service oriented.

SunStorm also has incorporated most of the capabilities of their last generation computing platform known as “J2EE”. New web services are provided inside the operating system to provide messaging, management, database connectivity and other “enterprise grade” functions. When asked about potential performance concerns, Sun representatives showcased their dynamic web service optimization techniques. Similar in concept to the “HotSpot compiler” release years earlier, SunStorm has the ability to dynamically determine the most performant mechanism to enable services to interact and in certain cases to recompile themselves into a single service which is later cached or deleted.

Sun is also rolling out their new tagline, “The Network is the Application”. This tagline replaces the old line of “The Network is the Computer”. With the launch of SunStorm, Sun is initiating a new era of loosely coupled, network computing for the enterprise.

Interestingly, Sun has not decided to enter into the development market for this space. Having learned from earlier mistakes made in their iPlanet group, Sun is now focusing on building out the operating system and supporting it for their enterprise customers. New to the Sun organization will be an emphasis on professional services.

Apparently, SunStorm has been years in the work only recently emerging from it’s stealth status. As part of this effort, Sun has end-of-lifed both it’s Jini and JXTA technologies. The J2EE technologies have been placed in a ‘mature’ status and will continue to be supported.

================================================================
This press release is 100% false. I completely made it up. However, it is really sad that it is 100% false.

The Four Tenets of SOA

As we near this Christmas holiday, it seems only appropriate to throw the bible into my opponent's face :-)

Consider the following:
'You shall love your neighbor as yourself.' - (can, shall, may, etc.)
'Don't bang your neighbor's wife' - (don't, shall not, etc.)
Subtle difference? I don't think so. Imagine the first "rule" without the second "rule". Talk about a HUGE clarification!

You see, leaving room for interpretation leads to problems. That's why the Four Tenets of SOA are utterly useless (even as philosophies). Don't do X; Don't do Y. Don't do Z. Simple isn't it? Constraints. Ok, hopefully I've established the difference between philosophy and constraints. That said, let's go back and take a look at the four tenets that Don Box provides us peons to guide our professional existence for the next decade:

Boundaries are Explicit
Point: Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation.
Response: This is the "distribute OO was a failure" rule. Rather than saying
A. Distributed OO was a failure because Microsoft screwed the CORBA guys and fragmented the space.
B. Distributed OO was a failure because CORBA was a pain in the ass and it was competing in a rough market (BPR with 2-tier RAD and RAD groupware).
C. Distributed objects was a failure because we didn't adequately factor out the non-functional concerns and network programming was in its infancy.

No, rather than saying A,B or C - Microsoft says "Distributed objects failed because programmers were so stupid that they couldn't tell the difference between local and remote calls. Their solution was simple. Make remote calls a pain in the ass (oops, I mean ... make them non-primitives.) Yes, create some sort of language add-on (an API?) that mandates a mapping layer between the internal type system and the 'generic' type system - that'll teach those stupid programmers to realize when it's local and when it's remote! Oh, but let's also build runtime optimization of local service calls into the Indigo stack just to mess with them!!!
Sweet.

In case you're wondering, the constraint for 'boundaries are explicit' is this:
"language designers and extension builders SHALL NOT create a programming extension or primitive that enables service calls to be or appear to be first order primitives." (This might be the rule, but I can't wait to break it!!!)

The second part of 'boundaries are explicit' is: "The notion that boundaries are explicit applies not only to inter-service communication but also to inter-developer communication...." For the life of me, I can't find a rule, philosophy or anything actionable in this one.

==================================================================

Services are Autonomous
This is a case of bad terminology and description (I believe). If I understand Box correctly, his real intent was this line, "services are almost always deployed atomically". Whew... that was hard. Let's cut to the end and find the constraints:
1. Newly versioned services MUST BE deployed in a manner that DOES NOT negatively affect the prior version.

The second part of 'services are autonomous' states: "Service-orientation encourages a model that increases ubiquity by reducing the complexity of service interactions."
Here, Box takes jumps out of the versioning issue and lands in the 'loosely coupled' issue... although, in perhaps the most ambiguous manner possible. All I can offer is that if you want loose coupling to be a tenet, than you have to make an effort to create an index on loose coupling . Saying that 'leaking intentions is bad' doesn't quite do it for me.

==================================================================
Services share schema and contract, not class

Box states, "Rather, services interact based solely on schemas (for structures) and contracts (for behaviors)." Schema and contract only? Crap, we have to kill SOAP with Attachments and DIME ;-)

I've stated before... I don't have a good answer for this one... the problems with this tenet are clear: sharing binary information, passing predicates, bypassing the schemas with optimizations and sharing instructions other than classes like DSL's and metalogic.
==================================================================

Service compatibility is determined based on policy

Again, I agree.
Rule: Service consumer MUST be able to determine the capabilities of a provider via a publicly accessible, standard policy.
==================================================================

Now, I know that Don Box didn't mean for these to be rules... just some high level philosophies. However, if you watch the MS video, you'll see the MS-band-of-jokers butcher his material.

Back to my original point; we need simple rules or someone might accidentally misunderstand things like 'love your neighbor'. It's all about constraints. :-)

"Schneider... you ignorant slut!"

Like every other morning... I woke up, made coffee and read my emails. One email caught my attention; it was titled, "Schneider... you ignorant slut". Clearly, this one deserved immediate attention :-) Yes, it was another 'anonymous' WS-CTO laughing at me for getting BlogSlapped by Jef Newsome.

Well, I thought I was going to make it all the way through 2004 without a good whipping. Thank God for people like Jef. Block, block, kick, block, punch... here we go...
======================
Jeff said: "All exchanges of data, metadata, logic or other binary asset MUST BE exchanged through a contract. Runtime changes to code dependencies are NOT allowed."

Jef commented: "Boundaries are explicit" is about the fact that you can't see what kind of underpants I am wearing."

Agreed. To me a 'contract' is a combination of the interface and the policies. The contract should black box both functional and non-functional requirements. The failure to block box non-functional requirements was the limiting force of components and the primary driver to move to services. However, neither of us answered the "boundaries are explicit, except for predicates" problem. (Not that I expect us to, but people like MS can't just punt on it either.)
=======================
Jef commented: "I have absolutely no idea what "Runtime changes to code dependencies are NOT allowed" has to do with explicitness of boundaries. In my estimation, nothing. I can think of a number of ways that runtime code dependencies could change in the context of a service oriented system, and everything would be just fine."

It was an example of a constraint - perhaps a bad example, but an example of a constraint. My point was that we need a constraint based description language for the architecture; I wasn't really trying to write the architectural constraint language. The constraints will be probably number in the hundreds, not four or six. That was my real point.
======================
Jef says, "I believe that services *should* be autonomous, that their implementation dependency on other services is an implementation detail (because boundaries are explicit), and part of the responsibility of an autonomous service is to gracefully degrade service when its dependencies have availability or other issues."

It sounds like you believe that a service is autonomous as long as the 'boundaries are explicit' rule is enforced; if that is the case, then this is a duplicate rule and we should remove it. I'm game. I've been game - it sends the wrong message.

I firmly believe that enterprises will eventually be required to migrate to Model 4 services. Yes, their boundaries will be explicit, but their lack of sovereignty or self governance will be more significant. Services will NOT be viewed as single individual entities, but as ecosystems of contracts that need to migrate together.

As we move from noun-first programming to verb-first, we find ourselves dealing with "cross-cutting subjects" (not concerns) that have to be dealt with as a group. What I have found is that the 'verb' holds still while everything else changes (noun, adjective, predicate, context). This isn't to say that you can't have autonomous services; you can. However, I don't believe this approach will work from an economic perspective. I'm probably talking more about the 'development-time' aspect of autonomy than the runtime aspect; however after re-reading the MS description, I have no idea what they were talking about. Their description is largely just a bunch of mumbo-jumbo. They mixed a half dozen concepts under this one tenet that largely don't even relate to each other. If you want to turn "services are autonomous" into a set of constraints, I'd love to see them.

======================
Jef: And I trust he can explain what he means in a way that someone as simple as me can understand. I hope he does, because I don't get it.

My goal isn't to explain it in the simplest fashion, but in the most precise fashion. I want to take the current state of WS-MumboJumbo and bring clarity via a set of constraints. A tenet should be a set of constraints rolled together to satisfy a high-level architectural requirement. Walking into the next paradigm of computing without constraint based design practices is silly, insulting and financially dangerous.

Service Creation Styles

At Momentum SI, we've been working on an EA framework for producing reference architectures and candidate architectures. This work has an emphasis on SOA but also considers many of the other architectural remedies.

One aspect of an architecture is the style that is used to create/execute a service. Although we've seen many variations, here is our high level categorization scheme:













Note that this doesn't cover registries, configuration management, version control or other essential part of the architecture. Those remedies are covered in other areas of our SOA reference architectures.

The Death of Software?

Paul Brown had commented on a note from Eric Newcomer on the "death of software":

"The most significant inventions are over. Some would call this the "death of software." But only as we know it. Software will continue. But we are not likely to have any new languages, or see any significant new inventions. Twenty years ago no one knew what a database was, or middleware, or Java. But now I think innovation like that has stopped because IT doesn't need it any more."

Oh me, oh my. I've been there before - when you just run out of steam and everything looks grey; progress halts and the future dims. The good news is that history tells us that this is dead wrong.

Where to begin? Software remains in a pre-natal stage (not even infancy). I still think is hysterical that I type on a keyboard, look at a fixed monitor, and have to tell the computer what I want it to do. I'm amazed that I am so much smarter than my computer - this is just silly. I hate the fact that the state of artificial imagination is at ground zero, that we don't have digital metaphors and that a computer can't reverse engineer strategy. I think it is funny that we continue to use silicon at the computing substrate, that our programs are explicitly programmed and that they are not self-improving.

Maybe I'm just a kid at heart. Maybe I'm too stupid to know the obstacles. Either way, I am thankful. May I continue to be cursed with creationary optimism.

And may I offer an ounce of optimism to those of you who wore black pants and white shirts today. When your desk is scattered with employee status reports, when your walls speak of posters of UML and Java libraries from 5 years ago, it is time for a change. Not a little change, but a big change. Web services, orchestration, intermediaries, blah, blah, blah - these are the incremental improvements decades in the work. Refill your mind with childlike optimism. Imagine. Invent. Destroy. Laugh. Repeat.

Stuck in the Middle

This is from a slide deck that was put together in 1997:
The [I] box stands for 'intermediary'.


See: http://www.almaden.ibm.com/cs/wbi/Publications.html

Intermediary-based programming is still an art. Everyone wants to talk about services... service oriented this, service oriented that... but no one want to talk about "intermediary oriented". I guess it isn't very sexy. I've challenged the SO group to discuss the NFR's of an intermediary; should be interesting. I've also had some interesting discussions on the categories of intermediaries, including stateless, stateful, 'context oriented', header-only processing, payload processing and more.

I have a feeling that a significant focus of 2005 will be on the intermediary programming model. We'll see great discussions on IOD (design) and IOA (architecture) and refactoring portions of fat services into bumps on the network.

Charcoal Services

If a black boxed service sits in a grey container, what is it?

The Financial CIO

A great quote from a recent Sterling-Hoffman newsletter:

Angel Mehta: Why do you think the climate is so difficult for enterprise software companies, even with the economy having recovered?

Stu Schuster: For as long as I can remember, the process of growing a company has been the Geoffrey Moore, ‘Crossing the Chasm’ approach. When taking technology to market, find the innovators first who buy the technology, and then you go through the early adopters and late adopters, etc. This process, I think, all good high-tech marketers understood.

Throughout my entire career, there were always people who wanted to do something for their organizations that they believed would be a leap forward and would make a name for themselves. Call these the entrepreneurial Chief Information Officer… they were at companies like FedEx or Wal-Mart. They would try to do something with technology that would give the entire company a competitive edge.

After the technology crash, the focus became so oriented around cost-cutting that the entrepreneurial CIO was replaced with a financial CIO. There are so few entrepreneurial CIO’s out there that today everything has to be easy to implement, available by the drink, with very short time to ROI. The innovative buyer, the entrepreneurial CIO has been terminated out of the industry.

Eventually, it’ll cycle again because people will eventually feel that they’ll need a competitive advantage and will look to technology to do that. But today, it’s brutal. It’s amazing how hard it is to find people who are willing to take a chance on a new company or new technology. Fear dominates every IT department. As a result, growing an early stage software company is just harder than ever.

That’s why I place so much more emphasis on the people-side of the equation these days. It doesn’t matter how great the technology is – if the right people aren’t in place, you’ll never convince customers to take a chance.

The Role of the Application Servers in SOA

I've been using the term "service containers" to refer to the sandbox that services live in. However, in J2EE land, there isn't a 1:1 relationship between the 'software server' and the 'container'. You've got the 'servlet container', 'ejb container', etc. The Application Server is really an all-in-one - super-duper server. It slices, it dices, it does transactions, messages, web pages - you name it - it does it.

App servers can host web services. They can also host platform specific services (RMI, EJB). The services provided can be governed via the container and can leverage additional functionality via components and class libraries (most of the J2EE API's).

Just to restate - build services on top of components that are a combination of classes which sit inside of a governed container while making calls to libraries, some of which are runtime generators, perhaps being fed metadata from model-driven tooling. Agreed?

Ok. Now we need to move into "specialized servers", like identity servers, orchestration servers, etc. Many of these servers were built from the ground up as stand-alone servers, while other implementations were built to sit on top of application servers (e.g., a bpel implementation that runs in JBoss). In the J2EE world, specialized servers are really just new API's governed by the JSR process to encourage yet another container or library to be added to the J2EE server. In some ways, J2EE has become a dumping ground for 'bright ideas'. Hence the movement to 'lightweight containers'.

Now, let's ponder...
Question #1: Would you describe the application server as a "Rapid Service Development" (RSD) environment?

Question #2: To what extent should specialized services reside in generic containers?

Question #3: Should the server be 'pre-glued' together (mandate the download all 55MB of libraries) or should it be more loosely coupled and segmented?

Question #4: What are the necessary cross-cutting concerns that should be considered part of the server or container?

Question #5: What functionality is not a cross-cutting concern, but rather a specialized function?

Question #6: Would you consider the app server to be a "Contract First Container"?

The application server has evolved to meet the needs of the masses. It was designed to be an all-purpose server. Can a developer build SOA based applications in it? Sure. It is more work than it needs to be? Absolutely. And the big question... Do I believe that Rapid Service Development to Contract-First Containers is the way of the future? Yes I do.

Application servers will continue to be used when the architect chooses to use a platform specific component-based development model. Here, the focus is on 'building the application' (hence, "application server") versus building the services.

There are some real challenges that app servers will face in the SOA environment. We can be sure that BEA and IBM will be morphing their offerings to make them more acceptable to the SOA crowd. However, I fear that they will leave all of the garbage behind as well.

It's real simple - if your platform fails the loosely coupled test then so will your business applications that sit on top of them.

Remember: "Rapid Service Development to Contract-First Containers".

MS Patterns & Practices are Shallow

The Microsoft Patterns & Practices group released a set of SOA principles. I was eager to see what the brains came up with but found myself beyond disappointed; I found myself actually in awe of their incompetence. Perhaps these are tough words but I'm so extremely disappointed in the group-think that is currently going on in the SOA world that I feel compelled to point it out.

First, the show can be found here:
http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?culture=en-US&EventID=1032262538&CountryCode=US

MS stated four primary tenets to SOA. My first thought was "holy smokes, they got it down to four?!!!" Well, here they are:
1. Boundaries are explicit
2. Services are autonomous
3. Services share schema and contract, not class
4. Service compatibility is based on policy

Boundaries are explicit - yep, hard to argue. But what does this actually say? How about this instead: "All exchanges of data, metadata, logic or other binary asset MUST BE exchanged through a contract. Runtime changes to code dependencies are NOT allowed."

Services are autonomous - Most people know that I firmly believe that "services are not autonomous", they are synergistic. Services work together in a 'service ecosystem'. Services will influence each others metadata. In no way are services autonomous.

Services share schema and contract, not class - I believe the point is that you want to share data using self describing, self validating, extensible schema systems that are ubiquitously deployed. That's cool - use an abstract typing system.

Service compatibility is based on policy - Yes. Agreed.

The patterns group then identified some "anti-patterns" or things not to do:
1. Avoid CRUDy interfaces. Don't use the CRUD pattern.
Guys - this is so wrong I felt sorry for them. After listening to them babble on this for a while all credibility was lost. I think what they meant to say is "Use CRUD, but don't violate the age-old wisdom around fully encapsulating and abstracting the persistence mechanisms. "

The second anti-pattern was avoid "loosely goosey" interfaces. The example they used was something like "runQuery(qry)". This one is tougher. If you really don't know what query combination needs to be run at design time then... you pretty much need something like "runQuery(qry)". This gets into a real interesting space that I wish MS would have talked about. Is it ok to pass XPath statement? Is it ok to pass an XQuery command? What is the right way to solve the runtime predicate problem? Surely the answer isn't "don't solve it". What about the surrogate key problem - where the only primary key is vendor specific key?

The list goes on and on... MS has the brains to do this stuff right. I wish they would ask some of their smart people to help the patterns & practice group get this stuff right.

Languages are Less Important

"Languages are Less Important" is one of the simple philosophies in Awesome(tm):

Traditionally, the choice of programming languages in an I.T. department was a near-religous decision. One reason was that the language tended to slither its way from one application to the next. Knowing this, managers would be very careful about allowing a new language into a development shop. In addition, reuse was considered a language specific mining activity (no one tried to pull Java code out of one application and put it in a Perl application). On top of this was the obvious fact that all of the enterprise API's were language specific, including J2EE and .Net.

Enterprises that implement Contract First Design and believe that Platforms are Contract Driven are much less dependent on specific languages or language specific libraries. Now, with this said - languages are still important for some real good reasons including: Developers skilled in one language shouldn't have to learn a new language for every different service they run into. Most I.T. shops already have large investments in platform specific code bases that they will want to leverage.

Although languages are less important in a service oriented world, they are still important. As services become more popular it will be common to find languages which are more friendly towards service development. This may be based on their ability to work with the Web Service standards, to expose themselves as a service or to invoke a service.

Well, it seems as though Peter Yared understood this some time back. Unlike the scores of startups that re-create the same damn product using all the same damn technologies... ActiveGrid made a hard left turn. The company appears to have come to the obvious realizations that no one else seems to have the guts to admit:
1. J2EE is a bloated set of deprecated patches.
2. Static Java objects and dynamic XML collide
3. Contract first programming make "languages less important"
4. If you're in an SO world, you might as well pick an SO friendly language
5. J2EE was designed to scale across a VM that sits on a multi-CPU box
6. Oh, Sun owns Java and J2EE, and what do you know... they sell multi-CPU boxes
7. Rip & replace Intel/AMD/Linux boxes are fast, cheap and reliable
8. LAMP is here to stay.
9. The growth rate of transactions inside an enterprise is significant.
10. The last-gen computing method will hit an 'economic breaking point'.

ActiveGrid is a "market starter"; the door has just been opened to commoditize the OS, the language and the vm while kicking big boxes in the groin and enabling service developers to be more productive while offering a lower cost of computing. Interesting? Uh, yes.

"Design by Contract" or "Contract First"?

I'm trying to clean up some terminology that I've been using in the Awesome(tm) method. The latest is the term "design by contract(tm)". This is a term that Bertrand Meyer coined (and trademarked). He used it in the context of object oriented programming where the focus was on "pre- and post-conditions and class invariants".

Don Box, Christian Weyer and others have favored the term "contract first" to describe a similar concept more suitable to web services. Here the focus has been on giving a service the ability to advertise:
- Type definitions
- Message definitions
- Operations (inbound and outbound)
- Message Exchange Patterns (e.g., request/response)
- Interfaces (groups of operations)
- Bindings

However, it is apparent that the contract goes beyond the basic wsdl stuff. It must also include:
- Pre-conditions / post-condition assertions
- Non-functional policies (security, reliability, transactional integrity)
- Hosted service level agreements (availability, latency)
- Synchronicity capabilities (asynch, synch or bi-synch)

Eventually the contracts will include stuff like:
- Semantic ontology (or other semantic broker)
- Costing /recharge functions
- Estimates to run service (think big-O)
- Preferred translator service (don't understand SOAP 1.5, but do understand SOAP 1.3)
- Code mobility agreements
- Service-to-service automated recompilation agreements (HotSpot for SO)
- Service dependencies (other services and resources)
(many of these will show up as standardized interfaces implemented by a service container)

I guess what I'm saying is that "design by contract" doesn't mean the same thing that we are talking about in the service oriented world. Hence, "Contract First" it is!


-----
Here's one to ponder... what pieces of the contract should be in a document format (like WSDL) versus in an interface format (like ws-mex). Or should they be in both? Why doesn't WSDL have a WSDL interface: getOperations()?? Is the ws-mex, getMetadata() the right way to go? Or should you look at it as an enterprise vocabulary problem? "GET" + "OPERATION" (think ws-transfer). Or is it a semantic ontology problem where you have "GET" and a pre-condition stating you can only get things that 'are a kind of' "METADATA"... Well, doesn't really matter - looks like we will have every combination possible. :-)

One of my favorites...

One of my favorite posts:
http://schneider.blogspot.com/2002_12_01_schneider_archive.html#90283350

(still waiting).

Service Oriented Methodology

In 1998, a few of us at Momentum (James Higginbotham, Mark Spitzer, Ed French) wrote a software development method that we called Reality(tm). The method borrowed from best practices of Booch, Rumbaugh, Jacobson, Coad-Yourdan and Schaler-Mellor. Later we incorporated UML and concepts from Extreme Programming. We also threw in platform specific (Java & .Net) best practice guides and reference architectures. The Reality method has served us on over 100 engagements. I'll admit that we haven't always follow it to the letter of the law, but the intent seems to find its way into all of the projects.

However, at the heart of Reality is an "object oriented" development cycle. We were so heavily influenced by the "object people" that many of the mandatory artifacts and diagrams are OO in nature. Anyone that remotely knows me, knows that for the last half-decade I've been exclusively focused on service oriented computing. And for the last half-decade I've been waiting for a "service oriented" method.

Well - we decided to bite the bullet and start knocking one out. We are currently calling the skunk-works project "Awesome!(tm)" (Advanced Web Service Oriented Method) but my marketing team is telling me, "the name has got to go". I'm open.

The Awesome! method revisits a handful of concepts:
- Service centric diagrams & modeling notation
- Service centric artifacts
- Service network artifacts
- "contract first" development lifecycle
- Service level agreements as a first order concept
- Business process 'binds' to service networks

We also have started a repository of best practices - however these seem to have a half-life of about 10 months.

In 2005, we expect a half dozen SO-methods to come out - we know, this isn't exactly a new concept. As others come out, we plan on learning and upgrading our own. We figure it will take about 3 or 4 years before the SO-Methods come together with a set of base features and common notation. We also believe that people will want more than the "Microsoft Method" - and that choice is good.

Unlike our Reality method, Awesome! will be made publicly available. We aren't quite ready to share it with the public yet but are eliciting ideas on what people want in an SO-Method (email me). After the straw-man is together, we'll drop it out for public review and begin the upgrade cycle. Let me know if you have an interest in being an active contributor.

Methods are repeatable processes. They enable companies to reduce risk around projects and to leverage resources by assigning them to well-defined tasks. As service oriented styles move into the enterprise, the use of SO-Methods to complement your existing methods (RUP/UML, etc.) will be common. We need to move SO from an art to a science (or at least a craft) - and soon.

On Grand Venture Fund

Take a look at the application form for receiving venture funding from the On Demand Venture Fund:



Step 1 - get a Grand Central account and build an application :-)

How subtle is that?

The Importance of the Registry

Jeff Tonkel of Infravio recently pointed out the importance of the registry in a service oriented architecture. I'd like to second that thought. While I'm at it, I'll go so far as to state my opinion on who is leading in the space.

For starters, you have to look at BlueTitan. These guys get one real simple fact - you don't implement a registry for the sake of having a registry. Many vendors wrote a registry and encouraged their customers to dump crap in it. Bad idea. BlueTitan took the other approach - they started with the 'ilities' and worked backwards into the metadata that needed to be captured to manage a service network. Their registry is designed to capture only 'usable' metadata (actionable information); don't store anything that can't be used. Also they focused on the needs of enterprise grade service networks, the people with the really hard problems. I'm thrilled to see BlueTitan recognize that services are often just 'bumps on the network', that policies are 'service oriented aspects' and that the metadata and registry are the heart of making it functionally scalable. Most networks grow exponentially for some period of time; BlueTitan makes it possible to manage high growth networks on a linear cost scale.

After BlueTitan, you have to look at Systinet. Although these guys started with a strong UDDI slant, I believe they not only understand the limitations of UDDI, but are well on their way to overcoming the issues. I anticipate that Systinet will also have strong 'developer-side' integration into the registry (think reusable assets).

The registry is quickly turning into the 'metadata warehouse' - it is the point where a 'metadata driven architecture' and the 'service oriented architecture' merge. Architects are slowly figuring out that an important aspect of the SOA is the ability to factor the ilities out of applications. The registry is key to facilitating this movement.

The AToM Splits Definition

The AToM, Anne Thomas Manes, took the time to mark up some definitions of SOA. I love it. Four things I'd change:
1. Services don't have to be 'autonomous'
2. Services don't have to be used for 'business'
3. In many cases services shouldn't be designed for 'reuse'
4. In many cases services won't be 'discovered'
Thus, try not to use those four words in the definition.

Posted on the soa news group...
===============================================
SearchWebServices asked this group of people to provide 50 word descriptions
of SOA. It's very hard to describe SOA in just 50 words (and obviously some
folks ignored the 50 word limit). You simply can't describe the tenets of
SOA so succinctly. Burton Group recently published a 22 page report
describing SOA, which I think does a much better job than any of these short
descriptions. I could easily write an entire book on the topic.

In my description, I endeavored to convey two of what I consider to be the
three major tenets of SOA: application factoring and abstraction, although I
was not able to convey the multiple aspects of these two tenets. I was also
not able to convey the third tenet: the requirement for standard protocols
for description, discovery, and communication.

Please see my comments below bracketed by ::atm::::/atm::.

-----Original Message-----
From: Gervas Douglas [mailto:gervasdouglas@yahoo.co.uk]
Sent: Thursday, October 21, 2004 12:53 PM
To: service-orientated-architecture@yahoogroups.com
Subject: [service-orientated-architecture] Re: Definition of SOA




Here are some definitions provided by SearchWebServices:

::::-------------------------------------------------------------------
-------------
"SOA is a framework enabling application functionality to be
provided, discovered and consumed as re-usable Web Services sets.
While Web Services do not equal SOA, it's one of the enabling
standards. SOA abstracts complexity and implementation details,
making it an ideal architectural mindset to leverage functionality
trapped within mainframe/midrange systems."
Scott Rosenbloom is chief strategist with WRQ Inc.

::atm:: SOA is an architecture -- not a framework. It is a style of design --
not a tool. On the other hand, "web services" is a framework -- a set of
tools (middleware) to help you implement applications and systems that
adhere to SOA design principles. (Although note that building applications
using the web services framework does not guarantee SOA.) SOA requires
standard protocols for description, discovery, and communication. The web
services framework standards define these standard protocols --
XMLSchema/WSDL, UDDI, and SOAP respectively, at a fundamental level,
although these core standards are not sufficient to support all SOA
requirements (secure, reliable, transacted applications supporting diverse
message exchange patterns) -- hence the ongoing effort to develop the WS-*
standards.

From my perspective, the best way to describe SOA is in terms of design
principles. Abstraction is one of the core tenets of SOA -- abstraction
between the interface and the implementation, and abstraction between
infrastructure and business logic. WSDL facilitates the first, and SOAP
facilitates the second. Another core tenet is proper application
functionality factoring. What granularity should a service have to best
enable reusability? I think this is the most challenging aspect of SOA. My
rule of thumb is that a service should represent a business task. But it's
hard to quantify this rule. ::/atm::
---------------------------------------------------------------------
-----------
"Secure, integrated delivery of IT solutions meeting business
requirements. Solutions must implement, optimize and guide business
process execution by combining the functionality of separate,
discreet, reusable services. SOA moves away from complex application
development, promoting a focus on standardizing interfaces between
atomic service components with centralized management and
distributed implementation."
Dave Morris, I.T. Security Lead TransAlta Corp.

::atm:: For the most part, I don't think this definition helps explain what
SOA means, but I like that fact that he associates SOA with "centralized
management and distributed implementation". As I said in my response to the
first definition, one of the core tenets of SOA is the requirement for
abstraction between infrastructure and business logic. Via the separation of
the SOAP Header from the SOAP Body, and via the requirements of the SOAP
processing model, SOAP enables the clean separation of infrastructure from
business logic. It permits developers to externalize most, if not all,
infrastructure functionality to the middleware. This feature enables
centralized management of the distributed implementation. ::/atm::
---------------------------------------------------------------------
-----------
"The SOA models the business as a collection of self-contained
services that are available across the enterprise that can be evoked
through standard protocols both internally and externally."
Dave McComb, president, Semantic Arts

::atm:: An accurate description, but it doesn't give you any real information
to work with. ::/atm::
---------------------------------------------------------------------
-----------
"Service Oriented Architecture is nothing but business oriented
architecture, which allows the flexibility of business applications,
to become independent but collaborative, while providing their
services. The applications under this architecture are both 'client'
and 'server' at the same time with freely available services."
Satheesan Kunnel, USWWI

::atm:: While it's definitely a brilliant idea to align IT with business, I
don't think this definition helps you figure out how to do so. ::/atm::
---------------------------------------------------------------------
-----------
"A service oriented architecture is an approach to design and
integrate software in a modular method where each module is
precisely a 'loosely coupled service' that is accessible over a
network and has the capability of being dynamically integrated with
other services at run time. A service must present a standard
Interface (be it WSDL today) for its functionality and invocation
methods while the real implementation of the service is not a
concern of an SOA."
Rajesh Dawar

::atm:: I like this definition, although I'd like to see a little more
definition of what a "precisely" loosely coupled service is. From my
perspective, the standard communication protocol has equal importance to the
standard description protocol, although this definition obviously gives more
weight to the description language. I also view XML Schema as equal weight
to WSDL. Again note that SOA does not require XML Schema, WSDL, and SOAP
specifically, but it does require that applications rely on a set of
standard protocols. When using web services to implement SOA, those standard
protocols are XML Schema, WSDL, and SOAP. ::/atm::
---------------------------------------------------------------------
-----------
"Services provide something of value to those who know how to
request and consume them, without having to know how to produce that
value. SOA is an approach to building software applications as
collections of autonomous services that interact without regard to
each other's platform, data structures, or internal algorithms."
Michael Champion, R&D specialist, Software AG

::atm:: Excellent short description! ::/atm::
---------------------------------------------------------------------
-----------
"A pattern of design, development, deployment, and management of (a)
applications and (b) software infrastructure and frameworks in
which:

Applications are organized into business units of work (services)
that are (typically) network accessible
Service interface definitions are first-class development artifacts
Quality of service (QoS) characteristics (security, transactions,
performance, etc.) are explicitly identified at design time
Software infrastructure takes active responsibility for managing QoS
and enforcing policy for service access and execution
Services and their metadata are cataloged in a repository
Protocols and structures within the architecture are, optionally,
based on industry standards (e.g., the emerging SOAP stack of
standards)
Randy Heffner, vice president, Forrester Research Inc.

::atm:: I have to question Randy's requirement that QoS characteristics must
be explicitly identified at design time. Perhaps he's talking about
composite application design time? -- the time when you are assembling
services? But even here I disagree. I contend that one of the most brilliant
features of SOAP and the web services framework is that it permits you to
externalize QoS functionality from the application so that it does not need
to be defined at service design time. Instead these requirements can and
should be specified at configuration time. QoS functionality can be
externalized from the application, specified via policies, and enforced
through the middleware. This is a required feature for reusability. The
security, reliability, transaction, and performance requirements of a
service may vary depending on how it is used in a particular composite
application or based on the specific client invoking the service. Rather
than hard-coding this complex functionality in the application, you can
configure the functionality using deployment and configuration wizards --
just tick the box to add security and reliability functionality. This
configuration process should generate a policy description (e.g.,
WS-Policy), which becomes part of the overall metadata/description contract
of the service (consisting of XML Schema, WSDL, WS-Policy, and potentially
other description languages).

Almost no one is using WS-Policy today, and I view it as one of the most
critical holes in the current web services framework. ::/atm::
---------------------------------------------------------------------
-----------
"SOA is a style of design that strives to enable easy integration
and flexible applications. In SOA, application functionality is
designed as shared reusable services. A service is a piece of
application functionality that exposes its functionality through an
abstract interface, which hides the inner workings of the service
implementation."
Anne Thomas Manes, analyst, Burton Group

---------------------------------------------------------------------
-----------
"An SOA is an enterprise-scale architecture (typically spanning
multiple applications within an enterprise or across multiple
enterprises) where the primary structuring element is a service (as
opposed to modules, systems, applications or components).
A service is a set of related business functions that are interacted
with locally or remotely using a message-passing/document-oriented
communication style. A service is composed of (1) a (functional)
service interface and (2) a service implementation that implements
one or more service interfaces and adheres to a certain set of (non-
functional) capabilities. Specific services are defined in terms of
the transport/application/messaging protocol, not in terms of a
specific programming model.
An SOA will typically include technical services to manage metadata
about service interfaces and implementations, service providers and
service consumers; and services for managing and enforcing policies,
access control, security features, and transactions, although all of
these are optional within any specific SOA instance."
Stefan Tilkov, CEO, innoQ

::atm:: Excellent description! (although Stefan obviously exceeded the 50 word
maximum) ::/atm::
---------------------------------------------------------------------
-----------
"Service-oriented architecture is an architectural discipline that
centers on the notion that IT assets are described and exposed as
Services. These Services can then be composed in a loosely-coupled
fashion into higher-level business processes, which providing
business agility in the face of IT heterogeneity."
Ronald Schmelzer, analyst, ZapThink LLC

::atm:: Excellent description! ::/atm::
---------------------------------------------------------------------
-----------
"Service Oriented Architecture (SOA) is an approach to the
development of loosely coupled, protocol-independent distributed
applications composed from well-defined, self-contained software
resources accessible as Services across the extended enterprise in a
standardized way, enhancing re-usability and interoperability."
Ankur Gupta, marketing manager, Fiorano Software Inc.

::atm:: Good description, although hard to parse. ::/atm::

The State of SOA - Quick Summary

Here's a quick summary of what I've seen recently in the SOA space:

1. Styles
A. HTTP / XML-RPC / AddToCart *** xml over http
B. HTTP / Post / Cart/Add *** REST Rules!!
C. AnyTransport / SOAP WS-Hdr / AddToCart *** SOAP Rules!!
D. AnyTransport / SOAP WS-Hdr / POST / Cart *** SOAP with WS-Transfer Rules!!
E. All of the above (we'll use a bump in the network to translate between them)

2. BPEL
A. If you can't put J in front of it than it doesn't exist (J-BPEL). You "J" people are so myopic. No, you "X" people insist on doing things the hard way.
B. "When I say we support BPEL, what I mean is that we support X/Lang."
C. "So, the really powerful part about BPEL is Abstract Processes... What??? Is that why none of the vendors did an implementation???"

3. Vendors
A. Our CTO quit and then the team working on the ESB quit; other than that we're on schedule.
B. Startups: We have a better product!! So what - we have customers, and have already hit 10 million in revenue - go suck an egg.
C. Oh shit, this web service thing is for real. Let's go acquire someone with our devalued stock.

4. Analysts
A. ZapThink said "blah blah blah"... A recent report from ZapThink stated, "blah blah blah" and the market size is going to be "blah blah blah".
B. All others said, " ".

5. Consultants
A. "Yippee!!!"
B. "If we keep changing the specs, perhaps the offshore people won't be able to keep up!"

6. Offshore People
A. "Yippee!!"
B. "If they keep changing the specs, there is no way the customer can afford to keep using the onshore guys!"

7. Grids
A. Who cares about service oriented; it is about 'resource oriented'.
B. IBM: "A grid is a device whereby our consultants bill you on a regular basis; hence the billing is - on demand."

8. Fortune 500
A. "We're going to have a meeting to discus our SO [fill in blank]."
B. "The budget around our SO [fill in blank] looks good for 2005.
C. "Let's tell our big vendors that lost all their people to acquire some of these little guys, then convince the offshore contractors to team up with onshore consultants while partnering with the merged ISV's; I'm sure that will work out..."

All good fun.

What scares you?

Does this Burger King commercial scare you?


It should - it's freaky!!!

Well - this is why people like web services... you see, that Big ugly king is the VENDOR. And that guy that is scared shitless - that is you. So you're wondering why the BIG PLASTIC vendor is in bed with you... right?

People are using web services to decouple them from the BIG PLASTIC guys so that they don't have to wake up with them in the morning. Go ahead, put a little distance between you and the KING. :-)

Wake up with the King?

New Office

Momentum has launched a new Enterprise Architecture practice. This group will also be responsible for all of the Service Oriented offerings (service network design, service adapter for legacy, realtime SO-eventing, management, etc.) It will also be working with many other architectural concerns (metadata, business process, I.T. governance, etc.)

The new practice will be based out of Reston, VA and will service clients globally. With this change, I have also move to the D.C. area. The continued growth of service oriented environments is keeping us very busy. More to come soon.

WS-MetaDataExchange and WS-Transfer

WS-Mex, also known as WS-MetadataExhange, was updated this month.

However, in an odd move the spec does not reflect the WS-Transfer udpates:


Am I missing something? I thought this was the entire point of WS-Transfer? Don't get me wrong, I still think it is absolutely hysterical that the 'Metadata' spec is mostly hard-coded and doesn't use metadata!

I've been thinking about replacing the WS-MetadataExchange spec with my own. Here it goes:

SOS-GetWSDL
1. Append "?WSDL" to the end of a call to a valid port.
2. Expect a WSDL to be returned.
3. The end.

On WS-Opposition

Tim Bray, who has spent the last several months crapping on the WS-* stack has agreed to shut up or put up. Thank God.

If Tim and team are able to create something better than the Service Oriented Standards (S.O.S.), then I'm the first one on board. I'd love to see Sun get active. Their pissing and moaning that they weren't invited to the party is just too old. So, here are my recommendations to S.O.S. V-2:

1. Start with a vocabulary. Write down the words you will need. Don't write them in XML - write them in English (or other non-XML markup).
2. Publish your vocabulary in the simplest manner possible.
3. Ask smart people if the words and domains are correct.
4. Compare the list to existing vocabularies (J2EE, .Net, etc.); identify the delta.
5. Apply the 80/20 rule to the words, cutting as many as possible.
6. Create a constraint based architecture. Never say "You MAY", only "YOU SHALL NOT".
7. Create profiles of constraint architectures (SMB, Enterprise, etc.)
8. Publish the candidate architectures along with the vocabularies.
9. Wait for people to crap on S.O.S. V-2; they will.
10. Defend your ideas, learn from the debate and prepare for V-3.

Good Luck. Jeff

WS-CaveMan

I invented a new game for web service engineers called Cave Man. Here are the rules:
1. The game lasts for an hour.
2. When speaking to each other, all engineers are only allowed to use the following verbs: GET, PUT, DELETE and CREATE.
3. All predicates must be spoken in Latin or a language other than your native tongue.
4. Nouns and adjectives may be used but only if they have been written on a common whiteboard first.

Ok, I’m only joking – but you do realize that this is the game the web service engineers are being told to play with business systems.

What do I think of WS-Transfer? Easy, we have now created a vocabulary of four common words. At this pace, we should catch up to the pre-historic cave man in about 20-25 more years.

Don Box makes an interesting comment. He basically states that using four verbs can be dangerous. And I agree. However, I am of the opinion that the WS-* spec teams are doing a severe disservice to the community by releasing these specifications without also identifying the best practices. In my opinion, it is no longer acceptable to release paradigm changing specifications without also releasing an implementation or best practice guide to go with it.

In regard to the spec itself, I think it is fine. I don't agree with Ole' man Baker that we are just re-inventing HTTP. Pure REST has its place and SOAP has its place. Standardizing verbs is good. I was a bit curious why the group didn't create a 'verb extension' or 'verb introduction' mechanism, but rather just 'hardcoded' a handful of verbs.

IMHO, the creation of four verbs is a good thing. It will make people think about why WS-Transfer is limited, which will lead discussions on why an 'Enterprise Vocabulary' is needed.

Value-Added-Clown (VAC)

A recent IM conversation (names changed to protect the innocent)

----------------
BILL says:
I have a couple of developers working for me and I'm not terribly impressed with their abilities thus far.

BILL says:
I'm working for XYZ Finance Company

Jeff Schneider says:
*use opportunity to learn more about Finance software*

BILL says:
they were wanting to use SWT as their UI technology

Jeff Schneider says:
screw SWT and UI

BILL says:
is there something to gain from learning about finance software?

Jeff Schneider says:
learn finance domain

BILL says:
SWT blows

BILL says:
what is the advantage that I should be looking at with Finance domain?

Jeff Schneider says:
Business domain is more important than changing technology

BILL says:
yeah, I'm figuring that out

BILL says:
from a manager's POV, how should I approach my manager to tell him I don't think my developers are cutting it (if I perceive that)

Jeff Schneider says:
value is a function of cost over output

Jeff Schneider says:
make sure you know what the cost are before determining value

Jeff Schneider says:
(you see output)

BILL says:
you mean cost of bringing in new developers and getting them up to speed?

Jeff Schneider says:
if he's paying $4/hour for current people than you may be giving him bad advice

BILL says:
true

Jeff Schneider says:
don't advise until you know costs

BILL says:
aha

BILL says:
since I don't know their rates, I can't make that call then

BILL says:
right?

Jeff Schneider says:
yep - but feel free to ask

BILL says:
is that something they call tell me?

Jeff Schneider says:
sure

Jeff Schneider says:
you should also ask the manager if he/she wants your input on such items

BILL says:
what other info should factor into that decision?

Jeff Schneider says:
(output, quality, etc)

BILL says:
it has been relayed to me that I should advise on such matters so I assume that is a responsibility for me

BILL says:
but I should assume nothing

Jeff Schneider says:
if mgr doesn't want to tell then tell him what the perceived bill rate of person is "I'm guessing that your paying about $35/hr for Balu - - which seems appropriate..."

BILL says:
ok

BILL says:
you tell me if this would be a warning sign to you

BILL says:
I asked one to write a Java2D demo (as a replacement for a flash module) that was a simple graph that when values were changed, the graph adjusted)

BILL says:
he proceeded to download sample code from Sun and paste it and tried to modify it (this was after he told me he felt very comfy with Java2D)

BILL says:
would that be a red flag?

Jeff Schneider says:
maybe - but not necessarily...

BILL says:
what would your thoughts be?

Jeff Schneider says:
1. I'd assume person lied about expertise in the technology (people always do)

BILL says:
oh yeah, I had to help him get a demo going cuz he couldn't figure out his compiler error (tried to instantiate an anon inner class from a main)

Jeff Schneider says:
2. I'd be more concerned about ability to get up to speed and then get dedicated to knocking it out

BILL says:
yeah, I've already figured out that they lied

Jeff Schneider says:
3. Then concerned about willingness to refactor until the design was correct (on his own dollar)

BILL says:
would you say micromanagement is going to be my best bet at getting this delivered as it should be?

Jeff Schneider says:
no, but tight iterations would be advised

BILL says:
yeah, weekly code reviews a good thing?

BILL says:
I'm already drawing up a standards doc as a guide

Jeff Schneider says:
sure, lots of builds, demos with mgmt, etc

Jeff Schneider says:
Visibility = "Putting glass box around large turd."

BILL says:
hehe

BILL says:
kewl

Jeff Schneider says:
your first duty is to make them succeed though

BILL says:
right

Jeff Schneider says:
more times than not, you'll end up with Bozo's on your team. Great engineers are ones that can turn clowns into productive team members

BILL says:
guess that would be a good test of my ability ?

BILL says:
hehe

BILL says:
how yucky

Jeff Schneider says:
yep - but that's what separates the pack

Jeff Schneider says:
give mgmt visibility while backing clowns
-----------------

So, I really believe the advice that I gave to *Bill*. Any jack ass can throw another incompetent engineer under the car. The manager should have visibility into *suspect* engineers, but until a *suspect* is proven guilty, it is the job of the more senior engineer to make the rest of the team succeed.

The Mob Effect on the CEO

I've been witness to a recent phenomenon. I don't know what to call it, but I'll attempt to describe it to you.

Software companies have been hiring huge teams (Mobs) of programmers in India and other low cost labor supply areas. The purpose for these hires is to 'create more software, faster'.

When the CEO goes to see the Mob they get excited. They see dozens of cheap bodies lined up and ready to work on whatever brilliant idea is floating in the heads of the executive team. They become drunk on this new, cheap power. They get 'beer muscles'. At the sight of all of these cheap bodies they feel like they have their own little army and can take on the world (or at least a larger competitor). And this is where it begins.

Mob Mania has struck more CEO's than I can count. With their new perceived power they begin to do the unthinkable. They build products in spaces where they know very little. They create a 'portfolio of checkmarks'; that is, they build products purely for the sake of rounding out a portfolio and checking off products. Checkmark Portfolios are never about having the best product - they are about having all products. It is almost an acknowledgment that the executive team was too stupid or lazy to do their homework and determine which products really needed to be built and which should have been OEM'd or left out of the portfolio all together.

Checkmark Portfolios lead to an interesting destination. Buyers see dozens of vendors all claiming to have the exact same set of products. They look under the covers and find that the products were designed with minimal depth while maximizing breadth. The amazing similarity of the products being offered by direct competitors leads to a new buying process.

Now, the focus is moving off of: 1. Having the product or 2. Having the rounded portfolio and since the product was built at dirt cheap prices, the focus usually isn't on the price. The focus is on 'trust'. Said another way, the focus is on brand.

Like all products that are easily commoditizable, the focus moves off of product differentiation and to brand awareness and brand identity. However, it is an interesting time to create brands. With the low investment model required to create a Mob, more and more small companies are sprouting up determined to get a piece of the action (many of these companies are now based in Europe or South East Asia).

The room of vendors is crowded and everyone is yelling for attention. The CEO sees this and thinks two things:
1. "It's going to cost some serious money to get my message above the noise level."
2. "Oh shit, what an awful time to have a commoditized product."

Life has taught me that heard mentality pays for the few. I've seen it over and over again. I watched the VC's pull a heard model and now I watch another. The leaders of the heard will win. The followers will be left in the dust. Second and third tier ISV's will be crushed in a battle whose outcome is so obvious that it is almost uninteresting.

Things are busy...

Well, I've been busy.

We've seen our consulting business pick up considerably and the sales pipeline remains very strong. Momentum has been adding a new employee about every 5 days and I believe this will continue through the end of the year.

Most of our projects now have some component of web services. Here is a sampling:
- SOAP gateways to access legacy systems
- business objects being converted to business services; data layers being turned into data services
- laying down foundational ws-infrastructure (policy, monitoring, balancing, etc.)

Some other trends:
- desire to have pure SOAP as well as a slimmed down XML services (no RM, Sec or TR)
- "ESB" has made its way into RFP's
- brand awareness of ws-pure plays is much higher (Systinet, Blue Titan, AmberPoint, etc.)

Still, the world remains divided on how they view web services. As I recently told one of our customers:
"The web service protocols are at an adoption level equivalent to where you may have found TCP/IP in 1993 or 1994. Mass adoption has not yet occurred, but the writing is on the wall and no one is buying proprietary standards (Novell IPX/SPX). As we begin 2005, web services will enter into their 5th year in existence and remain the cornerstone strategy for vendors like IBM, SAP and Microsoft. This must be part of any go forward integration strategy for a Fortune 2000 company."

Come to think of it... some of those customers are still running IPX/SPX :-)

Seminar: Architecting the Enterprise Service Network

I've been giving closed-audience presentations on the SOE and service networks for over three years now. I finally feel like the material is good enough to do the road show.

So, if I hit the road with my PowerPoints and bright smile... where do you want me to go (cities)? U.S. or otherwise.

Send me a note. The more interest, the more likely I'll add it to the list. jschneider@momentumsoftware.com

Cover of WebSphere Journal

IBM is going for the throat.

Table Oriented Programming

I have found the following site fascinating as a complementary programming style to service oriented: http://www.geocities.com/tablizer/top.htm (thanks Hugh)

Object oriented programming tends to put nouns as the primary concern, while services have been leaning towards verbs. The discussion on Table Oriented Programming has me thinking that there isn't a need to make either the noun or the verb the primary concern. Rather, the focus should be to enable both in an equivalent manner.

Disposable Applications

In the early 90's, the company that I worked for had 3 different application platform strategies:
1. Purchase large, off-the-shelf (OTS) packaged apps
2. Build applications that were core to the business (Core)
3. Build rapid apps for non-core or non-long-living applications (Non-Core)

Back in those days, we used PowerBuilder for #2 (Core). PowerBuilder gave us enough structured techniques to build fairly complex business applications. It recognized the number one use case in business apps: Capture user information, Save it, Make it available to others. Surprisingly, this DB intensive technique could be used on a high percentage of problems. It stunk at workflow / groupware - but luckily this is when Lotus Notes was emerging. Now we had a tool that could rapidly create collaboration environments for non-structured data. We could also use the people in the company who couldn't code worth a shit (and we couldn't fire) to actually get something done. Yes, we called this people "Notes Programmers". And we had a ton of them.

Generally speaking, we considered the Notes team to be creating 'Disposable Applications'. The people were cheap, the licensing was cheap enough, and the throughput was high. We allowed them to knock out small applications - and they did - they churned through them. Then an interesting thing happened... our users started telling us that they wanted the new application to be done in Notes (not PowerBuilder). "What? You want it in Notes??? Those are the idiot programmers that we couldn't fire." I thought to myself.

An interesting dynamic had occurred. Our users realized a handful of things:
1. The Notes team delivered applications faster than the PowerBuilder team.
2. The Notes team didn't make the users feel like technical idiots; thus they became friends with the team.
3. On occasion, the users would make small changes to the Notes applications themselves.
4. The users realized that in addition to collecting, storing and retrieving data, virtually all business processes involve workflow and collaboration. In fact, the collaboration was often more important than the structured data.

I have the unique opportunity of seeing lots of service oriented offerings. Virtually all of them are of the 'PowerBuilder' classification. Most of them start off by integrating into Eclipse or Visual Studio. Ok - with this as a starting point you have already determined that you aren't a disposable application. The next thing that I see is that vendors expect people to understand XML Schema. This again, precludes the disposable community. Should they know XSLT? Nope. BPEL? Hell no. XQuery? Uh... No.

In order to create a Disposable Application environment around a service oriented infrastructure, one thing is absolutely necessary:
- The developer / author shouldn't have to know anything about SOA. No Exceptions.

Most talented engineers hate environments like Lotus Notes. They roll their eyes thinking about scripting hell, inability to enforce uniform constraints and business logic, inability to leverage a common data model and perhaps most significant, it allows dumb shits to look smart.

Talented engineers would much rather create a distributed state machine leveraging a set of 30 WS-Protocols across a messaging infrastructure that leverages a VM that facilitates heap size manipulation, while programming to a set of 17,000 classes that represent the "enterprise API". And oddly, customers are preferring to buy packaged applications that are already fully developed over having custom apps built. Who'd have guessed that one?

So, you ask, am I in favor of disposable applications? Hell yes. It may hurt your ego but it will be kind to your wallet. The real trick is to determine how to design a service network to facilitate disposable applications. It should be possible to create a constrained and structured set of services that contain the end development environment enough to allow 'power users' to do their thing. Then, it is off to the races.

A Note on BEA

The recent departure of Adam Bosworth from BEA has many in the industry talking about the overarching issues that seem to be facing that company. As an ex-BEA partner, I witnessed first hand several unfortunate but not so uncommon events take place, including:
- trouble with channel conflicts
- strong sales people being fired (even in very tough environments)
- less than capable people hired into marketing and bizdev
- inability to articulate a new vision
- failure to create an open source strategy 
- failure to answer the IBM global services threat
- inability to create vertical offerings
- chose not to acquire when their stock price was down - despite industry consolidation
Now, I get the feeling that there were more issues on the inside - but I can only tell you what I witnessed from the outside looking in.

In my opinion, BEA was failing during an absolutely critical period of change in the software industry.
- Venture funds over invested and killed the cash cows of medium sized ISV's
- This fortified the positions of the large ISV's (MS, IBM, SAP)
- Customers were moving to process based applications, not technologies
- Large ISV's moved to an agility based packaged app model

This set the stage for:
-- Microsoft building out .Net and growing Microsoft Business Solutions
-- IBM will build out WebSphere and acquire packaged app companies (Siebel, etc.)
-- SAP continuing the component/service push on NetWeaver
-- Oracle continues to build apps and grow the technology platform (Collaxa acquisition, etc.)
All in all, we are seeing a trend; major application companies are going to market with an application platform that they provide. Smaller ISV's are often picking JBoss or .Net. And medium sized businesses are being attacked by aggressive IBM sales teams.

So, where does this leave BEA?
For starters, BEA should have seen this coming and pushed a deal with either SAP or Oracle some time back. Upon seeing that Open Source offerings was having a significant effect on the low end of the ISV market via rapid commoditization, BEA needed to make a move. Going forward, variations of open source models will dominate markets that are easy to commoditize. Integration will likely be on of those markets - technologist understand the use cases. 

The departure of Adam Bosworth is only one of many issues for this company. Although their products are sound (from a Java perspective), they moved extremely slowly in their web services strategy. Could BEA acquire enough companies to be considered a 'service oriented infrastructure' company? Maybe - but the market is up and the web service startups are making sales which will substantially increase their valuations. This makes it much tougher to stomach the acquisitions. Let's say they pull it off. Is it a good move? I'm not sure. Mostly, I think it just keeps them around until the next wave of mergers takes place.

I wish that I could offer BEA some magical advice. It is a company that has been good for the software community. I don't have the answer, but I have some inclinations:
1. From a technical perspective, look beyond Java. And way beyond J2EE.
2. QuickSilver and the ESB space are already commoditized. Back an open source project.
3. Web Services are easy, "Integrated Service Networks" are hard.
4. Understand why people like the idea of a "Google computing platform"
5. Packaged apps will dominate until the build vs. buy dynamics reverse. Unless you plan on being acquired soon, your job is to reverse it. It's that simple.

As I look at these 5 items, I realize that this is good advice for any of the ISV's that plays in the app-dev, or integration space. Here are a couple more things to chew on:
- web services and service networks increase the complexity of the enterprise I.T. stack
- the more moving parts there are the more it costs to bring them together
- not all buyers need the same level of complexity (if only we had J2EE Beginners version, J2EE advanced version, etc.)
- service oriented infrastructure will be most beneficial to very large I.T. shops
- if you refactor software out of the application and into the network, you should be able to reduce the development and integration time - if you didn't; you failed.
- Remember - dBase, PowerBuilder and ColdFusion. Developers like when you make them look like rock stars - give them the tools to beat expectations.

I feel a bit sorry for them, but mostly I'm mad. As the turnover continues at BEA we can be hopeful that they'll either acquire the companies with the talent or bring in some new vision to steer the ship. Despite my frustrations with them, I wish them the best of luck.

Distributed Noun Processing

Lately, I've been very concerned about the state of Service Network designs.

The service oriented world puts an emphasis on verbs (authenticate, format, translate, etc.) This forces the knowledge about nouns to be distributed across a set of services. Service oriented programming usually looks like a pipelined or staged computing model. Often the messages that are being processed refer to the same schema. However, the processing instructions vary according to the specifics of the schema.

Let's say that we have a four stage service plan:
Step 1 - pull message off of bus and authenticate
Step 2 - transform message according to local needs
Step 3 - process business service
Step 4 - business services encapsulates a call to a data service

Each service focuses on one stage and will typically have a control language (DSL) for performing functions on the nouns (business schemas) that are passed to it. What seems to be missing from most commercial and homegrown services is the ability for a service to easily add or modify the nouns that it is aware of or modify the routines that perform the functions on those nouns.

This is a huge issue. For some reason, people continue to focus on the virtues of SOA/WS-*; loose coupling, reusability, etc. However, they are not talking about the new problem: Distributed Noun Processing. The realization of a use case is now spread across multiple services. Changes to nouns will likely result in changes to all services in the plan. It is likely that the DB schema changes, the O/R plan changes, the XSLT changes, the message type in the WSDL changes, the evaluation logic in the business class changes. The impact of noun changes must be measured (part of the agility/fragility index).

Now a seasoned person might say, "but that is the current state of art as well". Yes - this is true. Generally speaking J2ee and .Net fail to provide facilities for cross-cutting functional concerns across their tiers. So, the good news is that we're only failing as bad as we did last time. However, I think we failed pretty hard last time.

As we look at services, we must identify the best programming model associated with the functionality. Generally speaking, this means that services rely on specialized verb engines. These verb engines usually rely on their own DSL (think DML and DDL for relational databases, axiom languages for business rules, transformation languages, etc.) As the number of DSL's increases, so does the importance of coordinating changes across them.

At Momentum, we have been spending a significant amount of time looking at this issue. We have not found the magic "DSL propagation language" yet, nor do we expect to anytime soon. The MDA world has made progress but is still very disconnected from the service oriented world. For now, the effort is either done by hand or with some tooling that helps to automate impact and change management across a set of predetermined DSL's. Failure to manage the service network from a functional perspective will likely lead to excessive costs for new application introduction and change management. Trust me - get it under control now.

Composite Application: Def #1

Phil at Loosely Coupled was kind enough to send me a definition of "composite application":
 
================================
Here’s Loosely Coupled’s definition:
 
An application built by combining multiple services. A composite application consists of functionality drawn from several different sources within a service oriented architecture (SOA). The components may be individual web services, selected functions from within other applications, or entire systems whose outputs have been packaged as web services (often legacy systems).
http://looselycoupled.com/glossary/composite%20application
 
It’s a favourite term of Gartner’s Massimo Pessini, the man who also gave us ‘ESB’
================================
 
I had great conversations on this topic recently with some really smart people. Unfortunately we were all rather drunk and I don't remember what we concluded - although I think it had something to do with molecules. Ok, this in only partially true. We basically agreed that the definitions of composite apps was still too loose (SOI, SODA, integration at the glass, how many of the NFR's need to be addressed by services, how much of the app needs to be SO enabled, etc.) Bottom line is that we need to move to a 'definition by constraints' view.
 

ToDo: Create Agility Index

On my list of things to do is to create an agility index (and a fragility index). Before I go too far down the road I thought I'd post to see if anyone already has a favorite agility/fragility index in use. Lemme know.
 
Oh yea... so it isn't about being service oriented. Service networks must be evaluated from an 'ility' perspective. I'll start with agility.

Nice paper

Google led me to this nice paper by ACM Fellow, Carlo Ghezzi:
http://www.elet.polimi.it/upload/ghezzi/_PRIVATE/paper.pdf

Composite Apps?

I just caught this from one of the Oracle guys :-)

"The focus of this new blog will continue to be around BPEL and the design of composite applications." (Collaxa)

Has anyone defined "composite application" or determined the criteria for an application to be considered "composite"?

If so, please send me a note - or just post it.

The economics of integration has already changed.

The economics of integrations have already changed.

NEW YORK, July 2 (Reuters) - WebMethods Inc. (NasdaqNM:WEBM - News), whose software links different applications, expects a wider quarterly loss than previously forecast, citing an "unusual number" of large deals that did not close by the end of the period, it said on Friday.

Its stock tumbled 33 percent, or $2.74, to $5.60 on Nasdaq, its lowest level in more than 18 months.

Consider the effects of service oriented integration. The game has already changed. I'll repeat this - the game has already changed.

Fun Facts

Today was a slow day in India.

Only 163 new jobs requiring Java development experience were created on MonsterIndia.com - today (of the well over 5000 current openings).

Most of the positions were paying around $12,000 USD annually.

Today was a slow day in India. Perhaps things will pick up tomorrow.

How many languages do you use?

I love asking Java developers the following question, "How many languages do you use when developing a J2EE application?" They almost always look at me with this pitiful look. It's the, "Jeff is so old that he doesn't even know that J2EE uses Java" look. I'll usually string them along while they try to describe what J2EE is. The answers are great.

After they finish 'educating' me on Java, I follow up with the following set of questions:
1. Is HTML a language and do you ever use it on your J2EE applications?
2. Do you use a 'batch' or 'make' language on your apps?
3. Do you ever use the 'structured query language' (SQL)?
4. Do you ever use JavaScript?
5. Do you ever use JSP?

Usually I stop there - by this time they understand my point.

J2EE is a combination of languages and libraries. We use Domain Specific Languages (DSL) all the time. In the Java world we tend to wrap them with 3-letter acronyms that start with the letter 'J'. We often glue the our languages and libraries together in the form of an application by using an object oriented 3GL (Java).

In many cases, our libraries (JMS, JDBC, JNDI, etc.) merely act as a standardized API to a server (or set of services). These services often need to be customized. We have several ways to customize services: upload metadata, pass in parameters, etc.

J2EE is our container of DSL's and libraries. It acts as the vehicle to pass information from one DSL to another DSL, from one library to another library, from a DSL to a library or from a library to a DSL. The Java language is used to type the data, transform it, perform Boolean logic and pass the data on.

The questions that I've been asking myself are:
What should a library/DSL integration language look like?
Is Java (or any OO-3GL) the best fit for a library/DSL integration language?
What common functionality of the DSL should refactored into a generic 'DSL engine'?
How does a platform architecture provide more consistent extensibility mechanisms across the suite?
To what extent should the components of the platform be mandated to have a symbiotic relationship with the other members?

Back to the original question, "How many languages do you use?" The answer is usually 5-10, and the number is growing. Writing J2EE applications is becoming less about programming in Java and more about mastering domain specific languages, libraries and integrating them across contextually bound Use-Cases.

J2EE has grown organically over the last 7 years. It has not had the opportunity to be massively overhauled. In my humble opinion, the enterprise software community is ready for a massive refactoring of J2EE; one that isn't dragged down by backward compatibility issues. I believe that we are ready to incorporate our lessons learned to design the next-gen software development platform.

Beyond Services

I've been actively writing about services (and mostly web services) for three years now. We've moved from the base specs, to the WS-* functionality, to BPEL, Service Network design, Service Oriented Integration and Service Oriented Development of Applications.
However, the one thing that has been clear to me for the last year is that services will play an important role in enterprise architecture, but will only be successful when combined with other architectural concepts.

In the past, I've thrown out terms like, "Model Driven Services" or "Model Driven Metadata" and "Metadata Driven Services". The fact is, most "web service" companies have been so busy trying to knock out the ws-* spec implementations that they have not had time to think about the converging programming model. But they must!

After the WS-dust settles, we will have solved our LCD problem and hopefully even our GCF problem. What is less clear is how service based development will interact with many of the popular programming concepts (eventing, aspects, code generators, process engines, policy systems, rules engines, etc.) It is relatively easy to create a new specialized brand of computing (like services). It is much harder to weave it with the concepts that are already in place. Failure to do so results in a massively disjointed architecture. The symbiotic relationship between our architectural components must grow.

Dynamic Coupling

Loose coupling has more up front costs than tight coupling. The assumption is, over the life of a component (or service) that the loose-coupling device will pay for itself.

A common belief is that software engineers should design services to be 'loosely coupled'. That is, they should verify that the service can speak the 'least common denominator' and do it in a manner that mitigates the impact of location: attempt to minimize the impacts associated with distributed computing (location transparency).

In the physical world, we use devices to decouple entities. Instead of soldering two components together, we connect each component to a plug and then use the plugs to connect the items together. This is necessary in the physical world.

In the logical world, we have the opportunity to attach a 'plug' to the end of a component or a service. And just like in the physical world, it costs more money to buy and attach the plug. But, the plug should make the component more reusable. That is the investment.

Imagine an electrical component that is attached to a printed circuit board. The component has nice plastic plug coming off of it enabling other components to easily connect to it. In addition, the wire lead on the component is exposed and it is mounted in such a way that additional wires can be soldered directly to it. In essence, the component was designed with loose coupling in mind (the plug) but the instantiation (or the use) of the component allowed the engineer to bypass the plug by directly connecting to the wire lead.

Here, the component has both 'component design-time' and 'assembly design-time' properties. The 'component design-time' properties describe the features that the component has; in our case, we are interested in the plug at the end. The 'assembly design-time' deals with the decisions that the engineer makes at design time when they assemble multiple components together (e.g., does the engineer use the plug or solder directly to the wire?)

When we discuss loose coupling or tight coupling, we must remember that there are different issues to consider:
1. The level of investment that was placed on enabling a component to be loosely coupled (e.g., it comes with a plug attached). [POTENTIAL COUPLING]
2. The decisions that the component assembler made when combining the piece components (e.g., they used the plugs everywhere vs. soldered them together everywhere vs. some combination) [ASSEMBLY COUPLING]
3. The option of delaying the assembly interface mechanism decisions to runtime. [DYNAMIC COUPLING]

A significant amount of literature is available on making components/services reusable (creating high potential decoupled devices); thus I won't revisit. Less is written on Assembly Coupling, but I'm still happy with my, "Service Coupling Index" as a foundation. However, very little information is available on the concept of Dynamic Coupling.

In many ways, Dynamic Coupling contradicts the 'loosely coupled' manifest. Rather than stating that something must be implemented in a certain way (asynchronous, document centric, XML based, etc.), it states that the component need only have Decoupling Potential AND that the component be aware of any Dynamic Decoupling Device.

I like to think of the Dynamic Decoupling Device as a "magic printed circuit board". Here, the assembler inserts components into the board. But rather than soldering the components together or using plugs to connect them, the assembler will never execute the actual assembly mechanism. Instead, the assembler tells the printed circuit board which components need to be connected to each other. The "magic printed circuit board" is now responsible for querying the components and determining the best mechanism to connect the piece parts together. We have moved from explicitly stating the connection mechanism to only stating the intent.

In the physical world, creating the "magical printed circuit board" is... well, challenging. In the logical world it is quite easy. At the core, it means:
- We use policies to describe capabilities and interfaces.
- We publish the policies in a consistent manner.
- We perform protocol negotiation at runtime.
- We build components with high decoupling potential.
- We build components that were designed to connect to Dynamic Decoupling Devices.
- We quit hardcoding our assembly mechanisms.
- We mandate the use of Dynamic Decoupling in assembly.

If an architecture is indeed defined by the constraints rather than the features, than the aforementioned may serve as an architectural style that facilitates reuse without giving up performance.

The Suckline

The other day I bumped into an old friend (Tim) that specializes in Java and database development. He asked what me about my interests and I told him that I have been working with web services. He said, "Oh, aren't those slow?"

Sure, I had 101 smart-ass answers loaded in my arsenal, but this individual was asking a serious question that deserved a real answer.

I answer him, "Well, I guess it depends. Generally speaking, anytime you use distributed computing techniques you will incur extra overhead - and that overhead could lead to poorer performance or response times." This answer clearly annoyed him. He responded, "Yea, I know that you use 'distibuted computing' to distribute computing and the effects that this has... my concern is that web services leverage a bunch of abstract protocols that just weren't designed with performance in mind."

"Ah... I get your point. The protocols suck?"
"Yes - the protocols suck - if you have any concern for performance."

Ok - now we were getting somewhere.
"Tim, the reason they suck is because the big-boys (IBM/MS) chose to roll out 'Least Common Denominator' (LCD) solutions to distributed computing first. When you need to create an ad-hoc computing platform and you don't know the capabilities of all of the participants you must choose only those capabilities that are guaranteed to be available across all nodes. You have to create an LCD baseline (which I call the suckline). The suckline is the decision to use the lowest common denominator capabilities across participating nodes in order to guaranteed communications."

Tim nodded in agreement and commented, "But what you implying is that there is something else out there that doesn't suck - - like a Greatest Common Factor (GCF) solution?"

"EXACTLY - - but there isn't! And that my friend, is the beauty of it...(laugh)..." He looked at me like I was insane - which is a look that I thrive on. "Web services are about protocol negotiation. Currently, we only have one solution for just about any facet of web service computing - encoding, encryption, reliability, transactions, discovery - you name it - we have a version of it that more or less sucks! And you know what - that's ok! IBM and Microsoft have had the patience to work on an LCD platform for as long as they have - and in the face of 'performance critics' - I tip my hat to them. If the protocol negotiation is done correctly you should be able to use a metadata described invocation and interface description mechanism that is linked to a protocol policy for runtime resolution."

Tim asked, "So you believe that it really is just an issue of 'premature optimization'"?

"Absolutely - two communicating java applications will be more efficient at sending serialized java objects then they will be at sending and receiving XML documents. If the applications are on the executing on the same vm, it would be more efficient if they used some kind of shared memory. The GCF capabilities of the participants MUST be determined dynamically (or cached). Should the two applications be prepared to go down to the suckline? Sure - the suckline represents the ubiquity baseline of modern computing."

"Jeff, you just crossed from distributed document oriented to distributed object-oriented computing. I thought that web services was all about document oriented, XML based, loosely coupled, asynchronous..."

"No! - and yes..." - I injected. "This is where I disagree with most of the world. If you ask most people that live and breathe web services... they would agree with your comments. However, I'm not one of them. To me, web services describe a programming model that is evolve-able. The SINGLE most important feature for me is the ability to use protocol negotiation to determine the instance of the programming features and functions that you deem necessary to create a dynamic virtual computing platform. The document oriented problem is really a fatty-chatty problem - and there are solutions to this as well."

Tim commented, "That's cool. But we still have the problem that today web services are slow and we are stuck with the suckline."

"Yep - you are correct. Microsoft has already moved Indigo towards a GCF programming model. IBM is doing the same. What is unclear is if a generic J2EE type solution will become available (in time) to compete against the MS solution. Otherwise, you will see a splintering in the next-gen Java SOA marketplace with only 1 or 2 vendors agreeing to agree on the GCF solution. And yes, it is very possible that the GCF solutions will have patents hung all over them."

"Tim, I can't predict the future, however - I bet on smart people and funded projects. Right now MS, IBM and others are putting their best people on web services and throwing lots of cash at it. Trust me - we will move way beyond the suckline - there is too much money on the table."