On behavioral programming

Yesterday I took the train to Rehovot and visited the Weizmann Institute.  I gave a talk in a local seminar and  was hosted by David Harel's research group.  An interesting work they are doing is on behavioral programming, which is a formal language independent model to specify reactive applications.  They have been experimented with games, robots and toy helicopter.   They are actually specifying event processing systems;  I see a possible benefit in their approach that their formal model is a basis for validation which in event based systems is tricky,  due to the temporal nature.   I'll  work with them on some use cases  from our universe to learn more about their model.    Can be interesting.  

Reactive and proactive as relative terms

This is one of the most famous visual paradoxes, in this picture one can see a young girl or and old woman, some people see only one of them, with some concentration one see both.   This is a kind of relative view, in event processing there are some relative terms, I have written a while ago about the fact that the terms raw and derived events are relative,  a derived event in one sub-system can be raw event of another sub-system.. 
There are other cases of relative views (an entity may be both consumer and producer, for instance).  


I have reminded on relativism while reading an article about the keynote talk of Jeff Shulman (Gartner's manager of the application integration and web services analysts team).  Shulman is talking about cloud, mobile and CEP as the leading trends for application development and integration.   
As a remark to Shulman's keynote, the article bring an interesting response of  Chris Dressler, VP Technology in Cablevision,  he sees a  CEP can be used to find and correct issues before the end user has the need to make a complaining call.
This is an interesting example,  from the service provider's point of view (cable TV company in this case), this is a reactive application, tracking events that already happened and react to them, from the home consumer point of view this might be proactive, since the consumer may not yet felt the impact of the problem, so from the consumer point of view, this is elimination of problem that has not really happened.    More on this distinction - later. 

Revisiting "Right Time"

This illustration (as indicated in the bottom) is taken from the "enterprise irregulars" site, posted by Ray Wang.
Ray Wang cites a relatively old posting of mine, talking about real-time, right-time and other time related concepts.   I admit that sometimes I abuse the term real-time (like other people do, but this is not a good excuse!), but I have not adopted the term "right-time".   In that posting I bring some classical definitions of various types of real-time.  Wang is making a somewhat different classification as a matrix with two axes:  the reactive/proactive axis, and the business value axis (low/high).  The high value proactive is called "anticipation", and the low value of proactive is called "nice things to do".   My interpretation is that both deal with notifications that may allow proactive behavior, but not necessarily automated proactive behavior of the type that we talk about (see my keynote talk last year in the OMG conference),  on the reactive front, the high value are mission critical reactions. and the low business value are called "timeless responses".  Here, I am not sure it is the best title, as there are reactions that have low value, but are time dependent, since they lose their relevance in time.  Example here is that getting an alert on available discounts in a nearby store may not be that important for me, but the discount is applicable only within the next hour, so if I would like to respond, there is time bound on this response.   Anyway - interesting classification. 

On the 4Ds -- past version and the proactive version

The climate in Israel this year is quite strange, it is December, and today I still saw people going in the street with short dress, the summer just did not go away.  However, the forecast for the next three days, starting tonight is of a major winter storm (which here means a lot of rain, not snow) and much colder weather, so getting the winter clothes ready.   


Today I've read a blog posting by Jeff Adkins,  one of the people with most practical knowledge about event processing, who relatively recently joined IBM  GBS (Global Business Services).  Jeff blogged about the 4Ds-- detect, derive, decide, do.    These four are part of smart systems that sense and respond, what is known as reactive system.  I knew that this looked familiar, so went back to my archive and found that seven years ago we  were engaged with a project called "active integration"  (the actual application was in the insurance area, but we have generalized the concept), roughly what was known by Gartner as "real-time enterprise".    Here is the original flow from that project:

  
I don't think it has been original invention, it was a variation of concepts from control theory, but it looks very similar to the 4Ds, with a more detailed granularity.


The first D: Detect spread into two phases on our model: sense and detect, where the sense dealt with instrumentation and sensing of raw events, and detect with a detection of the meaningful situations by pattern detection.


The second D: Derive is the same:  created a derived event (and sometimes also derived data) as a result of this detection


The third D:  Decide is partitioned to three phases:  Analyze - determine the possible alternatives and recommend,  Collaborate - in case of "human in the loop" within the decision, and Decide -- apply some decision procedure to select among the alternatives (simulation, analytic methods, predetermined rules).


The fourth D: Do we called "effect", since it had to effect a running system.  


The loop indicates that the after effecting the system, it feedbacks through its instrumentation mechanism and the sense phase is looking again for things that needs reaction.


I agree that the 4D is much more catchy then our more detailed drawing.


And one comment:  when we did this work, we worked on REACTIVE system - something has occurred, we detect it, and then do something to repair.   This drawing is also a good description of our current project that deal with PROACTIVE systems, but the semantics is somewhat different:  the detection is of predicted undesired states instead of situations that require reaction since something already happened.   

On event driven vs. business intelligence drive viewpoints

The last few days in Israel were extremely hot, one local newspaper claimed that Friday was the hottest day in Israel within 112 years.   Now it is somewhat less hot, but still very hot.  Relief is expected later this week. 


I am playing now with the new editor of the Blog editor, which looks like Wiki editors, it seems that web editors are starting to converge into some form.


Anyway -- recently I have read some "business intelligence" stuff  - ("analytics" is now a hot buzzword in IBM, and probably outside IBM as well).    In business analytics terminology people talk about three phases: descriptive, predictive and prescriptive, while in event processing we also talk about three phases: responsive, reactive and proactive.  So I was asked - are those terms equivalent.   The answer -- not exactly.   


Let's start with business intelligence, or analytics in general.   The main starting point is:  we have historical data,  we can present it in different ways,  we can learn from it something that can provide observations, and can predict future data (e.g. by trends) and then we can propose actions to bridge gaps towards our goals.


The basic starting-point --  analyzing existing past data.  The first phase is descriptive -- describes what is seen in the data, this is the most common use of business intelligence.

The second phase is - predictive, find trends and extrapolate into the future, predicting future values of the same data.  

The third phase is prescriptive - given the predicted data, and possible gaps between this predicted data and the enterprise's goals -- propose a way to bridge the gap,  e.g. change inventory policies, change risk policies,  even getting to change business processes.  


Event processing is starting from different viewpoints - there are events happening now, and we would like to react to them -- the metaphor is  -- a dangerous bear is approaching and I need to react.

In event processing the evolution is starting in "responsive"  - in this case, indeed event is treated as data, information about events arrive using queries, search, or even applying any kind of analytics, this is the regular mode of programming, but it is data-driven rather than event-driven. It may be applicable to some applications, will not be very helpful in the case that the bear is chasing you.    Event driven architectures and programming has enabled the next phase in the evolution - reactive programming, in which predefined alerts or actions can be triggered by the fact that an event has detected, or that an event pattern has been identified. Currently the state-of-the-practice in what is defined under event processing applications fits this category. The next step in the evolution is proactive, which means that by computerized means we'll be able to identify predicted events, and then a decision of how to mitigate or eliminate the event is being taken,  for example when a bear is chasing me, I need quickly to decide whether my best bet is to hide, escape, or shot tranquilizing darts at the bear

The decision is done on-line, and has some timing constraints (depends on how close the bear is). 


Having explained the basic terminology, back to the original question, how are these terms related.  
First, the goal of business intelligence and event processing are typically distinct, however there are some points of overlap.   From the BI perspective,  reactive event patterns can be used as a component of predictive analytics.  Proactive event-driven processing can be thought as a type of prescriptive analytics.  The overlap occurs when the analytics system has real-time component, which requires that the prescriptive analytics will be done on-line and with some timing constraints, this turns it from being data-driven to be event-driven, but one can think of prescriptive system that is totally off-line - analyzing data in batch, predicting shift in trends, and change the policies for the next year/quarter. 


From event processing perspective, analytics tool can be used in populating the event patterns, but this is not that easy -- I'll write soon about some thoughts on the feasibility of patterns learning.    

More on responsive, reactive and proactive computing

Earlier this week, I've posted in this Blog short definitions of the terms: responsive computing, reactive computing and proactive computing. Somehow, terminology always gets questions and responses, and there has been a thread in the Complex Event Processing forum that started to discuss it. So as a follow-up to this discussion, some clarification. The computing modes (responsive, reactive, proactive) are indeed mutually exclusive, however a single system may have combination of all of them.

Active databases are example that start with responsive computing, the basic operations are: insert, modify, delete or retrieve from a database. Then the active database engine takes these database operations as events, and apply reactive computing to execute rules reacting to these events.

The opposite direction is a reactive system, doing some kind of event processing, which during the event processing operations need to consult a database in order to enrich an event with more information. The database query issued is a responsive computing.

Proactive computing may also be combined with reactive and responsive systems.

Hans Gilde has posted on the complex event processing forum an example that combines all three.

While responsive computing is the bread and butter of computing, and reactive is now being more understood, proactive is still lagging behind in terms of realizing the potential, maybe the next hill to climb.

On Proactive Computing

In the picture above you can see some products that are designated to protect your skin from various things under "proactive skin care", actually preventive medicine is also aimed at protecting one's health from future problems.

Mani Chandy has recently posted in his Blog an article about Proactive computing. I thought it is worthwhile to say a few things about this notion.

We can view three types of computing: responsive computing, reactive computing and proactive computing.

Responsive computing is the way most computing is done, I am using now some software for Blog editing, sitting on some server and waiting for me to connect, and use it, in the rest of the time it sits idle (well, the server is, for sure, being used for other important stuff), and does not do anything unless explicitly requested, this is typically the way we are working with Web servers, databases and mostly everything else when using computing.

Reactive computing is the way that event-based systems are working. An event occurs and then it triggers one or more computational processes, without being response to any request. Event processing is a vehicle for doing reactive computing,

Proactive computing is the case when a computing system arrives at a conclusion that some entity will reach an undesirable state, and attempts to eliminate it from getting to this state, or mitigate it by transferring this entity to a more desired state instead.

Some examples of proactive computing:

• The traffic lights policy are set before a football game is going to end and the traffic is expected to flow in certain directions.
• A repair technician is late in one of its sessions, and combining with traffic conditions he will be very late to his next customer, something that will cause several chain effects.
• A progression in the state of a patient will make his treatment protocol inadequate

There are many other examples as well.

As said that we are now at an era of the responsive computing, reactive computing had caught on in some segments, and more segments of computing applications are now discovering it, but it is far from reaching its full potential. Some of the reactive programming applications are aimed at proactive computing, by the fact that some combination of events may be interpreted as getting to some future state that should have some action, and there are some commonality among them, it is still a good idea to look at proactive computing as as its own notion, and understand what a proactive computing application is like, and what are its characteristics.

I'll write more on that later.