CEP Engines, Zen and Hollow Brain Theory
For the sake of discussion, let’s assume that the company you work for has a number of very challenging problems that your organization has spent millions of dollars trying to solve over the course of many years. For many enterprises. these challenges include a number of near real-time challenges including fraud detection, intrusion detection, cyberattack detection, counter-terrorism, and network system monitoring and outage prediction (to name a few).
If your organization is like most others, a lot of time and money has been expended to purchase or develop the best state-of-the-art software available from highly educated domain experts with years of operational knowledge and experience. Employees in your organization are highly likely to be a member of a number of professional societies that share knowledge in various problem domains. More-than-likely the near real-time challanges I mentioned above still exist despite large capital expenditures buying domain-specific software developed by the best and the brightest minds in the industry.
If you have been following discussions regarding the marketing buzzword “CEP”, you undoubtably know that software vendors have been saying that forward-chaining event processing rule engines and continuous stream query engines are the solution to most, if not all, of our complex problems. One claim by these vendors is that these “white-box” engines (the current de facto definition for CEP) are superior to the years and years of purchasing domain specific software created by fat brain experts. In other words, the claim by a number of vendors is that a hollow brain is superior to a room full of experienced experts. Since certain problems are so challenging and remain unsolved, get rid of the fat brains, toss those folks out the door, and get a nice shiny new hollow brain. After all, if the best and brightest big fat brains (BFBs) cannot create a highly reliable near real-time complex event detection application, then the solution must be to fire the BFBs and replace them with a hollow brain.
During this post I Googled “Hollow Brain Theory” and this was the result:
No results found for “Hollow Brain Theory“
Of course that will all change with this post as we have found it! Eureka! Out with the Fat Brains, in with the Hollow Brains! The future of the world is better, brighter! Hollow Brains! Hollow Brains! Hollow Brains! Out with Fat Brains! I want a Hollow Brain RIGHT NOW!
Hollow Brain Theory (HBT), in a nutshell, is a new theory that is held by certain people (wink wink) that if years of investment and expertise by the best-and-brighest “domain brains” cannot solve a problem, then all you need is a hollow brain engine (HBE) and a lot of Zen. An HBE is far superior to a fat brain because complex problems, according to HBT practitioners, can only be solved when you start with a hollow brain. You have to admit, there is a certain Zen quality to HBT.
CEP engines on the market today have the rare distinction of being some of the early adopters of HBT. We have read recently in Wall Street and Technology (and other trade rags) where CEP could have prevented the global financial crisis and more. The point is that we are now blessed from heaven with forward-chaining rule processing engines that have no prior knowledge! CEP is moving HBT forward, and it is about time, don’t you think?
There are many complex business problems that have proven elusive to businesses over the years. Spending good money hiring the best scientists and engineers has not worked in many cases. Hollow Brain Theory is a new revolutionary concept that, according to a small number of CEP protagonists, is far superior to Fat Brain Theory.
Who am I to say the proponents of HBT are wrong? Perhaps the CEP vendors are really on to something?
After all, if all the fat brains in network and security management cannot easily detect complex situations in near real-time, maybe a hollow brain is the way to go. Just because a hollow brain does not meet my network and security management requirements, maybe, just maybe it meets yours?
Hollow Brains are Zen.
Otheus says:
Monday, April 27, 2009 at 1:38pm
Data. Twinkling. Fish.
Peter Lin says:
Tuesday, April 28, 2009 at 4:54pm
I got a good laugh from the entry. It’s shame the CEP community is hyping things to silly levels. It gives forward chaining rule engines a bad name that is not deserved. Based on what I see, many of these forward chaining CEP engines are ignoring 3 decades of prior art for the sake of claiming “I invented this cool technology.”
It’s really a shame many people in the CEP haven’t taken the time to study prior art. There are so many lessons one could learn, if they only took the time to study prior art.
This is just my bias perspective, but I get the feeling many in the CEP world are positioning CEP as a replacement for BPM and business rules. My feeling is the current CEP offerings don’t really handle complexity, so trying to wedge CEP into BPM and business rules feel selfish and questionable. The current crop of event processing languages aren’t well suited to bpm, business rules or complex event processing as described on this blog. I hope it gets there, but that’s going to take a decade or more.
peter
Tim Bass says:
Tuesday, April 28, 2009 at 7:42pm
Hi Peter,
I agree wholeheartedly. Furthermore, we have people in the CEP community in the process of writing books as they continue to ignore the prior art and try to pass themselves off (or establish themselves) as an “authority” in a field they simply do not yet understand.
It’s sad, but not uncommon. It is simply another instance of “Hollow Brain Theory”, ignoring the prior art as it they invented concepts that are decades old, and as you point out, at the rate they are going, it will take them a decade to catch up. Luckily, the state-of-the-art advances despite the insular nature of the “not invented here” CEP/EP community.
Yours sincerely, Tim
Charles Young says:
Wednesday, April 29, 2009 at 1:14am
I’m obviously missing something here. The only CEP engine I know of that supports forward chaining is Tibco’s which is, of course a Rete engine. As far as I can see, all the ususal suspects (StreamBase, CorAleri, IBM etc.,) base their technology either on data query engines or the direct application of composition operators, within given event consumption contexts, over event streams. Forward chaining doesn’t appear to be a built-in feature of these engines – indeed I can’t really see how it could be, especially for that second category.
David Luckham illustrates a kind of forward chaining at the EPN level in which, if I have interpreted his diagrams and description correctly, EPAs at the aggregation layer may feed composite event directly back to the IT layer to be potentially further processed through the EPN. That’s not a feature of any particualr EPA technology, though.
As I say, am I missing something? Maybe I have a hollow brain
Tim Bass says:
Wednesday, April 29, 2009 at 3:21am
Hi Charles,
In my post above, I was using the term “forward chaining” in a liberal, and perhaps hollow-brained way
, to describe a system that takes data and processes that data using if,then (select, join) constructs by processing the data in a forward manner, moving from facts to conclusions (causes to effects) verus from conclusions to facts (effects to causes)..
I was not using the term in the specific technical implementation context of Rete or a layer of granularity that would distinquish between a formal rule-based or query-based systems. I am using a more liberal, less granular abstraction.
Generally speaking, although not the main point of my post, I was referring to this type of processing as forward-chainng event processing rule / query engines.
I agree that if we use a strict rule-based definition of forward-chaining, then perhaps I should find a different term to describe the abstract concept above.
Charles Young says:
Wednesday, April 29, 2009 at 8:37am
Oh, OK. ‘Forward’…without the ‘chaining’ bit
. I understand now.
Tim Bass says:
Wednesday, April 29, 2009 at 11:01am
Hi Charles,
Thanks for correcting me. You are right. It is a bit of a leap to call the logic in event stream query engines “chaining”. In fact, it is incorrect to call these query engines “rules engines” as I believe you might have said at one time. So, I should modify that part of the post to reflect your correction.
I was using “chaining” too informally, to basically mean the application of a “chain” of IF-THEN-ELSE query rules. I stand corrected. Thanks again.
Yours faithfully, Tim
Edit: Updated post based on your correction. Thanks again!
Peter Lin says:
Wednesday, April 29, 2009 at 4:54pm
I would agree many of the CEP engines do not fit the classic AI definition of forward chaining rule engines. They’re really query engines and lack the functionality a classic expert system shell or production rule engine provides.
None of the simple query engines can support AI, machine learning, case base reasoning, assisted learning, unassisted learning or pattern discovery. To achieve that, they would have to be completely rewritten from the ground up. Before that can happen, they have to spend 5-10 years studying the domain.
Though, if one were to look at the hype from the vendors, a developer might get the impression they’re equivalent to a mature production rule engine. Clearly, that isn’t true, even if sales guy say it is.
peter