The Secret Sauce is the Situation Models
Alan Lundberg wrote, Intelligent Business Process Platform? in response to Bringing Order to Chaos where someone from PWC linked event processing to business intelligence and business process management. In turn, James Taylor penned Using decision management to deliver intelligent business performance where James rightly said that it does not require “heroic efforts” to integrate event processing, BI, BPM and other decision support tools.
As a reference, you may have seen this briefing, one of many where I show these functional relationships, Mythbusters: Event Stream Processing Versus Complex Event Processing, from DEBS2007. For example slide 23 shows the functional relationship between events, pre-processing, event tracking, situational detection, historical patterns (the output of BI tools, for example), visualization and business process management.
In Faithful Representation, Richard Veryard reminds his readers that the most challenging part is in the situation models (not the system integration). Unfortunately, by accident, Richard incorrectly attributes Opher Etzion’s “first order situation model approximation” to both Opher and I in this quote from Richard’s post, “a simple situation model of complex events, in which events (including derived, composite and complex events) represent the “situation”.
Actually, that simple situation model above is Opher’s, not mine. I have offered a more general and comprehensive (first draft) situation model, in A Simple Situation Model for Complex Events based on a cognitive situation model used by researchers at the University of Notre Dame. I do not believe that complex events and situations can be modelled accurately using Opher’s simple model of derived, composite and complex events. This model is overly simple, in my opinion. to represent the vast majority of CEP classes of problems, perhaps explaining why Opher and I do not agree on the state-of-the-art of CEP. Opher tends to view CEP as mostly an extension of active database technology where I see CEP as a technology that is much more closely aligned with the cognitive models represented in the art-and-science of multi-sensor data fusion (MSDF).
Complex events represent situations, and situations must be accurately modelled if we are going to accurately detect them in real-time. If your business cannot model a complex event (situation) then it does not matter what software you buy, how much money you spend, or what event processing and integration platform you use. The models are hard. The system integration is relatively easy.
The secret sauce is the situation and complex event models.
As mentioned here a few times, it does not matter how fast you process events in real-time, if your model is wrong, you just detect the wrong thing very fast. This is very bad and quite dangerous. You will make bad decisions fast. You will waste time, money and resources.
This is why CEP benchmarks should be based on accuracy in situation detection, not in latency and other low-level performance metrics. First, get the models right; then refine to detect faster, if speed is required. What has happened in CEP to date, is that the models are so simple, they do not really detect complex events, they just process and act on simple events that are easy to model.