Security Event Management
Security Event Management

This post was originally a comment On CEP Maturity and the Gartner Hype Cycle by Changhai Ke of ILOG.  Changhai Ke’s comment was so well written, I have reposted it as a blog entry.

The “More” Part of CEP over ESP is Far from Mature

By Changhai Ke, ILOG

An EDA and CEP must be understood as 2 different areas. EDA is an architecture pattern for enterprise applications. The components are loosely coupled by the use of events. In its strict sense, this is more an architecture pattern than an algorithm.

CEP, on the other hand, targets at the event processing and pattern recognition level. For me, it’s the research for the right algorithm to use to recognize the situations. Pattern recognition, event correlation are all good characterizations for CEP. Back 15 years ago, the alarm correlation in the telecom area was done using production rules (it is still the case), and this perfectly falls into the CEP area.

In fact, EDA comes after CEP, but the CEP at that period was not explicitly called CEP. The nature of their respective study is not the same, one is at the architecture and middleware level, the other is at the algorithm side. As both are concerned by events, it seems that people more or less implicitly include CEP in EDA, mix the two and introduce confusion. Why not. But it’s important to understand that CEP (on its algorithm side) could mature on its way without being worried about the event transportation layer.

As a system, CEP needs input events for processing. If EDA is considered as the only way to bring and transport events to the CEP systems, then of course CEP won’t become successful without the prior success of EDA. But in my understanding, CEP targets some real-time or close to real-time applications, and the event transport layer in those applications are the most often ad-hoc and over-optimized. I fear that EDA has the same kind of performance goal.

Another distinction needs to be made. CEP is more general than ESP (event stream processing), characterized by an EPL for data aggregation with notifications. Even on the market most of the CEP vendors provide EPL languages, CEP has the vocation to cover more than that. The “more” part is not well defined, at least it should include the event correlation, and correlation is not just data aggregation.

The ESP part of CEP could be considered as quite mature. There are so many EPL languages, and tuning has been made on the runtime side. It seems also that some applications based on ESP have proved to work. But the “more” part of CEP over ESP is far from mature. It is often described that CEP could use several technologies, such as statistical models, Bayesian network, time series, rules, etc. I agree that there are a few systems using rules. But where are the others?


Changhai Ke