The EPTS Use Case Study

Posted on 11/28/10 7 Comments

Frankly speaking, I don’t consider myself a “contrarian” regarding CEP and EPTS.  I am simply not a corporate marketing person, living by quarterly earnings reports, and can live by principles, not by corporate greed.  I am sad to say that I don’t support the EPTS use case study because of how it is “mismanaged” (in my opinion).  EPTS is not a “technical society” .  EPTS  is mostly a group of corporate marketing folks, crafting messages that are, mostly, technically inaccurate.

Most of the core EPTS leadership, while well intended, have little to no practical experience in operational event processing solutions (solving complex event processing problems).

I consider the “use case” exercise of EPTS a type of mis-information dissemination exercise, designed to confirm the “marketeers view” versus the correctly technical view related to processing complex events.

Case-in-point, consider all the “steam processing” acting as transaction engines in financial services; this has nothing (absolutely nothing) to do with forwarding the state of the art of processing complex events (events as electronic messages).

Sorry for the rant, but the “EPTS Leadership” has driven me away from their mission because they are simply concerned about “product marketing”, so EPTS should be called EPSG (Event Processing Sales Group).

That is not my interest area (sales and marketing software products), especially since I don’t work for a software company.

Actually, I have a number of complex event processing problems I am working with, but I do most of the coding in PHP, since the “CEP engines” are basically “fluff in a box”.

On the other hand, there are no true CEP use cases for most of the self-styled CEP products in the market today; these products are not technically advanced enough to detect complex events, only simple ones.  If they were useful, I would be using them!

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks

7 Comments

  1. Brian Husted says:
    Tuesday, November 30, 2010 at 12:13am

    Hi Tim,
    I hope you enjoyed your holiday…mine was great!
    I am not sure I agree with the last part of your statement regarding CEP engines being “fluff in a box”. As we have discussed before, event-driven systems are typically implemented with a collection of techniques and technologies. My definition of CEP engine is a product that provides a subset of the following: Object Models, State Machines, Rules, Streams, Patterns and Events. The value of the engine is to provide an integrated framework that allows a developer to focus on solving complex problems while reducing time spent on glue code. I have successfully implemented systems using both Drools and especially TIBCO Business Events (BE). I have combined Drools with JAXB, Apache Camel and Spring to build some very nice solutions that otherwise would have amounted to considerable amounts of glue code in pure PHP, C++ or Java. In fact, I did a proof-of-concept for a customer this month where we proved how we could replace tens-of-thousands of lines of code with less than 300 in BE. I am certainly not stating that CEP engines are the end-all for event processing but they are certainly proving effective and growing in popularity within a large user community that I support.

    Regards,
    Brian

  2. Rainer v. Ammon says:
    Tuesday, November 30, 2010 at 2:37pm

    Perhaps also interesting what Opher commented http://epthinking.blogspot.com/2010/11/on-multi-billion-dollars-accounting.html

    But let’s discuss without polemic, not repeating the same allegations of the last years;-)

  3. Tim Bass says:
    Tuesday, November 30, 2010 at 3:14pm

    Hi Brian,

    I agree that Tibco’s BE is superior to other CEP engines.

    On the other hand, BE is still, at it’s core, a rules engine like Drools.

    I don’t think rule-based approaches alone are useful for detection in most complex classes of problems.

    And to Rainer, as ***** has plagiarized from most of my earlier presentations and posts without reference, I read from his recent presentations he (finally) understands that CEP is not based solely on rule-based approaches – however, he correctly said (finally!) that rule are good for “action”… detection of complex events is generally not a rule-based problem. I am really disappointed with *******’s style of arguing, denigrating then (finally) learning then plagiarizing others. Way too much “not invented here” and plagiarizing in EPTS…… On the other hand, it is encouraging to see many are slowing understanding what “complexity” is in operational detection problems.

    @Brian – if you are using rules-engines for detection, you are not detecting complex events, you are detecting simple ones (simple event processing). This is not complex event processing, it is simple event processing.

    • Brian Husted says:
      Tuesday, November 30, 2010 at 6:03pm

      Tim -

      Complex events are a coupling of events related to state, rules, calculations and patterns. Complex event detection would certainly leverage objects, correlation, time windows, patterns, and complex rules. This seems like complex event detection. In the cyber world, I would probably leverage something like Snort to detect a complex event like a system intrusion. However, I would leverage a CEP engine to detect that this is a distributed attack (using concepts and windowing), and consult rules to determine the appropriate response (who is trying to attack me, a measure of my capability and capacity at any one location, and what I have already done in the past X minutes). Again, CEP engines were not the appropriate tool to detect the intrusion (e.g., complex event), but Snort wasn’t the appropriate tool to detect the complex response.

  4. Peter Lin says:
    Wednesday, December 1, 2010 at 2:01pm

    Here’s my 2 cents on CEP. The question isn’t “can we use a rule engine to detect patterns on event data?” The question is more specific. “can a rule centric approach work effectively in an intrusion detection or prevention scenario?” My bias opinion is no. In college I knew some hackers and they were good. If I use rules to defend my system, I have to ask “how fast can I respond to new attacks?” I can look at new attacks and try to figure out the best way to implement a rule, but in the mean time those attacks are still coming. For the type of scenarios Tim works with day-to-day, using rules is a loosing proposition. For situations where the patterns are basically fixed or rarely changes, rule driven approaches “can” be effective. I’ve heard horror stories about huge failures with rule engines and I’ve seen failures first hand. Failures happen with every tool, even if vendors hate to admit it and often gloss over things for the sake of sales.

  5. Tim Bass says:
    Friday, December 3, 2010 at 4:59am

    Hi Peter,

    And yes… if the situations are somewhat static and the rule or patterns are basically fixed or rarely change, this is not “complex event processing” by definition.

    Complex event processing was designed (and defined) from inception for classes of problems where “complexity” is defined as you have defined it, for “complex” and dynamic detection scenarios.

    When the “rules” are mostly static, easily defined, or rarely change (relative speaking), this is “simple event processing”.

    CEP, as defined by the stream processing vendors, is a monumental failure, because the systems are nearly useless for every class of problem I work with on a daily basis.

    If stream processing based approaches using pattern detection based on rules was useful, I would have CEP engines running “all over the place”.

    For vendors to argue otherwise is folly at best, fraud at worst.

  6. Peter Lin says:
    Tuesday, December 14, 2010 at 9:53pm

    I feel the term CEP should be thrown away and a new term coined. Chandy’s sense and response is a more descriptive term and does a better job of describing the problem it tries so solve. Although it won’t happen, I wish the EP industry would find a new term like “event centric processing” or “event filtering”. That would make it less confusing and more descriptive.
    I’ve said this before, but the term “complex” is just too loaded to be of real use for technology. Everyone has a different idea of what “complex” means. I think of machine learning as complex, but most bpm or business rules don’t fit that category.