On The History of Event Processing: Global Network Monitoring
In A Short History of Complex Event Processing. Part 1: Beginnings, David Luckham opens his history discussion by saying;
“Event processing has been going on for more than fifty years.”
However, in On Event Processing as a Discipline and Some Subsets another colleague mistakenly blogs,
“… people who dealt in this area [network management and event correlation] have never investigated event processing in the larger sense (e.g. looking at additional patterns), and this area has also not spawned the event processing discipline.”
If you examine just one page from the CEP history at Stanford, researchers there outlined their view of the future applications for CEP, as follows:
- Instant Insight - hierarchical event viewing applied to the Enterprise IT layer.
- Network Level Monitoring and Management
- Cyber Security: Network Intrusion Detection
- Enterprise Monitoring and Management
- Modeling and Simulation of Collaborative Business Processes
- Business Policy Monitoring
- Analysis and Debugging of Distributed Systems
These applications areas mentioned by Stanford researchers, including Professor Luckham, support and validate our recent discussion Magic Quadrant for IT Event Correlation and Analysis, 2007 where we concluded that “event correlation and event analysis is Gartner’s closest magic quadrant (MQ) [...] relates directly to complex event processing (and event processing in general).”
If you take a detailed look at the 1999 CEP presentation, Defeating Large Scale Attacks: Technology and Strategies for Global Network Monitoring you will readily see that our colleagues are incorrect when they says that event correlational and network management folks have never investigated event processing in the “larger sense”. For example, the 1999 slides above, Stanford, slide 6, is titled “Complex Event Processing,” defining CEP from the application perspective of event correlation;
Complex Event Processing
- Accept network ‘events’ from any source
- CISCO NetFlow FlowCollector, tcpdump
- Correlates events based on content and temporal relationship between events
- Event Processing Agents (EPAs) connected in an Event Processing Network (EPNs)
- Both post-mortem and real-time processing
This single event correlational project example from David’s team at Stanford examined the challenging event correlation problems in the context of hierarchical events, maps, patterns, visualization tools, event processing models, patterns languages, network management abstraction layers, and more. Those core event processing problems from this 1999 example, very large and complex then, still exist today and are much more large and complex - precisely why it is called “complex event processing.”
It is quite obvious, in just this one example, that many folks have been looking at event correlation as a motivating application for event processing, in a larger context, for a long time, contrary to what our colleagues write in their “history of event processing” posts.
In a future post I will completely debuke these event processing “history revisionists.” I will illustrate very clearly how the history of event processing goes back at least a decade, and perhaps two (twenty years) before the history outlined in posts like On Research and Practice in Event Processing and The History of Complex Event Processing.
David Luckam stated that the art-and-science of event processing goes back around 50 years.
I am not sure I will go all the way back to 1960 in my next post on the history of event processing. However, I will go back at least to the early days of Internet Protocol (IP) networking and illustrate why distributed IP networking, network management and network security, is one of the key motivating factors for what we now call “event processing” and “complex event processing.”
Filed under: CEP Tutorials, Complex Event, Complex Event Processing, Event Processing, Use Cases
