Peter Lin on Situational Awareness and CEP
Recently Peter Lin was kind enough to stop by and post this comment to our post, CEP Software Saves the Universe!
Here’s my [Peter Lin's] bias perspective as a user and developer of expert system shell[s]. A business rule engine, expert system shell or CEP engine at best provide[s] a foundation for creating an expert system, but they’re not expert systems. Also, not all CEP engines are created equal. Many don’t provide inference capabilities.
To create an expert system, one has to create a knowledge base, which includes data model, facts and rules. Even then, for dynamic environments, the classic knowledge base system approach isn’t enough. A knowledge base system by itself, it’s not well suited to highly dynamic environments.
As we’ve discussed on this blog in the past, machine learning is needed to handle dynamic environments. Not all rule engines or CEP engines are capable of machine learning. This is true of any engine that compiles rules down to Java code or C# code. Many, if not most machine learning systems choose an interpreted design for this reason. Whether a CEP is capable of machine learning really depends on it’s design and implementation.
My bias perspective, proper support for dynamic SA would require machine learning and statistical analysis capabilities. Without that, it’s just business decision automation. I’m sure others will disagree.
Of course, it is no secret, I completely agree with Peter. “Situational Awareness” is “old and stale” unless is it dynamic and adaptive. Situational knowledge must change and update in real-time, with every new input to the system. This was one of the lessons that we learned in the USAF ten years ago when we were trying to build countermeasures for massive email-bomb attacks by writing rules. It does not matter if the rules are written in PERL, C, C++, Java, or some nice new event processing language. Compiled rules are not enough. They are only “baby steps” to true SA, command and control, sense and respond.
It is quite simple really. The “situation” constantly changes because an attacker (for example) can know (by systems analysis) the defenders are writing rules to counter their moves. They can easily know this by observing how systems behave when new attacks are published and how long it takes for the system to respond and adapt. Experts writing rules (no matter how clever the experts are) cannot adapt quickly enough to gain full advantage over a skillful attacker. Therefore, as Peter mentioned, “machine learning is needed to handle dynamic environments. Not all rule engines or CEP engines are capable of machine learning. This is true of any engine that compiles rules down to Java code or C# code.”
This is the very essence of what is “complex” about “complex event processing”. Most real-world “hard problems” are not static, they are dynamic and ever-changing. Skillful attackers in cyberspace (for example) are dynamic and determined. The new generation of cyberspace criminals test systems repeatedly for how the system reacts to new inputs. Static defensive systems are a huge weakness, especially when networks are moving bits-and-bytes around the global at wire speeds.
The same is true for almost any valid CEP application - real-time fraud detection, real-time risk management, real-time climate analysis, real-time air traffic control, real-time network management and diagnostics. Domain areas that are generally static are relatively easy to process, and as Peter concludes, these applications are business-as-usual “business process automation” tasks, not CEP tasks.
The “big elephant in the room” in the ongoing CEP dialog is that most of the current (CEP) software on the market is not capable of machine learning and statistical analysis of dynamic real-time situations. Software vendors have been promoting and selling business process automation solutions and calling this approach “CEP” when, in fact, nothing is new. There is certainly no “technology leap” in these systems, as sold today.
It is time to bring credibility back to the promise of CEP and to focus on solving complex, dynamic, adaptive problems, not static, business process management classes of problems, and marketing these relatively simply problems as “complex”.
Filed under: Advanced Event Processing, Business Optimization, Business Process Management, CEP Terminology, CEP Tutorials, Complex Event Processing, Cybersecurity, Event Cloud, Event Processing, Predictive Business, Process Optimization, Risk Management, Scheduling, Security Event Management, Sensor Fusion, Simple Event Processing, Situation Models, Systems Engineering, Threats and Vulnerabilities, Use Cases
I didn’t think my comment would be taken so serious
Back in my high school and college days, I knew a few crackers and hackers. My personal experience with hacking groups (not script kiddies) over the years has taught me they are smart and organized. Anyone foolish enough to think a static defense against organized attackers is clearly in denial. Honestly, I don’t understand why many in the CEP community resist change.
I’ve mentioned this before, but I think Stanford’s stanley paper outlines many of the challenges of dynamic environments. Blindly processing all inbound events isn’t scalable. As Sebastian Thrum’s paper explains, the system has to filter the data and only reason over a small subset. I could be wrong, but the same basic problem applies to situational awareness.
peter
oops, just noticed a typo. That should have been “I didn’t think my comment would be taken so serious”.
peter
(fixed)
Hi Peter,
You said “Honestly, I don’t understand why many in the CEP community resist change.”
By observation is, perhaps simple minded, is that most of the active people in the CEP community are not from the operational side of the house. They are either (1) software marketing, (2) academics or (3) analysts or (4) interested in capital markets trading platforms.
A lot of the concepts discussed in this blog are based on operational experience in network and security management, cyberattack and defense, and experience in complex SA-type systems for the government (air traffic control) and the military (air defense).
There maybe other reasons, but I think it is simply a lack of experience in day-to-day operations where processing complex events has been a mainstay for many decades.
Yours faithfully, Tim
I am coming from the DOD side of the house, and CEP being embraced enterprise wide for a very big agency that I support. Over the last two years, we have been implementing CEP to enable scalability for SOA. Specifically, we are implementing CEP for including intelligent routing of messages, runtime governance, system and business monitoring, and execute high volumes of complex workflows.
Our operational experience on multiple programs has exposed the model driven approach (aka ontology) combined with event processing allows us to be more agile and reactive to changing mission needs.
Peter and I have had some of these discussions, but I am interested in Tim’s view on the Tibco Business Events approach vrs something like Streambase or Esper.
Hi Brian,
Thanks for visiting and commenting.
I advised the USAF, DISA, OSD/DOD for many years, and depending on where you work, most top down IT initiatives fail and fail miserably.
What program(s) in the DOD are you talking about? NCES?
The DOD, especially at the HQ and DISA level, tends to be very buzzword driven, and it is quite easy to say somethink like: “we are implementing CEP for including intelligent routing of messages, runtime governance, system and business monitoring, and execute high volumes of complex workflows.”
This sounds like MITRE-speak, where MITRE gets paid good money to make these very grand statements; but in reality, very little operational benefits emerge.
So, I would be interested to learn exactly what program(s) and program office are you referring to?
Yours faithfully, Tim
Tim -
I am part of a large user community and we are not buzz word driven. In fact, I have been working closely over the last three years with Nelson Petracek at Tibco, i think you know him because he knows you. There are about three or four of us right now that have architected and implemented several operational solutions using Tibco Business Events (BE). My experience is that a small number of people (2 or 3) can turn out big things with BE.
In 2006, we developed our initial big success with CEP that involved service provisioning to drive IT resource consumption from the consumer side rather than the provider side as done with the classic static deployment model. The resulting dramatic increases in throughput realized from provisioning service based on demand allowed us to consolidate three systems (RMI and JavaEE based) running on 40 servers into a single distributed system running on 5 servers. To me, that is as non-buzz word compliant as I can be… : )
Here is a white paper I wrote that describes what we did in detail:
http://www.pyxisengineering.com/pyxis/stories/releases/0003.html
And yes, I have had the chance to work briefly with MITRE on an ONI engagement. ONI was interested in CEP and SOA but MITRE was very much an impediment throughout the process, and we really avoided the opportunity for that reason. Pretty sad…
I am interested in connecting with you to discuss both the vision of CEP and the operational realities in the trenches. Here is my linked in profile:
http://www.linkedin.com/in/brianhusted
Hi Brian,
Yes, TIBCO’s BusinessEvents (BE) is certainly in a class by itself in the event processing space. The scheduling and orchestration capabilities of BE are unsurpassed.
However, just to let you know, I don’t consider these classes of scheduling and orchestration applications, “CEP” per se; they are more accurately described as “event processing applications”, because the application is serving as a scheduling / orchestration controller - a business process automation application.
Cheers.
Yours sincerely, Tim
Tim -
Extending your concept, the ability to monitor, alert, and react to global events involves developing and orchestrating event processing applications that use both simple and complex event processing tools. The simple event processors reduce the event noise near the data sources by filtering and aggregating atomic data using sliding time windows, rules, and/or persistent queries. The aggregated data is then forwarded to a central distributed correlation engine(s) where large scale temporal processing can occur under a distributed cache.
The data cache can be exposed through a query engine to allow for visualization and human analysis of higher level events. Depending on the use cases, concept instances may live for minutes, days, or years. Ultimately, I summarize the impact CEP is having on the enterprise as reducing the complexity to enable a more agile and effective business model.
Hi Brian,
Excellent. Yes, that sounds great.
You are certainly on the right track with CEP.
Congrats! I look forward to hearing more!
Yours sincerely, Tim
The “Situational Awareness” is absolutely the problem area CEP vendors should be focused on. The fundamental tenet of the solution has to based on context aka knowledge around a critical situation (a pattern).
In the last few years, BE has had a very high adoption rate and part of the reason has been our focus on the complete lifecycle of an event and a situation. Tim may know this already but we in TIBCO are focused on addressing the following lifecycle stages:
1. Model the Knowledge Universe.
2. Encode Knowledge
3. Detect Situations and Patterns using the above.
4. Situational Actions and Responses
5. Visualize a key business event and the associated context.
6. Analyze the past and mine the related patterns.
The next step for us is to add elements of Knowledge discovery by incorporating Machine Learning, Fuzzy Logic, Neural Networks, Bayesian networks and statistical models using S+. Tim should be happy to hear this..
Cheers
Puneet Arora
I am very happy to hear that Tibco plans to introduce additional event processing analytics under it’s CEP hood. As Peter Lin and I have discussed, RETE and rules alone are not adequate to scale the temporal processing required to find the needle in the haystack of events. An event processing application developer would benefit from a combination of CEP analytics to more efficiently model patterns to detect events of interest. Thus, the state model (aka ontology) can then be abstracted from the CEP analytics that generate higher level events. Bayesian analytics could be used to tag concepts as fitting particular profiles (e.g., a Person Concept fits a fraud detection profile). From a consumer perspective, the state model provides the highest level of abstraction with hopefully the most value to the business analyst. Essentially, the ontology could be implemented to provide the most intelligent and near-real time view of the “situation”
Hi Brian,
The limitations of rule-based expert systems was well documents decades ago.
Anyone who actually works trying to solve complex detection in operational situations using only rules knows this from experience.
The fact that we are actually arguing this point with small companies who are new in the field, selling forward chaining stream processors, is what is surprising to me!
Yours sincerely, Tim
[...] see from Tim Bass’ blog that Brian Husted of Pyxis has published a paper titled “Event Driven Grid Computing“. [...]