Commentary on the Floundering CEP Market
In response to Complex Event Processing Vendors Flounder is important to point out that it is a fact that the so called “CEP software” on the market today has made littie progress outside a handful of niche areas in capital markets where there have been no generic event processing platforms available in the past. In the broader market, where many event processing platforms have existed for years, the current “CEP players” have made zero to little progress. So, of course the “CEP market” is “floundering” outside of capital markets. Furthermore, all this constant “attack those with different opinions” from same same players, over and over again, is not advancing the state-of-the-art of processing complex events.
For example, I recently worked with a team of experts to install an event processing platform to monitor events generated by a very busy operational web site. We found the so called “CEP software”, which most of the offerings are simple forward chaining rule-engines, to be unacceptable. Our team opted to use Zabbix, a free, extensible and open source monitoring platform where me montor over 300 events over time-windows, running rules, doing basic event correlation, visualization via nice charts, etc.
Frankly speaking, if the self described “CEP platforms” added value, we would have used them, of course. We tried a few of them, objectively. Nobody has any personal grudge against any CEP vendor; users (including me) simply have difficult network-centric operational problems to solve and we will use the best, most efficient, and most economical solution(s) available. Constantly beating up the users and analysts who do not find value with these “CEP engines” is not useful for anyone.
TIBCO acknowledges that the current generation CEP platforms are problematic. Just read Puneet’s recents comments (head of Emerging Technologies and CEP at TIBCO), where Puneet says:
The next step for us is to add elements of Knowledge discovery by incorporating Machine Learning, Fuzzy Logic, Neural Networks, Bayesian networks and statistical models using S+. Tim should be happy to hear this.
In other words, there are a lot of us out there in the market, who are not in capital markets, who do not find the current generation of “CEP software” useful (yet). Furthermore, we do not appreciate the “beating up the messenger” approach by a handful of so-called core CEP players, when it has already been acknowledged that the state-of-the-art of the software being sold as capable of processing complex events has a long way to go to reach that goal.
That is why I have always admired TIBCO. At least TIBCO has leaders who are not intimidated by experienced users who know what capabilities they need to process complex events. TIBCO has made the right acquisitions and are expanding their capabilities, not contracting or consolidating them.
For those who quote IDC and other analysts reports, you must keep in mind that most of their analysis is based on talking to vendors selling software and they are focused on the current users, who are mostly niche players in capital markets; players who, in the past, have not had generic event processing platforms to process market (and similar) data.
Editorial Note: While I am heaping praises on TIBCO, I think it is only responsible of me to note that I ((strongly)) disagree with Paul Vincent’s approach to recently attacking both Philip Howard and Curt Monash in their critical comments about the state of the CEP market.
Filed under: CEP News and Events, Complex Event Processing, TIBCO
Opher,
I cannot speak to your comments on IDC because I have not seen the report.
On IDC, I saw the reference to their report first on your blog!!
And you still owe me the link, which I asked for!
As for the Forrester report, I’ve read it, and I think its not “Vendor Oriented” at all. You should read it yourself.
Roy Schulte at Gartner would probably take exception to what you say too.
Your experience with current tools is not at all surprising. The only surprising thing is that you found it surprising!
I’ll deal with that in the Forum.
- David
I suspect that it is quite difficult for analysts and commentators to differentiate between the hype, spin and the reality especially since so many parties have a vested interest one way or the other. When you are the receiving end of unwarranted criticism or poor coverage it is hard to take. But eventually the reality seeps through. At least I hope so!
So first hand experience is often much more valuable than the research. In this vein, I can tell you that I moderated a round table at the conference “Accelerating Wall Street” last week and discovered that the two pressing topics for discussion, voted by the round table I was on were- CEP and Hardware Acceleration. I was delighted to lead a discussion on hardware acceleration but I started with CEP.
In fact, we had an excellent discussion of CEP for the the analysis of real time data in the financial market.
We went through a number of use cases from an exchange and broker point of view. There was also a hearty discussion about experiences with different vendors.
So as an outside observer, it was strong confirmation that firms have been using CEP and have experimented with various vendors. It’s not hype! But there was a general feeling that outside of government(intel), finance and the Telco industry, there were no obvious customers.
(My view is that that opinion about the limited applicability of CEP might just be based on lack of experience and the finance silo that we’re all in.)
It is worth noting that the personnel that have checked out CEP said they would prefer to use one or two vendors. They felt that it was important for the vendors to have financial expertise in-house.
They felt that it was good that Coral8 and Aleri merged but there are too many vendors even now. Nevertheless, they believe there is a future and no one used the word “floundering”.
Tim … sorry, for some reason I thought this was on Opher’s blog. Too many blogs!
- David
Hi Jeff,
You are talking about the perspective from financial services. The original article says that CEP is floundering outside of financial services (which I agree with). Your comments are basically saying “In a meeting of financial services folks they did not use the word floundering”…..
The discussion about “floundering” is not related to financial services and all you need to do is read the complete (almost) lack of customers and analysts outside of FSI to understand this. One or two customers is not a robust market!
There is no reason to argue about adoption. The lack of adoption outside of FSI is crystal clear.
Yours sincerely, Tim
Hi Tim,
True, the group was all from within the financial silo. They assumed that “government” uses this kind of CEP software and were guessing that telcos might find it useful also. So maybe the group itself was casting about rather hopefully.
I was just impressed that the firms seemed to have several decent use cases and were experimenting. I have read the CEP blog and trade press/analysts headlines and met some CEP vendors but was never sure how much was smoke and mirrors.
Perhaps the financial sector can make enough purchases to keep the sector healthy. Presumably CEP is at a low point in the Gartner Hype Cycle and the jury is still out.
(Tim- I promise to read the articles more carefully henceforth.)
Cheers
Jeff
Hi Jeff,
I don’t think you quite understand.
Myriad people have been trying to solve complex event processing classes of problems for decades. If all the self -described CEP vendors disappeared tomorrow, the state-of-the-art of processing complex events and situations would still move forward.
I have worked in telco, in network management centers for telco, and consulted for the military for decades. These simple forward chaining rules engines marketed as “CEP engines” are but toys compared to what many telcos and governments do. (Note: OK, maybe some third world countries do not process complex events today …. I note that in Thailand there is little computer-computer communications between any of the cities.)
How do you think a missle launch is detected? How about air-traffic control? How do you think weather is forecast? How do we look for terrorist threats in cyberspace? How to we track outbreaks of killer viruses like Marburg? The list goes on and on.
The original work funded by DARPA for “CEP type” research was to solve complex distributed computing problems, not to run rules against linear streams of data. There is no reason to fund this type of research, we can process streams of linear data with simple programs and scripts have have been able to do this for decades.
That is why there has been little adoption, because what is being sold based connected to the “CEP buzzword” provide nothing new to most industries. Finding one or two people or ten, or even 100 people, to try out aggressively marketed software does not define an evolutionary technological achivement for mankind.
The real forward progress and advances in technology are happening outside of forward chaining rule engine-based technologies marketed as CEP. I find it amusing to see the aggressive comments by folks in FSI, as if they have found something new. It simply shows how far behind the technology curve they are!
Yours faitlfully, Tim