Apache Mahout: Real-Time Decisioning in the MapReduce Framework
Here is a bit of good news for the complex event processing space. Folks on the Apache Mahout developers mailing list are showing an accelerated interest in topics related to real-time decision-making, starting with a Markov decision process. The Hidden Markov Model (HMM) is listed under “non map-reduce algorithms” on the Mahout wiki. The developer’s interest appears to be partially motivated by this presentation, Towards Learning in Probabilistic Action Selection: Markov Systems and Markov Decision Processes, the October 29 entry in these CMU AI Planning, Execution, and Learning Lecture Notes.
This is great news for fans of complex event processing, moving the state-of-the-art of CEP/EP closer to a Capability as a Service (CaaS) model for real-time detection and decision-making. This overall CaaS direction also aligns nicely with the blackboard architectural construct for complex, distributed event processing classes of problems. Distributed blackboard architectures can be realized in cloud computing CaaS models.
It is a safe bet that my favorite real-time software engineering team, headquartered in Palo Alto (wink, wink), is keeping a close watch on MapReduce-related, Apache Mahout and Hadoop. After a bit of a disappointing couple of years (2007 and 2008) in the CEP/EP technology space, with very little progress toward scaleable real-time analytics that are useful for decision-making, I am starting to be motivated again thanks to the application of Google’s MapReduce framework to suitable analytics.
Filed under: Advanced Event Processing, Agents, Analytics, Artificial Intelligence, Blackboard Architecture, Business Event Processing, CEP News and Events, Cloud Computing, Complex Event, Complex Event Processing, Event Cloud, Event Processing, Predictive Business, Sensor Fusion, Systems Engineering, Virtualization












Tim
I absolutely agree on the relevance and disruption potential of map-reduce techniques to analytics, and in particular, high throughput real-time event processing.
I posted an initial opinion on this: http://architectguy.blogspot.com/2008/11/map-reduce-relevance-to-analytics.html and http://architectguy.blogspot.com/2008/11/map-reduce-more.html
It would be great if your favorite real-time software engineering team worked with my favorite decision management software engineering team around these issues.
Great post again.
[...] Learning: apply advanced algorithms to event clouds to determine new complex event [...]