Apache Mahout: Real-Time Decisioning in the MapReduce Framework
Here is a bit of good news for the complex event processing space. Folks on the Apache Mahout developers mailing list are showing an accelerated interest in topics related to real-time decision-making, starting with a Markov decision process. The Hidden Markov Model (HMM) is listed under “non map-reduce algorithms” on the Mahout wiki. The developer’s interest appears to be partially motivated by this presentation, Towards Learning in Probabilistic Action Selection: Markov Systems and Markov Decision Processes, the October 29 entry in these CMU AI Planning, Execution, and Learning Lecture Notes.
This is great news for fans of complex event processing, moving the state-of-the-art of CEP/EP closer to a Capability as a Service (CaaS) model for real-time detection and decision-making. This overall CaaS direction also aligns nicely with the blackboard architectural construct for complex, distributed event processing classes of problems. Distributed blackboard architectures can be realized in cloud computing CaaS models.
It is a safe bet that my favorite real-time software engineering team, headquartered in Palo Alto (wink, wink), is keeping a close watch on MapReduce-related, Apache Mahout and Hadoop. After a bit of a disappointing couple of years (2007 and 2008) in the CEP/EP technology space, with very little progress toward scaleable real-time analytics that are useful for decision-making, I am starting to be motivated again thanks to the application of Google’s MapReduce framework to suitable analytics.