Real-time data processing

Many use cases across various domains require real-time data processing for faster decision making: credit card fraud analytics, network fault prediction from sensor data, security threat prediction, etc. Data Stream Processing or stream computing is the new computing paradigm for processing streaming data in real-time without storing them in secondary storage. Twitter’s Storm project is a distributed real-time computation system, designed to be scalable, fault tolerant and programming language agnostic. Although it’s at an early stage (currently in incubation at The Apache Software Foundation), Storm is used by well-known companies with significant volumes of streaming data, such as The Weather Channel, Spotify, Twitter, and Rocket Fuel.

Continue reading