PulsarIO Demo

PulsarIODemo demonstrates the visualization solution for data analyzing.

Data Feeding

We have some demo data feed into the pipeline and MetricCalculator does the aggregation and persistent the metrics into Cassandra.

1. In collector application, there is one processor push mocked PulsarRawEvent into pipeline and go through collector->sessionization->distributor->MetricCalculator
   Pause traffic: http://HOST:8002/Event/Processor/EventGenerator/?pause
   Start traffic: http://HOST:8002/Event/Processor/EventGenerator/?start
2. In TwitterSample application, it will listen on the twitter messages and push the data into MetricCalculator directly.

Report

1. Realtime tab
   MetricCalculator does the run-time aggregation on several metrics with 10s interval and broadcast the metric data to MertricUI through Jetstream message. And then MertricUI aggregates on the metrics in its Esper again in 10s interval then broadcast the metrics to browser via websocket channel.

   MetricList you can refer to the EPL statement: http://HOST:8005/Event/Processor/MCEPL?$format=xml
   MC_Metric
   PulsarDeviceCount
   PulsarOsCount
   PulsarBrowserCount
   PulsarTopCountryCount

2. Batch tab
   MetricCalculator does the run-time aggregation on several metrics within a longer window(5 min), aggregation result are stored to a metric store and it's Cassandra. MertricUI will pull the metrics from Cassandra through MetricService. It supports drill down queries and also draw graphs on historical data.

3. Twitter Feed tab
   MetricCalculator doing the run time aggregation on several metric with 10s interval. The metric flow is the same as the Realtime tab.

   MetricList you can refer to the EPL statement: http://HOST:8005/Event/Processor/MCEPL?$format=xml
   TwitterTopCountryCount
   TwitterTopLangCount
   TwitterTopHashTagCount
   TwitterEventCount