Trending Event Detection using Anomaly Detection for COVID-19 Twitter chatter
What are the most frequent words on Twitter related to covid?
The top 20 words ranked by frequency are ‘coronavirus’, ‘covid’, ‘covid19’, ‘trump’, ‘cases’, ‘people’, ‘amp’, ‘new’, ‘vaccine’, ‘via’, ‘pandemic’, ‘polmascegahcovid19’, ‘siagalawancovid19’, ‘bolsonaro’, ‘deaths’, ‘lockdown’, ‘china’, ‘dalam’, ‘dan’, ‘virus’.
What does the trend of “covid” on Twitter chatter look like?
Here we cannot find the data point that we believe should be an anomaly
What does the trend of the “vaccine” on Twitter chatter look like?
For ‘vaccine’, we can find the data point that we believe should be an anomaly.
What does the trend of the “death” on Twitter chatter look like?
What does the trend of the “trump” on Twitter chatter look like?
When did the trending of the “vaccine” happen?
In this part, we implemented the anomaly detection algorithm mentioned in this paper: Automatic Anomaly Detection in the Cloud Via Statistical Learning.
This paper introduced a few popular anomaly detection algorithms from Twitter (we also talked through them during the class) and they are also widely used in industries. For our project, we implemented the Seasonal Hybrid ESD (Extreme Studentized Deviate) algorithm mentioned in the paper using the Seasonal Hybrid ESD algorithm.
When did the trending of the “world healthy day” happen?
We have the trending keywords for each date. Below you can find a real trending event, we can review the trending keywords and see we can find some signal from them for the same date.
April 7 is the World Health Day. The signal detected matches the event.