Abstracts Category : Other

A Real-time Temporal Clustering Algorithm for short text, and its applications

by Nishant Agarwal

Real-world events of general interest trigger engaging discussions among peoplefor short bursts in time. These events then either evolve, merge or slowly lose publicinterest as time goes by. With the advent of social media like Twitter, people have aplatform to voice opinions, share news and make announcements, some of which gainmass popularity. Leveraging social media for early detection of these events, has thusbecome a problem of immense practical value for news media companies and peoplewho want to be aware about the world in general.In this thesis, we present the foundations of a system that runs on a real-time,dynamic, temporal clustering algorithm, that is capable of detecting bursts in streamingdata, as they happen in time. By using efficient data structures, we not only detect thesebursty topics, but also store them efficiently, to facilitate many independent applicationssuch as hashtag recommendations and community detection, which would requires acompletely different approaches, if handled separately. An efficient storage mechanismalso enables us to perform evolutionary queries on the discovered themes, to get a greaterinsight. We also extend this research on social media, to user behavioral analysis, usingtheir database access logs, to find bursty patterns, anomaly detection and related tasks.Lastly, we perform an in-depth evaluation to show that our model outperforms manypopular approaches in terms of topic quality and hashtag precision by more than 15%.