WSDM 2021 Tutorial on Deep Learning for Anomaly Detection

Abstract

In this tutorial we aim to present a comprehensive survey of the advances in deep learning techniques specifically designed for anomaly detection (deep anomaly detection). Deep learning has gained tremendous success in transforming many data mining and machine learning tasks, but popular deep learning techniques are inapplicable to anomaly detection due to some unique characteristics of anomalies, e.g., rarity, heterogeneity, boundless nature, and prohibitively high cost of collecting large-scale anomaly data. Through this tutorial, audiences would gain a systematic overview of this area, learn the key intuitions, objective functions, underlying assumptions, advantages and disadvantages of 12 different categories of state-of-the-art deep anomaly detection methods, and recognize its broad real-world applicability in diverse domains. We also discuss what challenges the current deep anomaly detection methods can address and envision this area from multiple different perspectives.

Any audience who may be interested in deep learning, anomaly/outlier/novelty detection, out-of-distribution detection, representation learning with limited labeled data, and self-supervised representation learning would find it very helpful in attending this tutorial. Researchers and practitioners in finance, cybersecurity, healthcare would also find the tutorial helpful in practice.

Schedule and Materials

[Slides] [Video]

1. Overview of challenges and methods  (30 min)

• Introduction to anomaly detection

• Problems and challenges

• Deep vs. shallow methods

• Overview of deep anomaly detection approaches

Q & A (5 min)

2. Methods (100 min, including 10-minute break)

• The modeling perspective

+ Deep learning for feature extraction

+ Learning feature representations of normality

– Generic normality feature learning

∗ Autoencoder-based approaches

∗ Generative adversarial network-based approaches

∗ Predictability modeling approaches

∗ Self-supervised classification approaches

– Anomaly measure-dependent feature learning

∗ Feature learning for distance-based measure

∗ Feature learning for one-class classification measure

∗ Feature learning for clustering-based measure

+ End-to-end anomaly score learning

– Ranking models

– Prior-driven models

– Softmax likelihood models

– End-to-end one-class classification models

• Break

• The supervision information perspective

+ Unsupervised approach

+ Semi-supervised approach

+ Weakly-supervised approach

• Implementation and Evaluation

Q & A (5 min)

3. Conclusions and future opportunities (30 min)

• Summary of the methods

• Six possible directions for future research

+ Exploring new anomaly-supervisory signals

+ Deep weakly-supervised anomaly detection

+ Large-scale normality learning

+ Deep detection of complex anomalies

+ Interpretable and actionable deep anomaly detection

+ Novel applications and settings

Q & A (10 min)

Presenters

Dr. Guansong Pang obtained his PhD degree in Data Mining at the University of Technology Sydney in 2019. He is a Research Fellow in the Australian Institute for Machine Learning at the University of Adelaide, and an incoming Assistant Professor at Singapore Management University. His research interests lie in data mining, machine learning and their applications; he has been dedicating to the research on anomaly detection for over six years. He has published more than 25 papers (most of them are on (deep) anomaly detection) in refereed conferences and journals, such as KDD, AAAI, IJCAI, CVPR, ACM MM, ICDM, CIKM, IEEE Transactions on Knowledge and Data Engineering, and Data Mining and Knowledge Discovery Journal. He is one of the key presenters of the KDD17's tutorial on ``Non-IID Learning" and the KDD18's tutorial on ``Behavior Analytics: Methods and Applications". He also gives a number of oral representations of his papers at top conferences such as IJCAI16, IJCAI17, CIKM17, KDD18, KDD19 and invited talks at various universities.

Prof. Longbing Cao has been a full professor in information technology at UTS since 2009. He is the founding Editor-in-Chief of Springer’s Journal of Data Science and Analytics and associate EiC of IEEE Intelligent Systems. He serves as conference general chair such as for KDD2015, and program co-chair, area chair or vice-chair of conferences such as IJCAI, DSAA, PAKDD and ICDM, and SPC/PC member on over 100 conferences. He initiated and leads research on non-IID learning, behavior informatics, agent mining, and domain driven data mining, in addition to general issues in data science, data mining, machine learning, artificial intelligence and complex intelligent systems. He is one of the key presenters of a large number of tutorials at top conferences such as IJCAI 13, 19, 20; CIKM 14; KDD 17, 18; AAAI 18, 19; PAKDD 15, 18.

Dr. Charu Aggarwal completed his Ph.D. in Operations Research from the Massachusetts Institute of Technology in 1996. He has worked extensively in the field of data mining, with particular interests in data streams, privacy, uncertain data and social network analysis. He is a recipient of the IEEE ICDM Research Contributions Award (2015) and the ACM SIGKDD Innovation Award (2019), which are the two highest awards for research in the field of data mining. He has served as the general or program co-chair of the IEEE Big Data Conference (2014), the ICDM Conference (2015), the ACM CIKM Conference (2015), and the KDD Conference (2016). He is an editor-in-chief of the ACM Transactions on Knowledge Discovery and Data Mining , and has served as editor-in-chief of the ACM SIGKDD Explorations. He is a fellow of the IEEE (2010), ACM (2013), and the SIAM (2015) for ``contributions to knowledge discovery and data mining algorithms''. He is the sole author of the popular anomaly detection textbook ``Outlier Analysis''. He delivers a number of invited keynotes at various top conferences such as ECML 06, ASONAM 14, ECML 14 and SIGIR 18, and is one of the key presenters of several conference tutorials such as CIKM13 and SDM13 Tutorial on ``Outlier Detection in Temporal Data'' and ASONAM13 Tutorial on ``Outlier Detection in Graph Data''.

Key References

Pang, Guansong, et al. Deep learning for anomaly detection: A review. ACM Computing Survey 54, 2, Article 38 (March 2021), 38 pages. https://doi.org/10.1145/3439950. arXiv preprint.

Aggarwal, Charu. Outlier analysis. Springer (2017). 

Contact

Any questions can be sent to Guansong Pang (pangguansong@gmail.com)