DETECTING ABNORMAL DISTRIBUTION USING CUSTOM ACTIVATION METHODS

Bui Duc Tho; Nguyen Thu Ha; Pham Thi Thanh Hang; Dang Viet Hung

Bui Duc Tho Hung Yen University of Technology and Education
Nguyen Thu Ha Hung Yen University of Technology and Education
Pham Thi Thanh Hang Hung Yen University of Technology and Education
Dang Viet Hung Hung Yen University of Technology and Education

Keywords: out of distribution data, activation function, neural network

Abstract

Out-of-distribution (OOD) discovery is an important challenge in the fields of artificial intelligence. In this paper, we introduce a new method called “Activation Shaping” (ASH), an activation function customization method, to improve the detection of OOD data. ASH allows the activation function to be adjusted based on each data sample, creating a flexible non-linear feature representation. We will present how ASH works, its experimental results, and its potential applications in broad application areas. The results show that ASH has the potential to significantly improve out-of-range data detection performance and provide a basis for the research of customized activation functions in the field of artificial intelligence.

References

Djurisic, Andrija, et al. “Extremely simple activation shaping for out-of-distribution detection.” arXiv preprint arXiv:2209.09858, 2022.

Yang, J., Wang, P., Zou, D., Zhou, Z., Ding, K., Peng, W., ... & Liu, Z., Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems, 2022, 35, pp. 32598-32611.

Liu, Weitang, et al. “Energy-based out-of-distribution detection.” Advances in neural information processing systems, 2020, 33, pp. 21464-21475.

Sharma, Sagar, Simone Sharma, and Anidhya Athaiya. “Activation functions in neural networks.” Towards Data Sci 6.12, 2017, pp. 310-316.

Z. Shen et al., “Towards out-of-distribution generalization: A survey,” arXiv Prepr. arXiv2108.13624, 2021.

P. Cui and J. Wang, “Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review,” Electronics, 2022, vol. 11, no. 21, p. 3500.