DETECTING ABNORMAL DISTRIBUTION USING CUSTOM ACTIVATION METHODS
Abstract
Out-of-distribution (OOD) discovery is an important challenge in the fields of artificial intelligence. In this paper, we introduce a new method called “Activation Shaping” (ASH), an activation function customization method, to improve the detection of OOD data. ASH allows the activation function to be adjusted based on each data sample, creating a flexible non-linear feature representation. We will present how ASH works, its experimental results, and its potential applications in broad application areas. The results show that ASH has the potential to significantly improve out-of-range data detection performance and provide a basis for the research of customized activation functions in the field of artificial intelligence.
References
Djurisic, Andrija, et al. “Extremely simple activation shaping for out-of-distribution detection.” arXiv preprint arXiv:2209.09858, 2022.
Yang, J., Wang, P., Zou, D., Zhou, Z., Ding, K., Peng, W., ... & Liu, Z., Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems, 2022, 35, pp. 32598-32611.
Liu, Weitang, et al. “Energy-based out-of-distribution detection.” Advances in neural information processing systems, 2020, 33, pp. 21464-21475.
Sharma, Sagar, Simone Sharma, and Anidhya Athaiya. “Activation functions in neural networks.” Towards Data Sci 6.12, 2017, pp. 310-316.
Z. Shen et al., “Towards out-of-distribution generalization: A survey,” arXiv Prepr. arXiv2108.13624, 2021.
P. Cui and J. Wang, “Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review,” Electronics, 2022, vol. 11, no. 21, p. 3500.
