Scientific Research Publishing

Estimating the Rectified Linear Unit Activation Function in Deep Learning Using the Grouping-Adjusted Median Estimator: An Evaluation of Max Pooling ()

The Rectified Linear Unit (ReLU) activation function is widely employed in deep learning (DL). ReLU shares structural similarities with censored regression and Tobit models common in econometrics and ...
Scientific Research Publishing

Pedamonti, D. (2018) Comparison of Non-Linear Activation Functions for Deep Neural Networks on MNIST Classification Task. arXiv.

ABSTRACT: Artificial deep neural networks (ADNNs) have become a cornerstone of modern machine learning, but they are not immune to challenges. One of the most significant problems plaguing ADNNs is ...
GitHub

[BUG] linear activation and activation_hidden in SimpleRNNRegressor

RNN regressor currently has linear for both hidden and final layer activations, which essentially defeats the purpose of using a neural network and reduces the whole setup to linear regression. If you ...
IEEE

Reconfigurable Nonlinear Activation Functions by Add-Drop Resonator Mach–Zehnder Interferometers

Abstract: We propose and experimentally demonstrate a reconfigurable nonlinear activation function (NAF) unit based on add-drop resonator Mach-Zehnder interferometers (ADRMZIs) for photonic neural ...
IEEE

Flex-SFU: Activation Function Acceleration with Non-Uniform Piecewise Approximation

Abstract: Modern Deep Neural Networks (DNN) increasingly use activation functions with computationally complex operations. This creates a challenge for current hardware accelerators, which are ...
marktechpost

University of South Florida Researchers Propose TeLU Activation Function for Fast and Stable Deep Learning

Inspired by the brain, neural networks are essential for recognizing images and processing language. These networks rely on activation functions, which enable them to learn complex patterns. However, ...