Learn in the textbookNeural Networks and Deep Learning
DefinitionAn activation function (活性関数, also 活性化関数) is a function that converts the total of the values a single neuron receives into the value it sends to the next layer.
ExplanationA neuron multiplies each value arriving from the previous layer by its weight, adds them up, and runs the total through the activation function before handing it on. Some functions send a signal only when the total reaches a threshold, while others turn it into a smooth value between 0 and 1. Because the activation function performs a non-linear conversion, a network of stacked layers can represent complex relationships.
AnalogyLike each person's own habit of judging how strong an incoming message is and then deciding whether to pass it on, and how forcefully.
Memory hookA biological nerve cell "activates" (fires) and sends a signal once a stimulus passes a certain level. The function mimics that behavior, hence the name.