A neural network (ニューラルネットワーク) is a learning model that links units of computation in layers, imitating how the brain's nerve cells (neurons) connect to one another.
Input layer (receives data)
Middle layer (hidden layer)
Output layer (produces the result)
Data enters at the input layer, passes through the middle layer (hidden layer), and comes out of the output layer as a result. Each line connecting one layer to the next carries a number called a weight, which sets how strongly a given input is passed on. Learning means adjusting these weights until the network produces results that fit the data.
AnalogyPicture a line of people playing telephone, where each person decides how much weight to give the previous speaker's message before passing it on, and those choices are tuned bit by bit until the last person says the right answer.
How to remember"Neural" means "relating to nerves" and a network is a web, so the name reads simply as "a web of nerves."
Exam points
ExamA neural network (ニューラルネットワーク) turns the workings of the brain's neural circuits into a mathematical model reproduced on a computer, and it is the technology underlying deep learning (ディープラーニング).
Backpropagation (バックプロパゲーション, error backpropagation) is a learning method that traces the error between the output and the correct answer backward, from the output-layer side toward the input-layer side, correcting the weight of each connection along the way.
Compute in order from input to output
Find the gap between output and correct answer
Pass the gap backward from the output side
Correct each connection's weight
The weights are adjusted in the direction that shrinks the gap, and the cycle repeats many times.
4 / 4
The calculation itself runs forward from the input layer to the output layer, but the information used for correction travels from back to front. Layer by layer, the method works out how much each connection contributed to the gap, and weights with more influence get larger corrections. Repeating this over a large amount of data brings the output steadily closer to the correct answer.
AnalogyLike tasting a dish, finding it too salty, and working back from the last seasoning you added to decide which ones to cut and by how much.
How to rememberSplit the name into "back" and "propagation" (spreading along) and you get "the gap spreads backward," so you won't mix up the direction.
Exam points
ExamWhat backpropagation (バックプロパゲーション) updates is the weight (重み) of each connection, and its aim is to shrink the gap between the output and the correct answer.
An activation function (活性関数, also 活性化関数) is a function that converts the total of the values a single neuron receives into the value it sends to the next layer.
Example activation function
How it converts
Step function
Sends 1 at or above a threshold, 0 below it
Sigmoid function
Sends a smooth value between 0 and 1
ReLU
Sends 0 for values of 0 or less, passes positive values unchanged
A neuron multiplies each value arriving from the previous layer by its weight, adds them up, and runs the total through the activation function before handing it on. Some functions send a signal only when the total reaches a threshold, while others turn it into a smooth value between 0 and 1. Because the activation function performs a non-linear conversion, a network of stacked layers can represent complex relationships.
AnalogyLike each person's own habit of judging how strong an incoming message is and then deciding whether to pass it on, and how forcefully.
How to rememberA biological nerve cell "activates" (fires) and sends a signal once a stimulus passes a certain level. The function mimics that behavior, hence the name.
Exam points
ExamAn activation function (活性関数) works inside each individual neuron, calculating the value to send onward from the values that neuron received.
Mix-upAn activation function is the conversion rule inside each neuron, whereas backpropagation (バックプロパゲーション) is the procedure that corrects the weights each time the network learns.
Overfitting (過学習) is a state in which a model fits the training data used for learning so closely that its predictions on unseen data become less accurate.
Learning state
Accuracy on training data
Accuracy on unseen data
Underfitting
Low
Low
Just right
High
High
Overfitting
Very high
Low
When a model memorizes even the chance quirks and noise in the training data, its grasp of general rules that hold for new data (generalization) weakens. You can spot it when accuracy is high on the training data but drops on data that was not used for learning. Ways to curb it include adding more training data and making the model simpler.
AnalogyLike a suit tailored so tightly to one person that it fits them perfectly but fits no one else at all.
How to rememberThe Japanese term 過学習 literally means "excess learning," and the English prefix "over-" says the same thing: the failure comes from going too far, not from learning too little.
Exam points
Mix-upWith overfitting (過学習), accuracy on the training data (訓練データ) is actually high; what drops is accuracy on unseen data (未知のデータ). When both are low, the state is called underfitting (学習不足).
Deep learning (ディープラーニング) is a machine learning method that uses a neural network with many stacked middle layers and lets the computer find the features of the data by itself.
Input layer (image pixels)
Shallow middle layers (lines and dots)
Deep middle layers (parts of shapes)
Output layer (decides what the image shows)
The output layer uses the gathered features to decide what the picture shows.
4 / 4
Deepening the layers creates a division of labor: shallow layers capture simple features, and deeper layers combine them into complex ones. As a result, features can be pulled out of large amounts of data automatically, without people spelling out in detail where to look. It delivers high performance in image recognition, speech recognition, translation, and similar tasks.
AnalogyLike learning to read Chinese characters: first the strokes, then the radicals, and finally the whole character, building up the ability to tell them apart one stage at a time.
How to remember"Deep" refers not to how clever the system is, but to how deep (how many) its stacked layers are.
Exam points
ExamDeep learning (ディープラーニング) is an AI technology in which a multilayer neural network (ニューラルネットワーク) lets the computer extract and learn the features of data on its own, without people specifying them.
ExamDeep learning can find patterns in large stores of accumulated data and put them to use in forecasting and decision-making.
Mix-upA neural network refers to the structure of a brain-inspired model, while deep learning refers to the method of stacking its middle layers deeply and training it.