A data scientist (データサイエンティスト) is a specialist who analyzes large volumes of data and extracts insights the business can use.
Analytical skill alone is not enough, because reading what the numbers mean takes business knowledge too. With sales data, for instance, statistics will not help you ask useful questions unless you also know how that business works.
AnalogyThink of a detective who gathers clues at the scene, forms a theory, and walks the client through it with the evidence.
How to remember"Scientist" is right there in the name: the job means forming hypotheses and testing them.
Exam points
ExamA data scientist (データサイエンティスト) uses statistics and machine learning (機械学習) as tools to analyze large data sets and look for leads to new value.
Mix-upBuilding database (データベース) infrastructure and supporting the security side of information systems (情報システム) are separate from the core of this job.
A decision tree (デシジョンツリー, also called 決定木) is a method that shows how decisions and outcomes split, in the form of a branching diagram.
Launch a new service?
Launch
Many users: probability 0.7, profit ¥1,000,000
Few users: probability 0.3, loss ¥500,000
Do not launch: profit ¥0
Multiplying each amount by its probability and adding (a loss is subtracted) gives 1,000,000×0.7−500,000×0.3=¥550,000 for launching, more than the ¥0 for not launching.
3 / 3
Follow the conditions or options at each fork and the conclusion appears at a leaf, the end of the branch. Put probabilities on the branches and you can compare options by expected value, the sum of each result multiplied by its probability. Machine learning also uses this method to classify data step by step.
AnalogyIt works like following signposts: you read one at every fork, and where you end up is your answer.
How to rememberThe name is just "decision" plus "tree", and calling the endpoints "leaves" extends the same image.
Exam points
ExamWhen you expect the other side to pick the move that hurts you most, work backward from the end values, keeping the minimum on their turns and the maximum on yours.
ExamCompare options that carry a cost by their expected value (期待値) minus that cost.
Mix-upA decision table (デシジョンテーブル) is a grid that matches each combination of conditions being met or not to a result, so its format differs from a branching diagram.
Modeling (モデル化) means choosing only the elements your purpose requires and representing a real-world subject in simple form with formulas or diagrams.
Decide what to find out
Choose the needed elements
Express relations in formulas or diagrams
Check against reality and fix
If the model's results drift from reality, go back as far as the choice of elements and fix it.
4 / 4
The real thing is too complex to handle as it is, and both analysis and experiments would take too much effort. So you narrow down the elements, accepting that whatever you leave out remains as a gap from reality. Whether to include chance is also a design decision made at this stage.
AnalogyPicture a subway map, which drops distance and terrain and keeps only the order of stations and the transfers.
How to rememberIt is the same "model" as in a plastic model kit, and the word originally meant a scale replica.
Exam points
ExamWork done at a constant pace can be modeled as a straight line relating time to the progress rate (進捗率). If you assume each person's output simply adds up, the slope when working together is the sum of the individual slopes.
A deterministic model (確定モデル) contains no element of chance, so the same input always produces the same result.
Assume, for example, that 30 items sell at ¥200 each, and the sales figure comes out as ¥6,000 with no other possibility. Whether 30 actually sell is another matter, so a gap from reality can remain.
Exam points
Mix-upHowever complex the formula, a model with no element of chance is still a deterministic model (確定モデル). The dividing line is not how hard the calculation is.
A stochastic model (確率モデル) builds in chance through probabilities and expresses results as a distribution or an expected value.
Customers per day
Probability
Customers × probability
10
0.1
1
20
0.6
12
30
0.3
9
Expected value
Total 1.0
22
It suits quantities that vary from one occasion to the next, such as customer numbers or machine failures. Listing how likely each value is gives a probability distribution, and from that you can work out the probability-weighted average called the expected value.
AnalogyIt is like a weather forecast that gives a chance of rain instead of declaring exactly what tomorrow will bring.
Exam points
ExamTo cap the chance of a stockout (品切れ), add up probabilities from the high-demand end; the stock (在庫) you need is the smallest amount that keeps the chance of demand exceeding it below the cap.
ExamThe option with the largest expected value is not always the one with the largest top amount.
Simulation (シミュレーション) means running a model to test what happens when conditions change, without using the real thing.
Prepare a model
Change the conditions
Calculate for each condition
Compare the results
Compare the waiting times across conditions and use them to decide how many registers to run.
4 / 4
Even when testing on the real thing would be costly or dangerous, a model lets you start over as often as you like. Reproducing variation with random numbers is called the Monte Carlo method (モンテカルロ法). How far you can trust the results depends on how good the model is.
AnalogyIt is like a rehearsal, where you try several run-throughs before the real event to find the arrangement that works.
How to rememberTo simulate something is, at root, to imitate it.
Exam points
ExamOne good fit for simulation (シミュレーション) is a problem where you make assumptions about future conditions and estimate the results that follow.
ExamA problem where plugging numbers into task-order relationships or a fixed formula yields a single answer is not a typical use of simulation.
Data assimilation (データ同化) is a method that feeds observations into a running model to bring the model's state closer to reality.
Calculate with the model
Compare with observations
Correct the model's state
Continue from the corrected state
Continue calculating from the corrected state, and keep repeating the cycle.
4 / 4
Every model differs from reality, and the gap tends to build up as the calculation runs. Feeding in observations as they arrive keeps that drift small. Weather forecasting is the classic application.
AnalogyThink of a car navigation system that keeps receiving satellite positioning data while you drive and keeps correcting your location on the map.
How to rememberIn biology, assimilation turns what an organism takes in into part of itself, much as observations become part of the model here.
Simulation vs. Data Assimilation
Aspect
Simulation
Data Assimilation
Key differenceMain function
Run a model to test results
Correct a running model with observations
Required input
A model and conditions
A model and observations
When to use
Comparing conditions that are hard to test on the real thing
Tracking phenomena where observations accumulate over time
Prediction (予測) means estimating future values or states from the information at hand, such as past data or a model.
Example method
Clue
How it works
Regression analysis
Past input and output data
Derive the input-output relation statistically
Model-based estimate
A model and assumptions about the future
Calculate results for each condition
Delphi method
Expert opinions
Repeat surveys and share the results
The methods are not limited to statistics and model calculations; gathering expert opinions is another. Whichever you use, an estimate can still turn out wrong.
AnalogyPicture a hiker working out when they will reach the summit from their pace so far and the distance left.
How to rememberThe kanji in 予測 literally say "measure in advance": you measure what cannot be measured yet.
Exam points
ExamThe input to prediction (予測) is not limited to past results. Assumptions about how conditions will change can be used as well.
ExamFermi estimation (フェルミ推定) gives a rough figure for a quantity that is hard to count directly, using known numbers and step-by-step reasoning. It is not limited to future quantities.
Mix-upThe Delphi method (デルファイ法) repeatedly surveys experts to find agreement on the future, while regression analysis (回帰分析) derives the input-output relation from data statistically. Neither is Fermi estimation.
A data scientist analyzes large volumes of data to extract insights the business can use.
A decision tree puts probabilities and amounts on its branches to compare options by expected value; when an opponent is involved, work back from the ends to choose your move.
A model is a simplification that keeps only what the purpose requires, so a gap from reality remains. Data assimilation narrows that gap with observations.
Simulation is an experiment that changes conditions on a model to test the results, so it can compare even options that are hard to try on the real thing.
The methods of prediction are not limited to model calculations; gathering expert opinions is another.