One of the most striking differences between Machine Learning and Human Learning is revealed by examination of the cumulated error curve, for many distinct instances of any specific algorithm, on any specific rule. As is clear in this image, the ML cumulated error curve gradually becomes flatter, and will eventually have an error rate that is in some sense negligible. This is typical of search behavior, as more and more "wrong possibilities" are eliminated.
For human learning (HL) on the other hand, there are two distinct qualitative behaviors. They are shown here for experiments on the rule "objects must be placed in the nearest bucket." The two colors indicate whether the subject encounters this rule first, or after having already dealt with another rule. One class of subjects seem to accumulate errors steadily over time, as shown in the figure. The rate of accumulation may vary, depending on how efficient the subject's search heuristic is. In other words, they are always searching, and never finding.
The other class exhibit what seems to be "understanding" with, at some point converting from the steady accumulation of errors to a relatively flat (error-free) performance. We take the beginning of this flat portion to the the "aha" moment at which the rule is discovered. For many rules the probability of a correct move, at random, is 25%. With this in mind, our analyses often use "10 good moves in a row" as the indication that the subject has understood the rule. For a random player, the chance of such a string is one in a million. [This is somewhat reduced because a typical player may make as many as 100 moves when trying to solve the game, and thus has as many as 90 chances to start the sequence of 10 good moves. The probability still remains negligible. In a study with thousands of subject-rule combination there will perhaps be a few cases in which the string of good moves occurs for a subject who has not understood the rule.
We note that for underspecified or ambiguous rules, the probability of 10 good moves in a row becomes substantially higher, as the probability of making a random error is less than 75% for some of the moves.
Cite as: Kantor, Paul; Bharti, Shubham. "Contrast of Human and Machine Learning." URL: https://wwwtest.rulegame.wisc.edu/ExampleFindings/MLvsHL.html
Roles. Analysis, Visualization and Human data by PK; Machine data computed by SB.
