Experts can feel most confident exactly when their predictions are least reliable

Daniel Kahneman and Amos Tversky named the 'illusion of validity' for the unwarranted confidence people feel when a pattern in their data looks coherent, even when that coherence has little to do with real predictive power. Kahneman first noticed it while rating army recruits for officer potential: his ratings felt rock-solid, yet barely predicted anything. The subjective feeling of being right and the objective odds of being right turned out to be almost unrelated.

— Daniel Kahneman and Amos Tversky, Judgment under Uncertainty: Heuristics and Biases — Science, 1974