Board index Other Hangar talk

The Dunning–Kruger effect

Talk about (almost) anything, as long as it is no serious FlightGear talk and does not fit in the other subforums.
Forum rules
Please refrain from discussing politics.

The Dunning–Kruger effect

Postby Johan G » Tue Jun 17, 2014 5:48 pm

Since this has been brought up, and since I do not want to start point fingers, here is some more information about it:

First off a starting point: The Wikipedia article Dunning–Kruger effect.

In their own words: Dunning, David; Kerri Johnson, Joyce Ehrlinger and Justin Kruger (2003). "Why people fail to recognize their own incompetence" (PDF). Current Directions in Psychological Science

A quote from the above text:
In many intellectual and social domains, the skills needed to produce correct responses are virtually identical to those needed to evaluate the accuracy of one’s responses. The skills needed to produce logically sound arguments, for instance, are the same skills that are necessary to recognize when a logically sound argument has been made. Thus, if people lack the skills to produce correct answers, they are also cursed with an inability to know when their answers, or anyone else’s, are right or wrong.

A very informative YouTube video:


Short, but very harsh, by John Clease about stupid people:
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Some YouTube videos
Johan G
Moderator
 
Posts: 6629
Joined: Fri Aug 06, 2010 6:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 2020.3.4
OS: Windows 10, 64 bit

Re: The Dunning–Kruger effect

Postby Lydiot » Tue Jun 17, 2014 6:21 pm

I suppose the irony would be that those who could use this knowledge might not agree with (read: comprehend) it...
Lydiot
 
Posts: 1016
Joined: Tue Oct 22, 2013 11:50 pm

Re: The Dunning–Kruger effect

Postby Johan G » Tue Jun 17, 2014 9:40 pm

Indeed, indeed... but, if approached in the right way it might be possible to help. From the pdf:
This double-curse explanation also suggests a crucial hypothesis: If poor performers are given the skills necessary to distinguish correct from incorrect answers, then they would be in a position to recognize their own incompetence.
...
Despite this paradox, we decided to put this hypothesis to the test (Kruger & Dunning, 1999, Study 4). In a first phase of the study, participants were tested on their ability to solve a certain type of logic problem. Not surprisingly, poor performers grossly overestimated their performance on the test. Then, in a second phase, we gave roughly half of the participants a mini-lecture about how to solve this type of logic problem, giving them the skills needed to distinguish accurate from inaccurate answers. When given their original test to look over, the participants who received the lecture, and particularly those who were poor performers, provided much more accurate self-ratings than they had originally.


The big question is, how do one approach them in a non-judging, friendly and encouraging way.
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Some YouTube videos
Johan G
Moderator
 
Posts: 6629
Joined: Fri Aug 06, 2010 6:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 2020.3.4
OS: Windows 10, 64 bit

Re: The Dunning–Kruger effect

Postby Lydiot » Tue Jun 17, 2014 11:02 pm

I find that as one (I) gets older one's (my) patience wanes. Unfortunately I think the ability to be anonymous on the internet as well as satisfy one's (other people's :D ) narcissism the probability of educating people just falls dramatically. I find that there's a type of person who is more interested in just getting attention which in conjunction with D-K is a recipe for willful ignorance. Add psychopathic trolls to that equation and...well...
Lydiot
 
Posts: 1016
Joined: Tue Oct 22, 2013 11:50 pm

Re: The Dunning–Kruger effect

Postby someguy » Wed Jun 18, 2014 4:14 am

Johan G wrote in Tue Jun 17, 2014 9:40 pm:The big question is, how do one approach them in a non-judging, friendly and encouraging way.


No, the big question isn't "how" but "why." As I grow older, I find myself more and more willing to allow natural selection to sort these things out. You can't help people who don't want to be helped. Reminds me of the old joke about bricking camels...

Look at the Middle East, for instance, where revenge evidently carries higher value than the survival of one's own children. Let them fight it out until all the stupid, testosterone-poisoned bigots kill each other off.
User avatar
someguy
 
Posts: 1650
Joined: Tue Nov 25, 2008 6:54 am
Location: USA
Version: 2019.1.1
OS: Mac OS X 10.11.6

Re: The Dunning–Kruger effect

Postby Johan G » Wed Jun 18, 2014 10:23 am

someguy wrote in Wed Jun 18, 2014 4:14 am:You can't help people who don't want to be helped.

True, true. That reminds me of one of those light bulb jokes: :wink:
Q: How many Psychiatrists does it take to change a light bulb?
A: Only one, but the bulb has got to really WANT to change....

Though this one is slightly more discouraging:
Q: How many psychiatrists does it take to change a light bulb?
A: None, the bulb will change itself when it is ready....


That aside I do not think it is impossible. :wink:
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Some YouTube videos
Johan G
Moderator
 
Posts: 6629
Joined: Fri Aug 06, 2010 6:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 2020.3.4
OS: Windows 10, 64 bit

Re: The Dunning–Kruger effect

Postby Jabberwocky » Wed Jun 18, 2014 4:40 pm

The real problem is not the impossibility to teach someone something, the real problem is the time factor.
IQ in large populations is distributed along a Gaussian bell curve. Which means the peak is per definition at 100. Now, there is for example a 90 percent percentile. Means 90% of all testants are in this range and it can be expected that they would produce to most common problems reasonable answers. But if you try, they don't. Why? Because the social imprint of high-speed information societies force them rather to a fast answer than a well-thought one. People are always in a hurry. That prevents them inadvertently from listening to logical arguments and from thinking through things before they spill an answer.
Now, and just as food for thought: One of the big thinking-things in the recent years was was the Black Swan theory. Basically a half mathematical explanation how unpredictable events can change the course of history. You can google it. The fun part is, the most used example for the Black Swan theory is 9/11. Allegedly nobody ever could predict, terrorists would hijack planes and fly them into high rises. I mean, as the promoters of the Black Swan theory argue, it was all too bizzare to be predicted this could ever happen, right?
Actually, novelists had used this kind of plots beginning with the 1980s. There were not less than five thrillers revolving about this kind of attacks. Only the guy who created the Black Swan theory is not a guy reading cheap novels. So he didn't know. Which actually proves the Black Swan theory, because it says implicitly, events are unpredictable because information is unknown.
So ... in doubt, take the time to think things through to the end. If Dunning and Kruger would have done that, they would maybe have found, the subject was already examined in Game Theory in the 90s and is known as unknown information problem as well. It's when a player has to make decisions based on incomplete information.

J.
Jabberwocky
Retired
 
Posts: 1316
Joined: Sat Mar 22, 2014 8:36 pm
Callsign: JWOCKY
Version: 3.0.0
OS: Ubuntu 14.04

Re: The Dunning–Kruger effect

Postby Lydiot » Wed Jun 18, 2014 5:26 pm

jabberwocky,

I don't think that's entirelly true. The difference is that some people with incomplete information will acknowledge that and downplay their own knowledge and "value", whereas others fall on the other end of things and overestimate themselves. This is something that lies outside of the actual knowledge about that which the person is initially commenting.

Using 9/11 as an example, if you go back and find someone saying it couldn't happen (before it did obviously) that person was clearly wrong, we know that now. But someone with an equal view on the probability of it happening could still have a different view on their own assessment. While both assert an extremely low probability the latter could in fact not be overestimating himself by saying that while unlikely, it is possible, and that he therefore doesn't know, whereas the former would simply assert the impossibility of it. So while both would be guessing incorrectly and know nothing about the actual 9/11 plot the latter would be an example of Dunning-Kruger as far as I see it.
Lydiot
 
Posts: 1016
Joined: Tue Oct 22, 2013 11:50 pm

Re: The Dunning–Kruger effect

Postby HJ1AN » Thu Jun 19, 2014 3:20 am

The John Cleese video...... :lol: just awesome.
User avatar
HJ1AN
 
Posts: 374
Joined: Sat Jul 25, 2009 5:45 am
Callsign: HJ888
Version: 3.4
OS: OS X

Re: The Dunning–Kruger effect

Postby Thorsten » Thu Jun 19, 2014 7:31 am

If Dunning and Kruger would have done that, they would maybe have found, the subject was already examined in Game Theory in the 90s and is known as unknown information problem as well. It's when a player has to make decisions based on incomplete information.


Well, as Lydiot pretty much nailed it - that's what probability theory is for. I don't factually know how the weather will be in Munich next year on 21st of August. It is entirely possible that it will be freezing rain. But if asked to commit to a decision now for what I promise to wear on that day, I would pick light summer clothes anyway, because the probability for summer weather is much higher than for freezing rain.

In game theory, faced with incomplete information, you have to adopt a probabilistic strategy where you choose certain courses of action with certain probabilities turn by turn. So we know how to solve that one, there's no mystery - we know what the best strategies in incomplete information situations are (we might not be able to correctly estimate the probabilities, but that's a different issue).

The problem in bringing this argument in online discussion is that people who don't know probability theory fail to recognize what it might be good for, argue that if I can't prove that X is absolutely right, adopting premise Y makes just as much sense even if X has a 99% chance of being right and Y a 1% chance, complain about all that useless techtalk and in general argue that it's all bullshit because after all that babble I still can't decide what's right and what's wrong.

Dunning-Kruger isn't about dealing with incomplete information, it's about the step before - recognizing the limits of your information. If you don't do that, you never adopt a strategy to deal with incomplete information because you hold the belief that you are in possession of complete information.

Which actually proves the Black Swan theory, because it says implicitly, events are unpredictable because information is unknown.


I don't need a line of math to figure that one out... but that's not the Black Swan thing.

In its core, the Black Swan thing is a criticism of economic modeling, in terms of models using the wrong probability distributions for risk assessment because then the math becomes simpler. So then you systematically underestimate rare events, believing that they might occur every 100.000 years while they actually occur every 100 years in the correct distribution.

However, natural scientists know these things... The probability to have a nuclear fusion reaction in a collision of two hydrogen isotopes in the sun is astronomically small. The overwhelming number of times nothing happens. But they collide so often and there are so many of them that in the end, the whole nuclear reaction is sustained from these tiny probabilities. So the tails of a distribution matter.

To argue that the unknown unknown can screw everything is a truism, but you can't do anything about it in your risk assessment. A Kamazozzupf event might destroy Earth tomorrow - but since we don't know what it is because it's a rare event and hasn't ever been witnessed before, so science knows nothing about it - how are we supposed to guard against it?

So there might be situations in which you can well be aware that you're not in the possession of all relevant information but still decide not to do anything about it because there's really no sensible course of action available - you just have to accept some risks. And that's very different from Dunning Kruger.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: The Dunning–Kruger effect

Postby Johan G » Thu Jun 19, 2014 11:19 am

Thorsten wrote in Thu Jun 19, 2014 7:31 am:Dunning-Kruger isn't about dealing with incomplete information, it's about the step before - recognizing the limits of your information. If you don't do that, you never adopt a strategy to deal with incomplete information because you hold the belief that you are in possession of complete information.

I do know practically nothing about game theory and have only a basic understanding of statistics (though I tend to see everything as being somewhere within a distribution anyway), but the above statement feels very right to me.

If you do know that there is an area of incomplete information you are also more likely being able to understand possible consequences at least of things in the edges of that unknown area and probably can interpolate what the consequences might be in the unknown area (an/or have an understanding for how big errors an estimate might have). I you do not know at the other hand...

When it comes to incompetence and war there is probably no one more iconic to quote than Sun Tzu (from the Project Gutenberg text):
III. ATTACK BY STRATAGEM
...
18. ... If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Some YouTube videos
Johan G
Moderator
 
Posts: 6629
Joined: Fri Aug 06, 2010 6:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 2020.3.4
OS: Windows 10, 64 bit

Re: The Dunning–Kruger effect

Postby Jabberwocky » Thu Jun 19, 2014 3:59 pm

Actually, Thorsten nailed it because the mistakes happen really in the step before. But to recognize information is incomplete, you would need a way to check your information for completeness and consistency. In my opinion, but please, that's only my opinion, the Dunning-Kruger effect is more likely caused by the wrongful perception that the information one bases his answer on is complete. Which in a way corresponds to the time factor problem because it means, nobody takes the time to check the own data.
ASn example for that was of course the 9/11 example in connection with the Black Swan theory. Because if the author of that theory would have taken the time to look up whether others used a similar plot, for example in novels, he maybe would have used another example. Because merely by logic, the glitch with that famous example discredits the example, it down't prove there are no Black Swan incidents at all, it only indicates they are really really rare.

Thorsten ahs a second interesting point there, when he wrote
In its core, the Black Swan thing is a criticism of economic modeling, in terms of models using the wrong probability distributions for risk assessment because then the math becomes simpler. So then you systematically underestimate rare events, believing that they might occur every 100.000 years while they actually occur every 100 years in the correct distribution.


But there is of course always a factor more. Once a low probability is recognized as an option, there will be, in Game theory, one player (or a few players in big games) start to bet on this "unpredictable" event. Because they hope to make an overproportional win (the big bet on an off-horse in horse races are an example) or just surprise an opponent (which is the common appearance in all strategical games includinf economical games). So, just alone by being extremely improbable, the probability of an option can increase and will if you have enough players.

However, Dunning-Kruger definitively can also be caused by arrogance and probably is often enough. I work with another non-profit that deals also with another special kind of statistical odd people, aka serial killers.
Now, there are without doubt more people looking for serial killers than actual serial killers. Count just the thousands of thousands of law enforcement officers. And there were a lot of studies about the subject. Still, there is no week, I don't hear lines like there are no female serial killers, there are no black serial killers, no yellow, no green and for sure there are never ever two active at one time in one area. There are for sure more than enough examples to the contrary (except for the green ones), but when confronted with such possibilities, people fall silent for a minute and then they repeat the same lines like a mantra with the added argument, they have let so many cases go cold already in the last so and so many years. Would that count as Extreme Dunning-Kruger?
Jabberwocky
Retired
 
Posts: 1316
Joined: Sat Mar 22, 2014 8:36 pm
Callsign: JWOCKY
Version: 3.0.0
OS: Ubuntu 14.04

Re: The Dunning–Kruger effect

Postby Lydiot » Thu Jun 19, 2014 4:47 pm

Jabberwocky wrote in Thu Jun 19, 2014 3:59 pm: In my opinion, but please, that's only my opinion, the Dunning-Kruger effect is more likely caused by the wrongful perception that the information one bases his answer on is complete. Which in a way corresponds to the time factor problem because it means, nobody takes the time to check the own data.


I disagree. I think you're putting the cart before the horse. The people to whom D-K applies are the ones that will ignore further evidence simply because it is unnecessary. Why is it unnecessary? Because they already "know" the truth.

Now, you could of course say that some people are not like that and that applying D-K would be wrong because they would change their mind about an event after some time with additional information. However, I would propose that the people to whom the term applies will repeat that mistake whereas those to whom it doesn't apply will learn from the fact that they once made a wrong conclusion about their assertion based on incomplete information and in the future be more modest when making assertions or wait for more information.

Jabberwocky wrote in Thu Jun 19, 2014 3:59 pm:Once a low probability is recognized as an option, there will be, in Game theory, one player (or a few players in big games) start to bet on this "unpredictable" event. Because they hope to make an overproportional win (the big bet on an off-horse in horse races are an example) or just surprise an opponent (which is the common appearance in all strategical games includinf economical games). So, just alone by being extremely improbable, the probability of an option can increase and will if you have enough players.


But that depends on what "game" is being "played". If we're talking about a battle between different people where there are winners and losers, particularly if the sum is zero, then of course you have a point. If we're looking at the stock market for example and are saying that a certain stock is undervalued because its value depends on some unlikely event, and people start betting in its favor, it could very well be that the likelihood increases.

However, if we're talking about physics that are outside of our control it doesn't matter what our opinions are, facts won't change. And this thread I think should be viewed in that context. So if we're talking about D-K in that context the issue is not that the person to whom it is applied by others is potentially not subject to it for statistical reasons - because statistics don't change the facts that the person asserts his knowledge about. So if we're talking about programming Nasal there are right and wrong answers. There are no statistics that mitigate whether or not a person is exhibiting D-K in that context.

Jabberwocky wrote in Thu Jun 19, 2014 3:59 pm:Still, there is no week, I don't hear lines like there are no female serial killers, there are no black serial killers, no yellow, no green and for sure there are never ever two active at one time in one area. There are for sure more than enough examples to the contrary (except for the green ones), but when confronted with such possibilities, people fall silent for a minute and then they repeat the same lines like a mantra with the added argument, they have let so many cases go cold already in the last so and so many years. Would that count as Extreme Dunning-Kruger?


In my opinion, 'yes', it would. Because there is an absolute truth out there and they're ignoring the information they're given in favor of perpetuating their own bias.

PS: I did respond to your previous post before Thorsten and we were pretty much saying the same thing in different words.
Lydiot
 
Posts: 1016
Joined: Tue Oct 22, 2013 11:50 pm

Re: The Dunning–Kruger effect

Postby someguy » Thu Jun 19, 2014 5:03 pm

The serial killer example is a common case of willful ignorance. Choosing to discard the outliers makes their already-simple lives even simpler. There's comfort in familiar dogma (viz., the military, Fox News, any religion), whereas critical thinking involves actual work...and humans tend to be lazy. There are always more followers than leaders.

Some people really are stupid, of course, but I think D-K is more often a matter of slothful choice, an unwillingness to tear oneself away from Twitbook or "reality" TV long enough to understand how the world really works. "I've already made up my mind, so don't confuse me with the facts."
User avatar
someguy
 
Posts: 1650
Joined: Tue Nov 25, 2008 6:54 am
Location: USA
Version: 2019.1.1
OS: Mac OS X 10.11.6

Re: The Dunning–Kruger effect

Postby Lydiot » Thu Jun 19, 2014 5:35 pm

someguy wrote in Thu Jun 19, 2014 5:03 pm:There are always more followers than leaders.


Well, by definition that's true, a "truism" if you will. But I don't think that necessarily has to do with what you imply.
Lydiot
 
Posts: 1016
Joined: Tue Oct 22, 2013 11:50 pm

Next

Return to Hangar talk

Who is online

Users browsing this forum: No registered users and 5 guests