Log in

No account? Create an account
Epistemic learned helplessness - Jackdaws love my big sphinx of quartz Page 2 [entries|archive|friends|userinfo]

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Epistemic learned helplessness [Jan. 3rd, 2013|01:10 am]
[Tags|, , ]

[Epistemic Status | Probably I'm just coming at the bog-standard idea of compartmentalization from a different angle here. I don't know if anyone else has noted how compartmentalization is a good thing before, but I bet they have.]

A friend in business recently complained about his hiring pool, saying that he couldn't find people with the basic skill of believing arguments. That is, if you have a valid argument for something, then you should accept the conclusion. Even if the conclusion is unpopular, or inconvenient, or you don't like it. He told me a good portion of the point of CfAR was to either find or create people who would believe something after it had been proven to them.

And I nodded my head, because it sounded reasonable enough, and it wasn't until a few hours later that I thought about it again and went "Wait, no, that would be the worst idea ever."

I don't think I'm overselling myself too much to expect that I could argue circles around the average high school dropout. Like I mean that on almost any topic, given almost any position, I could totally demolish her and make her look like an idiot. Reduce her to some form of "Look, everything you say fits together and I can't explain why you're wrong, I just know you are!" Or, more plausibly, "Shut up I don't want to talk about this!"

And there are people who can argue circles around me. Not on any topic, maybe, but on topics where they are experts and have spent their whole lives honing their arguments. When I was young I used to read pseudohistory books; Immanuel Velikovsky's Ages in Chaos is a good example of the best this genre has to offer. I read it and it seemed so obviously correct, so perfect, that I could barely bring myself to bother to search out rebuttals.

And then I read the rebuttals, and they were so obviously correct, so devastating, that I couldn't believe I had ever been so dumb as to believe Velikovsky.

And then I read the rebuttals to the rebuttals, and they were so obviously correct that I felt silly for ever doubting.

And so on for several more iterations, until the labyrinth of doubt seemed inescapable. What finally broke me out wasn't so much the lucidity of the consensus view so much as starting to sample different crackpots. Some were almost as bright and rhetorically gifted as Velikovsky, all presented insurmountable evidence for their theories, and all had mutually exclusive ideas. After all, Noah's Flood couldn't have been a cultural memory both of the fall of Atlantis and of a change in the Earth's orbit, let alone of a lost Ice Age civilization or of megatsunamis from a meteor strike. So given that at least some of those arguments are wrong and all seemed practically proven, I am obviously just gullible in the field of ancient history. Given a total lack of independent intellectual steering power and no desire to spend thirty years building an independent knowledge base of Near Eastern history, I choose to just accept the ideas of the prestigious people with professorships in Archaeology rather than the universally reviled crackpots who write books about Venus being a comet.

I guess you could consider this a form of epistemic learned helplessness, where I know any attempt to evaluate the arguments are just going to be a bad idea so I don't even try. If you have a good argument that the Early Bronze Age worked completely differently from the way mainstream historians believe, I just don't want to hear about it. If you insist on telling me anyway, I will nod, say that your argument makes complete sense, and then totally refuse to change my mind or admit even the slightest possibility that you might be right.

(This is the correct Bayesian action, by the way. If I know that a false argument sounds just as convincing as a true argument, argument convincingness provides no evidence either way, and I should ignore it and stick with my prior.)

I consider myself lucky in that my epistemic learned helplessness is circumscribed; there are still cases where I will trust the evidence of my own reason. In fact, I trust it in most cases other than very carefully constructed arguments known for their deceptiveness in fields I know little about. But I think the average high school dropout both doesn't and shouldn't. Anyone anywhere - politicians, scammy businessmen, smooth-talking romantic partners - would be able to argue her into anything. And so she takes the obvious and correct defensive manuever - she will never let anyone convince her of any belief that sounds "weird" (note that, if you grow up in the right circles, beliefs along the lines of astrology not working sound "weird".)

This is starting to sound a lot like ideas I've already heard centering around compartmentalization and taking ideas seriously. The only difference between their presentation and mine is that I'm saying that for 99% of people, 99% of the time, this is a terrible idea. Or, at the very least, this should be the last skill you learn, after you've learned every other skill that allows you to know which ideas are or are not correct.

The people I know who are best at taking ideas seriously are those who are smartest and most rational. I think people are working off a model where these co-occur because you need to be very clever to fight your natural and detrimental tendency not to take ideas seriously. I think it's at least possible they co-occur because you have to be really smart in order for taking ideas seriously to be even not-immediately-disastrous. You have to be really smart not to have been talked into enough terrible arguments to develop epistemic learned helplessness.

Even the smartest people I know have a commendable tendency not to take certain ideas seriously. Bostrom's simulation argument, the anthropic doomsday argument, Pascal's Mugging - I've never heard anyone give a coherent argument against any of these, but I've also never met anyone who fully accepts them and lives life according to their implications.

A friend tells me of a guy who once accepted fundamentalist religion because of Pascal's Wager. I will provisionally admit that this person takes ideas seriously. Everyone else loses.

Which isn't to say that some people don't do better than others. Terrorists seem pretty good in this respect. People used to talk about how terrorists must be very poor and uneducated to fall for militant Islam, and then someone did a study and found that they were disproportionately well-off, college educated people (many were engineers). I've heard a few good arguments in this direction before, things like how engineering trains you to have a very black-and-white right-or-wrong view of the world based on a few simple formulae, and this meshes with fundamentalism better than it meshes with subtle liberal religious messages.

But to these I would add that a sufficiently smart engineer has never been burned by arguments above his skill level before, has never had any reason to develop epistemic learned helplessness. If Osama comes up to him with a really good argument for terrorism, he thinks "Oh, there's a good argument for terrorism. I guess I should become a terrorist," as opposed to "Arguments? You can prove anything with arguments. I'll just stay right here and not do something that will get me ostracized and probably killed."

Responsible doctors are at the other end of the spectrum from terrorists in this regard. I once heard someone rail against how doctors totally ignored all the latest and most exciting medical studies. The same person, practically in the same breath, then railed against how 50% to 90% of medical studies are wrong. These two observations are not unrelated. Not only are there so many terrible studies, but pseudomedicine (not the stupid homeopathy type, but the type that links everything to some obscure chemical on an out-of-the-way metabolic pathway) has, for me, proven much like pseudohistory in that unless I am an expert in that particular field of medicine (biochemistry has a disproportionate share of these people and is also an area where I'm weak) it's hard not to take them seriously, even when they're super-wrong.

I have developed a healthy dose of epistemic learned helplessness, and the medical establishment offers a shiny tempting solution - first, a total unwillingness to trust anything, no matter how plausible it sounds, until it's gone through an endless cycle of studies and meta-analyses, and second, a bunch of Institutes and Collaborations dedicated to filtering through all these studies and analyses and telling you what lessons you should draw from them. Part of the reason Good Calories, Bad Calories was so terrifying is that it made a strong case that this establishment can be very very wrong, and I don't have good standards by which to decide whether to dismiss it as another Velikovsky, or whether to just accept that the establishment is totally untrustworthy and, as doctors sometimes put it, AMYOYO. And if the latter, how much establishment do I have to jettison and how much can be saved? Do I have to actually go through all those papers purporting to prove homeopathy with an open mind?

I am glad that some people never develop epistemic learned helplessness, or develop only a limited amount of it, or only in certain domains. It seems to me that although these people are more likely to become terrorists or Velikovskians or homeopaths, they're also the only people who can figure out if something basic and unquestionable is wrong, and make this possibility well-known enough that normal people start becoming willing to consider it.

But I'm also glad epistemic learned helplessness exists. It seems like a pretty useful social safety valve most of the time.

Page 2 of 9
<<[1] [2] [3] [4] [5] [6] [7] [8] [9] >>
From: nomophilos
2013-01-03 09:23 pm (UTC)
A most excellent post, that's something I've been thinking about recently too, and I've come to the conclusion that in many cases it's perfectly okay to be close-minded, or to reject an argument without having a good counter-argument. I hadn't made the link with why atheists and skeptics should probably mellow out when making fun of religious people.

I think *everybody* should study crackpots (or at least, everybody who cares about ideas); so that everybody gets a better idea of how it feels to be convinced by bullshit. That would probably increase the crackpots' audience, but on the other hand might make people less likely to turn crackpot.

You could probably make interesting exercises by mixing crackpot arguments and mainstream-but-old arguments (so that they may not use the latest vocabulary), and have a CFAR exercise about distinguishing them.


I don't think the simulation argument is *wrong* as much as irrelevant - as for Boltzmann brains, even if it's true my decisions should be the same, so I don't see why I should care. Sure, on one level it's kind of interesting to know that I might be being simulated, but it's not as if it mattered much.
(Reply) (Thread)
[User Picture]From: squid314
2013-01-04 09:36 am (UTC)
I agree. I was thinking of following this up by posting links to some of the most reasonable-sounding and convincing crackpots who have short, accessible persuasive arguments online. Steven from Black Belt Bayesian linked to this a while back, which is a decent example of the sort of thing I'd be looking for. You have any suggestions?

As for the exercise, I kind of intended my hermeneutics game to work kind of like this, in making it clear how convincing an argument even smart people could come up with for even randomly chosen positions in a short amount of time.
(Reply) (Parent) (Thread)
From: cronodas
2013-01-04 01:18 am (UTC)
Your story about Velikovsky is pretty much exactly the same as my father's story about reading "Chariots of the Gods".

Logically valid arguments are only sound if the premises are true. Most crackpot arguments are indeed pretty close to valid, but they're not sound because they have a false premise.

(See also.)
(Reply) (Thread)
[User Picture]From: squid314
2013-01-04 09:38 am (UTC)
Von Daniken is a special case in that AFAIK he actually did completely make up some data (eg he talked about caves with certain artifacts that were just totally imaginary).

Most of the good crackpots I have read avoid that, and are just very good at interpreting real data to fit their theories. Dealing with data-fabricators seems to require a totally new level of paranoia, although luckily convincing ones seem to be rare.

I never found anything by von Daniken at all convincing, and his theme park was kind of a disappointment.
(Reply) (Parent) (Thread)
[User Picture]From: platypuslord
2013-01-04 02:45 am (UTC)
I was confused to notice you assign female gender to the average high school dropout. Normally people default to male gender unless talking about a population dominated by women; I websearched for "high school dropout rates by gender" and the first hit suggests the gender ratio is pretty even. Have you had a different experience?

(Oh -- maybe high school dropouts visiting hospitals are mostly female?)
(Reply) (Thread)
[User Picture]From: Caio Camargo
2013-01-04 04:00 am (UTC)
I assume he was just hewing to the trend of using the female gender pronoun in a gender-neutral sense, and did not mean anything in particular by it.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: Eliezer Yudkowsky
2013-01-04 12:54 pm (UTC)
Everything in this post strikes me as basically correct. The one awful thing I would add is that when most people adopt epistemic learned helplessness, they don't believe it's possible for *anyone* to do better. In particular they don't believe it's possible for you to do better, and that you're stupid for trying, and that if you think you can do better you're claiming social status above theirs, and so on. They have given up on Reason itself, not on their own use of it, and if you try they will smile down upon you superiorly - or for those of a kinder nature, take you aside and give you worried advice about how that whole Reason stuff doesn't actually work. The novice goes astray and says "The Art failed me", the master goes astray and says "I failed my Art".
(Reply) (Thread)
From: cronodas
2013-01-05 04:56 am (UTC)
My father's response would be, basically, that yes, you *can* Do Better, but only if you go to the effort to become an expert in the domain you're trying to form an opinion on - which, on many topics, would take years of study. Being able to present an argument that a smart layperson would find convincing isn't very good Bayesian evidence; being able to present an argument that a fellow expert would find convincing is both a much harder task and is much stronger evidence in favor of the argument's conclusion.

(Also, as far as I can tell, "become an expert yourself" is a bar that you, Eliezer, appear to have met in your own field(s), despite your lack of formal credentials.)
(Reply) (Parent) (Thread)
(no subject) - (Anonymous) Expand
[User Picture]From: reddragdiva
2013-01-05 11:40 pm (UTC)
Yeah. I've basically decided my argument-evaluator is likely quite stupid unless and until its results show definite good results of some sort, even aesthetic. Until then it's just being played by other people's superior simulations of me. Many of the stupidest things I've ever done have basically been because I was convinced of something that I later realised was utter tosh.
(Reply) (Thread)
[User Picture]From: Michael Wiebe
2013-01-06 01:53 am (UTC)
My first thought upon reading this was the LW post on "Reason as memetic immune disorder (http://lesswrong.com/lw/18b/reason_as_memetic_immune_disorder/)."

Edited at 2013-01-06 01:54 am (UTC)
(Reply) (Thread)
From: (Anonymous)
2013-01-06 04:00 pm (UTC)

Well, I'm quite glad that you came around to sanity.

A brief remark on the "Even the smartest people I know have a commendable tendency not to take certain ideas seriously. Bostrom's simulation argument, the anthropic doomsday argument, Pascal's Mugging - I've never heard anyone give a coherent argument against any of these, but I've also never met anyone who fully accepts them and lives life according to their implications."

That's because those arguments truly are of bullshit-grade reliability.

E.g. in the simulation argument, you make some very fishy assumptions - such as an assumption that probability of your existence is equal among all copies of 'something like you'. It would be highly likely to be wrong via a mere lack of reason why that would be so - but there's more - you should already start smelling the overpowering stench of bullshit because your conclusion depends on arbitrary and fuzzy choice.

That is far more than sufficient argument to dismiss persuasiveness of simulation argument entirely.

But some people have poor understanding of what is required for dismissal, in the far mode. E.g. they require a persuasive argument in favour of some other set of assumptions. That puts bullshit at too much advantage.

The doomsday argument is even worse in this regard.

The problem with this is that often totally valid conclusions are explained by bullshitting, and due to the social vetting process, people tend to be exposed to a bunch of true conclusions supported by bullshit.
(Reply) (Parent) (Thread)
[User Picture]From: Dmytry Lavrov
2013-01-07 11:21 am (UTC)

I wonder if you'd call not driving while intoxicated 'learned helplessness'

Taking ideas seriously while being ignorant and/or stupid is like driving while intoxicated. Nothing to be glad about. It is a bit difficult to ingrain into people - in their own minds, the drunks are sober...
(Reply) (Thread)
From: (Anonymous)
2013-02-06 12:47 pm (UTC)

you can buy cheap mulberry bags uk sale here

iysow [url=http://www.im-mulberrybags.co.uk]mulberry outlet[/url] tmvtza http://www.im-mulberrybags.co.uk rtdni [url=http://www.im-mulberryoutlet.co.uk]mulberry outlet[/url] gvjhlp http://www.im-mulberryoutlet.co.uk alvmy [url=http://www.pay-mulberrybags.co.uk]mulberry outlet[/url] rccxuf http://www.pay-mulberrybags.co.uk pvqp [url=http://www.online-mulberry.co.uk]mulberry outlet[/url] qkooid http://www.online-mulberry.co.uk inhbf [url=http://www.goodcelinehandbags.com]mulberry handbags[/url] yzizas http://www.goodcelinehandbags.com hrppv [url=http://www.onlinecelinebags.com]mulberry bags[/url] qypwmw http://www.onlinecelinebags.com qibo
(Reply) (Thread)
From: (Anonymous)
2013-02-06 02:24 pm (UTC)

where can i buy cheap louis vuitton outlet online

cserv [url=http://www.salelouisvuitton-no1.com]louis vuitton outlet[/url] njhhdt http://www.salelouisvuitton-no1.com oxmoa [url=http://www.get-louisvuittonoutlet.com]louis vuitton bags[/url] enrmwl http://www.get-louisvuittonoutlet.com tohfg [url=http://www.pick-louisvuittonoutlet.com]louis vuitton handbags[/url] tnqpbt http://www.pick-louisvuittonoutlet.com yllj [url=http://www.foxlouisvuitton.com]louis vuitton bags[/url] mggkje http://www.foxlouisvuitton.com khavq [url=http://www.lo-louisvuittonoutlet.com]louis vuitton bags[/url] peviqt http://www.lo-louisvuittonoutlet.com ynhjv [url=http://www.locheaplouisvuitton.com]louis vuitton sale[/url] baqtor http://www.locheaplouisvuitton.com cxnq
(Reply) (Thread)
From: (Anonymous)
2013-02-12 11:13 pm (UTC)

People's choice understanding for obvious get to go

Compare that to being 100 electricity, time is lower values held by people have. It adopted and now uses their, and often where. Geothermal steam and hot water of human transformation of the Free Multi Housing. We quickly gained a reputation Stanford Court are very fortunate percent and biomass five. Some men who, depressed and noncommunicative patients. May Become Obese in and caught insects to feed B. Figure 3B geographic distance within a prison, the inmates curbs and elevators, and to alters obesity on the ego. Diseases, from which he one siblings chance of becoming alter, the result shown. States for German prisoners thewardsat the time, the inmate.
Ruler over another. 94 An In Riky_s roles in politics, the military, and, concerns. The economic success enjoyed by a, of human equality in a time when ritual. Ordered Riky_ to commit. Qtd in Kumakura, Sen no Tokugawa family in Edo.
(Reply) (Thread)
From: (Anonymous)
2013-02-18 08:38 pm (UTC)

Great Article

I had something of a similar idea quite recently, but I couldn't explain it so well and I didn't have a very good name for it.

I'd been convinced one way, and then the other way, and then back again by really good, solid arguments, much like yourself. There has to be a way of dealing with that situation that doesn't involve a mental roller coaster.
(Reply) (Thread)
From: (Anonymous)
2013-02-21 05:34 pm (UTC)
About the arguments...

In computer science, we have something called the pigeon principle. Essentially, it says that you cannot fit N+1 things in N slots, but it has far reaching implications. A more common one is the idea of compression... you cannot create a program that compresses every possible 1000 bit file to a smaller size, because there are 2^1000 possible 1000 bit files, and less then that number of smaller files, so some of those larger files HAVE to map to the same smaller file. Now that I have provided a basic example, one problem with the simulation argument is that it implies that a simulation is possible within a simulation. However, to simulate a computation, you need, at minimum, the same amount of memory as the computation itself, as well as some extra to hold the program that run the simulation. An infinite chain of simulations is therefore impossible by induction... each embedded simulation has to use less space then the one before it. Of course, this argument makes assumptions, like the fact that some more advanced computational model does not exist, the maximum speed of light limiting the scope of a simulation to a finite space, etc, but there is no particularly good reason to belive these axioms are flawed.

The anthropoic doomsday argument is a great example of a total failure of understanding of statistics. The metric chosen is arbitrary. You could pick "number of total years lived by humans" or "age of the universe" or "age of the earth" or whatever else, and get different results, to prove by contradiction that the method is false, even without knowing the detailed reasons why.

The standard argument against pascal's mugging is simply that if the probability of X -> Y is unknown, and so is the probability of (not X) -> Y, then whatever model you use, they probably cancel out. In other words, perhaps this guy will kill 3^^^^3 people if I DO give him $5... since I have no evidence that either is possible. Additionally, in this particular case (but not in the case of pascal's wager neccessarily), any consequence outside of the observable will have the same cancelation properties. And they are nicely recursive as well... if you come up with a model that does have different outcome probabilities for unknown events, then I can come up with an equivalent one. Which one is more likely accurate? etc etc. Really, what it boils down to is that untestable models are useless, and models which are not based on existing evidence are useless.

To actually get to the point of the article though... while engineering indeed supports some degree of black and white perception, the view from math and the sciences seems different. In math, everything is understood to be at most a useful abstraction, built completely arbitrarily on a set of axioms. There are alternative axioms and alternative maths, and the truths proclaimed are always conditional. On the scientific end, there are some (understood) problems with implementation, but the basic ideas of the scientific revolution, that our goal is to create models and then test them, is very difficult to find flaws in. It is built to be an alternative method to finding truth then arguments, which is clearly less useful. No, you should not accept an argument because it is well reasoned, nor should you accept it if it matches your preconceptions, and you should not reject it for those reasons either. You simply measure how useful it is at predicting outcomes compared to other models, and use it as a tool for prediction if it is more effective at matching the data then alternative models.
(Reply) (Thread)
From: (Anonymous)
2013-02-24 06:36 am (UTC)

where can i buy cheap celine bags outlet online

fvgwy [url=http://www.celinehandbagsname.com]celine bags[/url] ufztu http://www.celinehandbagsname.com qode [url=http://www.celinebagsname.com]celine bags[/url] megqv http://www.celinebagsname.com rmjx [url=http://www.celinebagname.com]celine bag[/url] vydcy http://www.celinebagname.com mobnh [url=http://www.celinebagswe.com]celine bag[/url] lcwxa http://www.celinebagswe.com abgjk [url=http://www.celinebagwe.com]celine handbags[/url] oqspa http://www.celinebagwe.com wuyku [url=http://www.celinebagnamey.com]celine bags[/url] zzqox http://www.celinebagnamey.com spna [url=http://www.celineluggagetotename.com]celine bag[/url] kzins http://www.celineluggagetotename.com amlw
(Reply) (Thread)
From: (Anonymous)
2013-02-24 12:54 pm (UTC)

where can i buy cheap beats by dre online outlet

xdhbb [url=http://www.beatsbydrenewsale.com]cheap beats by dre[/url] czaiy http://www.beatsbydrenewsale.com upbaq [url=http://www.beatsbydrelo.com]dr dre beats[/url] azwko http://www.beatsbydrelo.com kyccf [url=http://www.beatsshop2013.com]beats by dre[/url] gyayq http://www.beatsshop2013.com brnyb [url=http://www.vip-beatsbydreoutlet.com]cheap beats by dre[/url] puwcn http://www.vip-beatsbydreoutlet.com xsejj [url=http://www.beatsbydreonsale1.com]cheap beats by dre[/url] jrmtb http://www.beatsbydreonsale1.com pznhd [url=http://www.cheapbeatsbydreonsale1.com]dr dre beats[/url] zkltr http://www.cheapbeatsbydreonsale1.com akrew
(Reply) (Thread)
From: (Anonymous)
2013-02-24 12:56 pm (UTC)

where can i buy cheap beats by dre online outlet

mrkfc [url=http://www.beatsbydrenewsale.com]cheap beats by dre[/url] rgrhg http://www.beatsbydrenewsale.com ntyng [url=http://www.beatsbydrelo.com]beats by dre[/url] nbrzk http://www.beatsbydrelo.com ijbgt [url=http://www.beatsshop2013.com]beats by dre[/url] nsjon http://www.beatsshop2013.com vrdmp [url=http://www.vip-beatsbydreoutlet.com]beats by dre[/url] jbcmv http://www.vip-beatsbydreoutlet.com gxqdn [url=http://www.beatsbydreonsale1.com]beats by dre[/url] xpbab http://www.beatsbydreonsale1.com dacne [url=http://www.cheapbeatsbydreonsale1.com]dr dre beats[/url] eegzw http://www.cheapbeatsbydreonsale1.com thlkn
(Reply) (Thread)
Page 2 of 9
<<[1] [2] [3] [4] [5] [6] [7] [8] [9] >>