How valuable are prediction markets for matters where there isn't a substantial body of knowledge?
Terrible, but better than the alternative which is just guessing?
2012-10-26 06:47 am (UTC)
The alternative is almost certainly "trained intelligence analysts collaborating around the world produce an estimate." From a certain point of view, it's guessing, but...
Trained intelligence analysts can collaborate around the world to maximize their take in the market. Especially since markets can be subsidized.
The question is how to decide who's a trained intelligence analyst and how to aggregate their opinions. Prediction markets solve those problems. I would hope after a while only trained intelligence analysts would be using them.
Or people who had an interest in a particular outcome from that market. Or people who thought they could make money betting on the market moving a particular way.
I'm reading Kahneman's _Thinking Fast and Slow_; he says the evidence is that "trained experts" are terrible at such problems, underperforming a simple formula based on combining likely factors. It's not that the formula is great, it's that the problem is really hard, and the experts aren't actually trained in it, and the formula is less biased or overconfident.
 Like clinician psychologists estimating long term performance of their patients, or wine experts guessing the long term price of wine, or Israeli army interviewers estimating how well candidates would do in officer school, or as soldiers in general, or finance people picking stocks. For the wine a simple formula based on weather data did better; for the army, evaluating more narrow traits and combining the evaluations. Though "close your eyes and imagine this person as a soldier, then rate 1 to 5" also worked.
 The psychologist is well-trained in dealing with patients in the room. Evaluating how someone does years later is a much harder problem, with much less data or feedback, and much higher intrinsic uncertainty.
"This is actually not the conclusion I was expecting"
This is always nice to see.
(btw last link is broken)
That's really interesting. It fits my impression that Obama isn't stupid, even if the program may be morally wrong.
I guess he's found a way of swatting down opposition using overwhelming technological advantage without risking the lives of people on his "side", which is holy-grail like for a president. But that ignores (a) the moral wrongness of treating people neither as enemy soldiers nor as criminals and (b) the more you bomb people, the more you NEED to bomb people, wars, civil wars, and things-like-civil-wars-but-not-quite need to end by some sort of negotiated settlement without killing every individual enemy soldier.
I don't think claim b
(b) the more you bomb people, the more you NEED to bomb people
is quite true as stated. For example, the world's most infamous bombing campaign, in Hiroshima and Nagasaki, did in fact lead to a situation where further bombing was unnecessary.
And historians have long pointed to the Russian declaration of war shortly before the Japanese decision to surrender to the US as having been far more important to the Japanese decision-makers than some bombing neither of which even reached the death toll of past firebombings like Tokyo. So they are, perhaps, not the best examples to use.
I'm sorry, I was rather overgeneralising. But, I seem to think, the surrenders at the end of WWII were followed up by serious occupation and rebuilding, which led to eventual productive cooperation. If you turn it around, how much (justified or not) bombing, terrorist attacks, etc would America have to suffer before everyone, even extremists, gave up and stopped wanting to retaliate?
(a)... the moral wrongness of treating people neither as enemy soldiers nor as criminals ...
You ignore the third category, "unlawful combatants" or "franc-tireur" Franc-tireurs enjoy neither the right of soldiers to humane treatment under the Conventions of war, nor of civilians to arrest and trial by due process of law. The category exists in concept because it exists in reality: fighters who scorn to behave as soldiers, but are too numerous and heavily-armed to merely arrest as common criminals. If we ignore this category, we find ourselves helpless before such persons, we thus empower the strategy by success, and ensure its continuance.
(b) the more you bomb people, the more you NEED to bomb people, wars, civil wars, and things-like-civil-wars-but-not-quite need to end by some sort of negotiated settlement without killing every individual enemy soldier.
You are assuming that the foe possesses infinte morale. Wars can also end by unconditional surrender: they need not end by "negotiated settlement," nor is the enemy morale so great that such is impossible short of killing every indivdual enemy fighter (for they are not "soldiers.").
You ignore the third category, "unlawful combatants"
I think the ironic thing is that we maybe do need some legal recognition of this category of people, but the current problem is that many countries seem to be using it as an excuse to give people no rights.
Under the Geneva Conventions unlawful combatants have no rights. Or rather there's a set of four conditions fighters have to meet to get any rights.
I think that's understandable, but I think we've reached a situation where I'm not sure it's sustainable. Do you think there's any way of improving on the current situation?
I generally prefer killing all the unlawful combatants and anyone who helps them. That creates a powerful incentive to comply with the GC conditions.
I'm trying to sensibly organise my thoughts about this. I have a really bad feeling this doesn't work, but I don't know enough about the politics to be able to work out if I'm actually right or not, so I'm trying to feel my way through.
In order to detach the morals of "which side is right" from the morals of guerilla warfare, I'd consider a couple of other cases, such as the French resistance and Irish nationalists vs. the UK.
I agree that deliberate targeting of civilian population is pretty always wrong. (There are some grey areas.)
But the trouble with a case like the French resistance is that fighting as an army is impossible. They already did that, and lost. So the options are "give up and submit to Nazi germany" or "fight back as much as you can, even if it's illegal". We tend to like that case because we like the side they were on. But obviously, anyone in a similar position is going to think that their war against a supposed occupying power is justified, whether it is or not.
Or the case of Ireland, the situation is improved, with people on all sides eventually, reluctantly, cooperating with people on other sides who did awful things. If the answer was to kill everyone who didn't fight as a proper army until one side gave up entirely, I suspect the death toll would only have been higher, although I'm not certain.
The case of resistance movements was considered by the drafters of the current GCs (which are post-WWII). To quote the wikipedia summary, it applies to:
"4.1.2 Members of other militias and members of other volunteer corps, including those of organized resistance movements, provided that they fulfill all of the following conditions:
that of being commanded by a person responsible for his subordinates;
that of having a fixed distinctive sign recognizable at a distance (there are limited exceptions to this among countries who observe the 1977 Protocol I);
that of carrying arms openly;
that of conducting their operations in accordance with the laws and customs of war."
This is a great post, as usual.If I were the president, I would set up prediction markets on likelihood of civil war conditional on drone strikes, no drone strikes, and immediate retreat.
If you ran these markets openly, you would be forced to shut them down in short order. PRESIDENT squid314
APPROVED SICK BETTING RING GAMBLING ON AFGHAN LIVES. If you ran them secretly, their existence might still leak, plus of course a restricted prediction market isn't as good.
I hope that the various US and allied intelligence services run a secret internal prediction market to identify which analysts are actually right in advance, but given what we now know about how easy it was for the Bush administration to pressure the CIA into giving the answer they wanted to hear, I doubt such a programme would survive very long. I mean, it's hard to maintain the narrative that "everybody thought Iraq had WMD" if there's a clear record of which of your peers didn't believe that.
I'm not sure whether you're saying this because you already know the story of the Policy Analysis Market
or you're just very prescient.
I'm kinda surprised that one line gained so much attention here and on Facebook. I thought everyone was already on board with futarchy and the whole "use prediction markets to come up with a probability that gets converted into expected utility" thing.Edited at 2012-10-26 10:30 am (UTC)
I didn't consciously remember, otherwise I'd have linked to it, but maybe I heard the story and forgot about it.
Not to say that prediction markets aren't a good idea, but obviously not *everyone* in this stupid irrational world is on board with them.
As president, you could send a deniable intermediary to Intrade (and similar sites) and ask them to set up the markets you were interested in, and pay them a few hundred thou out of black budgets to ignore any outrage. That would work, and it would be immune to political interference, whereas a government-run prediction market might not be.
Edited at 2012-10-26 10:47 am (UTC)
No, I think of myself as a pretty generally well educated scientific mathsy type (although not an economist) and it was a new concept to me.
Yeah, I remember that story. :(
According to http://en.wikipedia.org/wiki/Prediction_market#Accuracy
prediction markets are no more accurate than nominal group technique and less accurate than Delphi method groups. If an organization with a bunch of experts, having them play out an internal prediction market seems wasteful.
This argument would seem to suggest that a consequentialist US government could just "surgically" kill small numbers of bad guys all over the world to prevent wars. Everywhere, not just in the places we're officially at war.
In fact, it seems that we are doing that, it's just classified. [I know this from a guy who has done assassinations in places we're not officially supposed to have troops.]
I never crunched the numbers, so it never occurred to me that such a policy might be *effective.* But I have to take seriously the possibility that by being really creepy world policemen, we may be saving a lot of lives. Whoa.
Of course, if the US were really a utility-maximizer for the whole world, it would have open borders, because we're rich and most of the world is poor. Most people seem to think there's such a thing as the national interest, which is different from global welfare. On those grounds, there's a much better case for staying out of other countries' wars, because it takes money out of Americans' pockets with little effect on American safety.
I think one of the biggest objections to Drone use from an international point of view is actually how safe they are, both in terms of civilian casualties and in terms of American casualties. It's easy to imagine that the lowered cost in lives and public opprobrium lead to a world with more and more drone strikes, and more and more US intervention in the rest of the world, with few objections. Depending on your views on the morality of one nation influencing the world in this way, drones that never kill civilians might be the worst of all. Would you trust America(or whatever stereotype you have of it) with a Death Note?
You're assuming that drones are infinitely capable and that defense against them is impossible.
We already trust every sovereign nation on Earth with the right to kill people. That's part of the definition of "sovereign."
Of course, the States whose citizens are killed also have the right to treat such attacks as wars and fight back. That's also part of the definition of "sovereign."
But additionally, in doing so, those States retroactively sanction the attacks those citizens may have made against other States, for which they were killed in retaliation. That's the part of the definition of "sovereign" that, for instance, Pakistan pretends to ignore.
Not to mention making drones more desirable. The current model seems to be that only the US has drones. How long is that likely to be the case?
A certain Star Trek episode comes to mind.
(Great post, though. I have the normal amount of doublethink suspicion of anything that appears to resolve a difficult moral issue in my favor.)
I think a problem with using a futures market to drive policy is that people who want to influence policy can then do so by buying futures.
For example a group sympathetic to terrorists could buy futures asserting that drone attacks are terrible and useless.
As a more concrete example, Nate Silver suspects
that a single entity has successfully influenced Intrade's prediction of Romney's chance to win. "what appeared to be a single trader bought a large number of Mr. Romney’s shares at Intrade, at one point boosting Mr. Romney’s chances to about 49 percent from 41 percent over the span of a few minutes..." "The overall amount of the bet early on Tuesday equaled about $18,000."
Not to mention terrorists buying shares before they strike. :) Granted that pushes the price up, but still easy profit.
(Cf. speculation that al Qaeda shorted the stock of airlines they were using. Unproven, I believe.)
People can also influence policy with campaign donations, lobbying, and the like. If anything, in the long run prediction markets ought to be more
resistant to being swayed by money, as the people who are buying incorrect predictions in order to influence policy are in effect donating extra money to people who are in the business of making correct predictions. (See also Hanson et al. 2005
.) This is not the case with money spent on lobbying today. Also, in a large enough prediction market, the sums you'd need to invest in order to have a chance of plausibly affecting policy might easily dwarf the lobbying investments you'd need to make in order to have a similar impact.
I don't want to get into it here, but the InTrade problem is probably caused by InTrade being insufficiently liquid, and I think it has already compensated for that problem.
2012-11-06 02:46 am (UTC)
http://hanson.gmu.edu/biashelp.pdf describes how it might be difficult for manipulators to influence prediction market prices, although the paper mentions significant caveats:
However, since this is not a fully general model, it cannot by itself support strong general claims about the price eﬀects of manipulation. For example, the model assumes risk-neutrality, normally distributed values and signal errors, interior choices of information quantity, no transaction costs of trading, no budget constraints, and a single rational manipulator with quadratic manipulation preferences and a commonly known strength of desire to manipulate. While we believe that these assumptions are natural ones for a ﬁrst modeling eﬀort, one should remember that some of the model’s ﬁndings may not be robust to changes in some of these assumptions.
Thank you! I think this is the first article or blog post I've read that takes a considered view of drone strikes that doesn't suffer from blindness in one direction or another about either a) the morality of using drones vs. soldiers, or b) whether our current president is evil or the best president ever.
Personally, I'd like to see a vigorous debate begin about the rise of unilateral executive power, including how it's being driven by congressional gridlock. The regular use of drone strikes within the borders of an allied country is just one instance of this power. Failing that, at least seeing a discussion about whether this strategy is even (theoretically) effective is very useful. Having a better idea of whether it saves large numbers of lives, and at what cost, really does help inform any other discussion about the morality of it all. You're quite right to point out that opponents to drone strikes need to do some hand-wringing about it as well.
2012-10-27 05:02 pm (UTC)
"I can't imagine how much trauma, but it should be at least sort of proportional to the number of deaths in each branch."
This strikes me as pretty improbable. If trauma were even roughly proportional to number of deaths, people would fear falling coconuts more than sharks, and the U.S. government wouldn't bother talking about terrorism.
It's really refreshing to see a careful consequentialist take on this issue. I'd like to argue that it's still good to oppose drone strikes for reasons similar to the reasons we don't follow naive consequentialism in the Transplant problem. But I don't know how to express those reasons in a clear quantitative argument, which is frustrating.
Is there a good argument along those lines? Or is the drone strike problem significantly different from the Transplant problem? Or is killing the patient actually the right thing to do in the Transplant problem?
Great post, as usual Scott. Now for the criticism.
You underestimate the force of argument 2, advocating a more radical departure from the status quo. The history of Afghanistan suggests that occupying forces get thrown out violently, causing civilian casualties in the process, followed by a period of civil war. If one thinks that the Americans will break this pattern by staying until the opposition is utterly defeated, or that Karzai will prevail after withdrawal, one needs strong evidence to outweigh this "outside view" perspective. If one accepts that the pattern will likely be repeated this time, then killing more Taliban fighters may increase the expected long run death toll.
On the other hand you overestimate the force of argument 1. The biggest factor in turning non-terrorists into terrorists is probably the killing and injuring of Afghans and Pakistanis, especially civilians. Non-drone forces do more of that killing and injuring, per military objective achieved, and so probably are even worse than drones, in terms of recruiting more terrorists.
How about a quick utilitarian calculus on conducting terrorist attacks?
Say you're a member of Al Qaida. You're deciding whether or not to bomb a building used to conduct affairs of the new U.S. puppet government.
Considerations of lives lost:
American lives lost. You might kill some American soldiers. Possibly a journalist or two as well. Like some Americans seem to value the lives of terrorists less than those of civilians, you probably value the lives of American soldiers less than those of civilians, and much less than the lives of Afghani civilians.
Afghani lives lost. Many of the people in the building will be Afghanis who have some business with the government or are simply taking whatever jobs they can get.
Puppet government lives lost. These folk are lapdogs of the Americans and therefore horrendous traitors. They probably only count as much as American soldiers do.
Less immediate effects of your terrorist attack:
You will interrupt the activities of the puppet government, which is obviously evil. You will therefore marginally hinder them from doing some evil things in the near future.
You will kill some Americans. This could go two ways. Either it could cause support for the war to falter back in the homeland, in which case your attack may not only save some marginal lives from drone strikes, raids, and bombings, but could marginally save many more lives in potential future wars that Americans are too chicken to start for the unpopularity of this war.
OR you could strengthen the resolve of Americans to kill more terrorists, in which case collateral damage will increase. You could still have a negative effect on future war-starting, however, since future wars will not be against The Terrorists.
You might engender sympathy with Americans among Afghanis whose families were killed in the attacks.
It's possible that the Americans will keep finding people willing to be in the puppet government even if you keep killing them, so nary a dent is made on it. (How does this compare to the Al Qaida recruitment rate conditional on drone strikes? Does anyone even know what the Al Qaida recruitment rate is?)
The probabilities of these after-effects are all kind of up in the air. But the benefits of some of them, note, are huge. Preventing something on the order of the Vietnam (or, say, the Iraq War) should win you the Nobel Peace Prize. And you're going to be killing far more puppet government lackies and Americans than innocent Afghani civilians, most likely, and you can plan the attack for a time of day in which that would be the case, so you can hopefully minimize the immediate cost. The probability of reducing support for the war is probably higher than the probability of strengthening the resolve of Americans to kill terrorists. American citizens don't really seem to have a good idea of how many terrorists there are or how hard it will be to kill them are. However, those who care about it count every single American soldier death to the number. In fact, press reports in the U.S. almost exclusively report American death counts; they tend to only give very rough estimates of Afghani death counts.
I don't have the time offhand to find average soldiers-per-citizen and government-per-citizen ratios per terrorist attack at the moment to find what probabilities would be necessary for the other events to justify terrorist attacks.
2013-03-12 03:08 pm (UTC)
nike Can Provide New Life To An Old Dilemma. . . Defacto Standards