You are viewing squid314

Tired of ads? Upgrade to paid account and never see ads again!
Jackdaws love my big sphinx of quartz - Kurzweil's Law of Accelerating Returns [entries|archive|friends|userinfo]
Scott

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Kurzweil's Law of Accelerating Returns [Feb. 2nd, 2013|10:37 pm]
Scott
[Tags|, , , , ]

My review of The Great Stagnation provoked sufficient discussion of Kurzweil and whether his ideas made sense that a friend (who wishes to remain anonymous) offered to do the research if I would write a post about him. This post is mostly zir research, which explains why it is unusually complete.

Kurzweil's Law of Accelerating Returns is the opposite of the Great Stagnation theory. It says that instead of technological change gradually sputtering to a halt as all the interesting ideas are exhausted, technological change builds on technological change to produce even faster technological change. Some technologies allow other technologies, as the combustion engine allowed the airplane. Other times they make research itself more efficient, like computer-assisted design of new circuits. And other times they allow society to become richer and more populous, driving future growth.

Most people think of technological advance as linear; technology advanced a little bit in our parents' generation, it's advancing another little bit our generation, and it'll advance another little bit in our kids' generation. Kurzweil's ideas imply it's exponential; our generation will have more and more exciting advances than our parents', and our childrens' more still, until eventually things go crazy and we're having new world-shaking technologies every minute (this could either be a reductio ad absurdum, or an argument for a technological singularity).



One of his most common arguments is that the obvious and basic insight that things happen slowly in geologic time, a little quicker in evolutionary time, quicker still in human time, and very quickly indeed in modern times is actually another way of saying there's exponential growth in novelty. One of his books includes this graph:



And despite agreeing with the basic insight, I want to mention just how terrible this graph is.

The horizontal axis is time before present, a perfectly valid measure. But the vertical axis is "time to next event". This moves cherry-picking from a potentially annoying but nondisastrous error to the force driving the entire argument. That is, once you use "time to next event" as a measure, you can't just pick a couple of representative data points and let the human ability to draw lines through a scatter plot do the rest.

For example, suppose that instead of using "art, early cities" as a single invention, he had split it into "art" and "early cities", with Early Cities being invented a few hundred years after art. In that case, "early cities" would be quite near bottom of the graph, a huge deviation from the line. In fact, note that the closer together early cities and art are, the worse his graph looks, but that if he assumes they were simultaneous and lumps them together into a single "early cities, art" category, his graph looks perfect.

This is super sketchy.

But he has a response. In order to prove he's not just cherry-picking events to fit his theory, he does the same thing with lists of important events put together by other people, and comes up with the following more colorful plot:



The black diamond on the graph above, Modis, writes a paper critiquing Kurzweil's methodology here as well. In it he protests that only one of the data sets (Sagan's) covered and dated the entire range of events from Big Bang to present, and that in fact some of the data sets on there are based entirely off the other data sets yet presented as independent confirmation.

A more damning critique is that in many of these data sets, the whole point was to talk about things at specific intervals. One of them (Heidmann, the purple square) is from a book explaining scientific notation by listing important events that happened 10, 100, 1000, etc years ago, so of course those are going to follow exponential laws. Even when the bias wasn't that blatant, it may be that people want to seem "fair" by giving representative events from each "era" - a list of "cosmic events" that included ten different scientific discoveries from the twentieth century, plus the Cambrian explosion, would seem pretty non-cosmic.

Overall I think it's obviously true that if you define "technological advance" broadly enough, it happens more quickly in the 21st century than back in evolutionary times when it took ten million years to come up with a slightly different enzyme configuration, and most of these criticisms are haggling over exactly how neat the line is.

A Moore Objective Measure

Besides, there is a much more measurable, non-cherry-pickable area where Kurzweil seems to have an equally strong proof. This is Moore's Law, which gets formulated in a couple ways but is usually something along the lines of "the number of transistors on a chip doubles every two years".

Different formulations of Moore's Law have different doubling periods and different accuracy levels, but they all seem to be pretty much on track. Here are graphs of transistor counts and PC hard drive capacity, respectively:



There are some physical limitations on whether Moore's Law can continue in its current form indefinitely, but it might be possible for paradigm-shifting technologies to continue to capture the "spirit" of the law. For example, a quantum computer might not literally have more transistors, but it might be able to do more efficient calculations as if it did. In this spirit, Kurzweil replaces very specific measures like "number of transistors" with his own "million instructions per second $1000" measure. Annnnd



This also looks pretty good, and his data have been mostly confirmed by various other sources. Although there's some confusion about the measures used, most sources agree that MIPS is likely if anything to understate growth, and that if anything the trend is better than exponential.

"Information Technology"

You can generalize Moore's Law to lots of different aspects of electronics, from number of pixels on an average camera to capacity of optical fibers. Kurzweil wants to take it even further and say that exponential growth applies to all "information technologies". He believes that "information technologies" will eventually include all technologies even tangentially associated with data, which causes him to include for example medicine:
"Drugs are essentially an information technology, and we see the same doubling of price-performance each year as we do with other forms of information technology such as computers, communications, and DNA base-pair sequencing. AIDS drugs started out not working very well and costing tens of thousands of dollars per patient per year. Today these drugs work reasonably well and are approaching one hundred dollars per patient per year in poor countries such as those in Africa."

This does not really seem to mesh with what anyone else believes is happening in medicine right now. AIDS is sort of a huge exception in terms of being one of the biggest medical success stories of recent decades, and even AIDS drugs are not doing as well as he thinks. According to the WHO, the median treatment cost per year for AIDS drugs declined from $245 in 2003 to $140 in 2006 to $107 in 2009, and in the last three years has declined only slightly, to $93 in 2012. To fit Kurzweil’s prediction, there would need to have been a more than 30-fold increase in the effectiveness of AIDS drugs in the last six years (it’s not even clear what that would mean, given that by 2006 the drugs had long been effective enough to keep AIDS from being a death sentence for those who could afford them). And the declining price of AIDS drugs has had at least as much to do with successfully getting Third World countries exemptions from patent laws than with any tech increase.

MRI is also kind of like this. A paper by Sandberg and Bostrom claims that it's "impossible" to get a resolution better than about 8 micrometers unless you want to make people sit in an MRI machine for thirty hours. And in fact there has not been significant progress in this field in the last ten years.



(this is also a good example of how Kurzweil's book can be misleading. This version of the graph removes a data point from a previous version in 2000 which was about the same level as the 2012 data point and would have made it clear that, contrary to how it looks now, MRI progress has in fact stopped.)

Software is another information technology that just isn't doing as well as predicted. Computer scientist Ernest Davis writes of AI research:
Moreover, the success rates for such AI tasks generally reach a plateau, often well below 100%, beyond which progress is extremely slow and dicult. Once such a plateau has been reached, an improvement of accuracy of 3% — e.g. from 60% to 62% accuracy — is noteworthy and requires months of labor, applying a half-dozen new machine learning techniques to some vast new data set, and using immense amounts of computational resources. An improvement of 5% is remarkable, and an improvement of 10% is spectacular.

In a lot of these types of fields (machine translation is a similar one) it looks more like progress is approaching an asymptote than growing exponentially.

Stuart Armstrong did a pretty complete evaluation of some of Kurzweil's more specific predictions that can be found here

Conclusions

Overall the idea that there is more rapid change now than in deep geologic time seems correct, and although we can dispute particular data points there does seem to be something to the idea that the growth is exponential.

Moore's Law applies to various digital technologies, is at least exponential, and seems to still be in effect.

A lot of technologies are not growing exponentially, or started growing exponentially and then plateaued.

In general it does seem like in the best case technology can grow exponentially, and that an outside view that it will keep doing so can trump an inside view telling us that come on, computers can never have an entire ten megabytes of memory, that would be ridiculous. But it also seems that there is no hard-and-fast rule that all technologies will always grow exponentially and never plateau. I'm disappointed how little research there is in quantifying medical technology growth as that seems to be one area that a lot of people think is plateauing.

If I had to take one lesson from Kurzweil, it would be that once again, the absurdity heuristic doesn't work. Exponential growth can go on a lot longer and change things a lot more radically than someone who believes in linear growth would think remotely possible. But I don't think it would be a good idea to count on it.
linkReply

Comments:
[User Picture]From: andrewducker
2013-02-03 10:42 am (UTC)
Technology more frequently follows an S-Curve - increasing dramatically over a short period of time before plateauing. Flight speeds would be one example of this.

Anything that can be digitised will be dragged along by Moore's Law, and that's a lot. But it's not everything, by any means.

However, if we reach the point where _we_ can be digitised, then that will be particularly interesting.
(Reply) (Thread)
[User Picture]From: jordan179
2013-02-03 04:34 pm (UTC)
A series of S-curves. The reason why is that you see an enabling technology (such as the piston aero engine) which increases performance at a rapidly accelerating rate until one runs into a barrier (such as atmospheric resistance) which causes the improvement to rapidly decelerate, until the next enabling technology improves performance again. Never assume that the rise of an S-curve will last forever; also never assume that this is the last S-curve ever in that field.
(Reply) (Parent) (Thread)
[User Picture]From: ciphergoth
2013-02-03 12:21 pm (UTC)
(Reply) (Thread)
From: jimrandomh
2013-02-03 06:35 pm (UTC)
Looks like that should be http://raikoth.net/Stuff/blog/kurz_line.png (png instead of jpg).
(Reply) (Parent) (Thread)
[User Picture]From: st_rev
2013-02-03 03:46 pm (UTC)
0) Exponential curves don't have a 'knee', that's entirely a visual artifact of the chosen scales. Throwing that into the graph immediately makes me suspect the writer's understanding.

1) The Jumping Jesus Phenomenon sidesteps Kurzweil's cherry-picking. So does aggregate wealth.

2a) As andrewducker points out, individual technologies tend to follow S-curves, and indeed there's a hint of that behavior in the graph of PC hard drive capacity. I'd go further and say that the appearance of exponential growth is the result of thousands of emergent S-curve developments along different axes, getting projected in aggregate onto a single measure like 'wealth' or 'knowledge' or 'progress'.

2b) It seems to me that the real question of technological progress is the rate at which a society finds and adds new S-curves to the aggregate, and this rate is i) random and fractally bursty and ii) strongly dependent on cultural conditions. Imperial China is a good example of a society with an early technological headstart, but which embraced cultural stability and stagnated. The West seems to be making a lot of similar decisions via the precautionary principle--cf. power technology (we could have had thorium breeder reactors by now if we hadn't turned away from nuclear technology in the 70s) and Eroom's Law in medicine.

3) The low-hanging-fruit issue and exponential progress shouldn't be treated as opposing arguments. Both the difficulty of new problems and our power to solve new problems increase superlinearly over time, and it should be clear (but seems rarely recognized) that meaningful progress at the margin is proportional to the ratio of the two. That ratio might be sublinear, linear, superlinear, who knows? But it's the measure we should actually care about.

4) Again, regarding progress in medicine, please take a look at the discussion of Eroom's Law. Derek Lowe has also tackled Kurzweil's arguments regarding medicine in more detail here and here.

Edited at 2013-02-03 04:08 pm (UTC)
(Reply) (Thread)
[User Picture]From: jordan179
2013-02-03 04:38 pm (UTC)
Both the difficulty of new problems and our power to solve new problems increase superlinearly over time, and it should be clear (but seems rarely recognized) that meaningful progress at the margin is proportional to the ratio of the two. That ratio might be sublinear, linear, superlinear, who knows? But it's the measure we should actually care about.

That's an extremely good point, and it comes up in other issues such as the degree to which society becomes dependent upon "high" technology which is concentrated in only a few production centers, which would seem to indicate that eventually society becomes utterly dependent upon single points-of-failure and crashes. The reason this never actually happens is that "high," "medium" and "low" technology is always relative to overall technology at the time, so that the "high" technology of one generation is the "medium" technology of another generation and the "low" technology of still another.
(Reply) (Parent) (Thread)
[User Picture]From: st_rev
2013-02-03 10:47 pm (UTC)
See also the mystery of why peak whale oil didn't destroy western society.
(Reply) (Parent) (Thread)
[User Picture]From: xuenay
2013-02-04 09:51 am (UTC)
individual technologies tend to follow S-curves, and indeed there's a hint of that behavior in the graph of PC hard drive capacity. I'd go further and say that the appearance of exponential growth is the result of thousands of emergent S-curve developments along different axes, getting projected in aggregate onto a single measure like 'wealth' or 'knowledge' or 'progress'.

It's been a while since I read The Singularity is Near, but my recollection is that this is exactly what Kurzweil says: that the exponential curves are produced because there are a series of S-curves, and that as one tech paradigm is starting to lose its juice, people move on to the next one. Then he provides a number of examples of new paradigms (e.g. 3D chips, reversible computing, other stuff that I forget) that could give rise to new S-curves and thus keep Moore's law on track.
(Reply) (Parent) (Thread)
[User Picture]From: mindstalk
2013-02-03 04:01 pm (UTC)
Huh, your discussion highlighted Kurzweil flaws I hadn't appreciated before. Look at "the Industrial Revolution" dot. That's lots and lots of advances all on top of each other, which expanded out would drop to the bottom (short intervals between) before rising back up (Great Stagnation since.)

City-states could be "sewers, administration, taxes, roads, democracy, medicine, alphabetic writing..."
(Reply) (Thread)
From: (Anonymous)
2013-02-03 04:33 pm (UTC)
Even if tech growth really is accelerating like that, shouldn't it hit a hard wall of how fast humans deal with (both invention and putting into use) new tech? Part of the reason for the computer thing is that they are not super expensive and having recent ones is a status symbol.

At that time the limiting factor for tech growth will be mental transhumanism (Assuming no powerful AI, which would still need transhumanism to integrate...)

That won't be a singularity, really. THat is my true rejection of Kurzweil.
(Reply) (Thread)
[User Picture]From: jordan179
2013-02-03 04:43 pm (UTC)
The term is "social resistance." But that's an elastic rather than rigid factor, since different human cultures have different degrees of social resistance -- compare America, Britain, France, Russia and China c. 1800 for a good contrast. It may be true that some degree of social resistance is necessary to avoid unwise applications of technology ("Wow! Gamma rays aid plant growth! They must aid human growth too!" Let's dump uranium salts in the water supply right now, Mayor LaGuardia!"). But it's also true that integrating technological progress usefully is one of the major measures on which civilizations compete.

We need not assume that America in the 19th to mid 20th century is the ultimate example humanly possible of a society able to handle technological progress. The ability to handle such progress is itself a "techology" (a "soft" technology), in which progress can occur: compare almost any modern civilization with that of, say, any of the Egyptian Kingdoms.
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-02-05 04:19 am (UTC)
I'm referring to a much harder human mental limit than that.
(Reply) (Parent) (Thread)
From: (Anonymous)
2013-02-18 03:04 am (UTC)
Part of Kurzweil's argument is that at some point we're going to start upgrading ourselves as well, both directly and by using more advanced AI's.
(Reply) (Parent) (Thread)
From: Luke Muehlhauser
2013-02-03 04:46 pm (UTC)
> MIRI sponsored...

We did? News to me.

Stuart at FHI did almost all the work. The only thing SI/MIRI did was to let him use our volunteer network at SingularityVolunteers.org to find evaluators.


Typos: There are a few brackets showing for you to fix.


> although we can dispute particular data points there does seem to be something to the idea that the growth is linear.

Did you mean "exponential" rather than "linear"?


> This version of the graph removes a data point from a previous version in 2000 which was about the same level as the 2012 data point and would have made it clear that, contrary to how it looks now, MRI progress has in fact stopped

The original graph from 2005's "The Singularity Is Near," showing 100 microns resolution in the year 2000, is available from that book's website:
http://www.singularity.com/charts/page159.html



Edited at 2013-02-03 07:45 pm (UTC)
(Reply) (Thread)
[User Picture]From: squid314
2013-02-04 09:19 am (UTC)
"We did? News to me. Stuart at FHI did almost all the work. The only thing SI/MIRI did was to let him use our volunteer network at SingularityVolunteers.org to find evaluators."

Sorry about that. I was confusing it with Stuart's AI timeline work, and have corrected the error.

I've also corrected linear -> exponential. I blame too many log plots.
(Reply) (Parent) (Thread)
From: deiseach
2013-02-03 07:32 pm (UTC)
Agreed that that is a bad graph. In biology, particularly as it pertains to evolution, the old cherished notion of one straight line of progress from the first cells up to marvellous, wonderful US!!! has long been discarded. Nature isn't selecting for anything but survival, and those long-ago straight line graphs are better represented by throwing a pot of spaghetti at the wall.

A human isn't any more 'evolved' than a bacterium, if both are thriving in their own ecological niche.

So straight-line graphs in any discipline - history, technology, etc. - are inherently flawed, and picking one that goes from "Life crawls out of the sea ---> Galactic artificial intelligence!" is nothing more (or less) than science fiction.

I think it's probably true that periodically, humans hit a plateau where we've gone about as far as we can do in certain areas, and it's always tempting to conclude that we've gone about as far as we can go - but then along comes someone who has just invented a flying machine, or an automobile, or a computer, and a whole new field of invention and advance opens up. Right now, somebody is probably working on something incredible that will change culture in ways we can't even imagine, until it happens.

Look how fast we went from "Mobile phones? That's science fiction!" to the "portable brick" models to the devices nowadays that do so much more than merely taking and making calls, and the effects on society of this constant availability, linked-in-ness, and capacity to work on mobile devices.
(Reply) (Thread)
[User Picture]From: st_rev
2013-02-04 07:02 pm (UTC)
This seems important: the resolution limit on NMR/MRI appears to have been smashed. http://pipeline.corante.com/archives/2013/02/04/singlecell_nmr_how_about_singleprotein_nmr.php

Probably not directly applicable to medical scanning, though.

Edited at 2013-02-04 07:02 pm (UTC)
(Reply) (Thread)
From: (Anonymous)
2013-02-14 12:50 am (UTC)

How to buy a best celine handbags sale online store

zjbkc [url=http://www.pickcelinehandbags.com]celine bags[/url] mewmbm http://www.pickcelinehandbags.com ilwgi [url=http://www.savecelinehandbags.com]canada handbags[/url] tyhsbw http://www.savecelinehandbags.com lqtfx [url=http://www.savecelinebags.com]cheap celine bags[/url] rntdgd http://www.savecelinebags.com nhfn [url=http://www.goodcelinebags.com]celine handbags[/url] jrnsdp http://www.goodcelinebags.com lylzc [url=http://www.goodcelinehandbags.com]cheap celine bag[/url] lsshjk http://www.goodcelinehandbags.com rkpbk [url=http://www.onlinecelinebags.com]celine bag online[/url] etwiwd http://www.onlinecelinebags.com jqpn
(Reply) (Thread)
From: (Anonymous)
2013-02-21 04:33 pm (UTC)
I really don't see why there is any evidence that extrapolating based on innovations in history should lead to valid data. There is no reason to expect "innovations" to have any kind of consistent curve, other then the effect of feedback. For all we know, though, we will be done in 20 years or something, and all the remaining problems will be inherently intractable. The idea that the potential of science and technology and culture "keeps going" is as religious as any.
(Reply) (Thread)
From: (Anonymous)
2013-03-13 11:01 pm (UTC)
[url=http://www.toryburchfactorysale.com/][img]http://www.toryburchfactorysale.com/images/flat/NF9063.jpg[/img][/url]

[url=http://www.toryburchfactorysale.com/%E3%83%88%E3%83%AA%E3%83%BC%E3%83%90%E3%83%BC%E3%83%81-%E3%83%99%E3%83%AB%E3%83%88-c-106.html] トリーバーチ ベルト [/url], [url=http://www.oakleyfactorysale.com/oakley%C2%A0radar%C2%A0lock-c-75.html] オークリー レーダーロック [/url], [url=http://www.cheapmiumiujas.com/] ミュウミュウ 直営店 [/url]

[url=http://www.toryburchfactorysale.com/][img]http://www.toryburchfactorysale.com/images/TBflat/TF821.jpg[/img][/url]

[url=http://www.toryburchfactorysale.com/%E3%83%88%E3%83%AA%E3%83%BC%E3%83%90%E3%83%BC%E3%83%81-%E8%B2%A1%E5%B8%83-c-104.html] トリーバーチ 長財布 [/url], [url=http://www.marcjacobsbagsjas.com/marc-by-marc-jacobs-classic-q-large-zip-around-wallet-p-195.html] マークバイマークジェイコブス Classic Q 財布 [/url], [url=http://www.discountpaulsmithjpn.com/%E3%83%9D%E3%83%BC%E3%83%AB%E3%82%B9%E3%83%9F%E3%82%B9-%E6%B4%97%E9%9D%A2%E3%83%BB%E3%83%A1%E3%82%A4%E3%82%AF%E3%80%80%E3%83%90%E3%83%83%E3%82%B0-c-5.html] ポールスミス ケース [/url]

[url=http://www.toryburchfactorysale.com/][img]http://www.toryburchfactorysale.com/images/flat/NF9063.jpg[/img][/url]

[url=http://www.marcjacobsbagsjas.com/%E3%83%9E%E3%83%BC%E3%82%AF%E3%83%90%E3%82%A4%E3%83%9E%E3%83%BC%E3%82%AF-%E3%83%90%E3%83%83%E3%82%B0-c-37.html] マークジェイコブス バッグ [/url], [url=http://www.toryburchfactorysale.com/] トリーバーチ 靴 [/url], [url=http://www.marcjacobsbagsjas.com/%E3%83%9E%E3%83%BC%E3%82%AF%E3%83%90%E3%82%A4%E3%83%9E%E3%83%BC%E3%82%AF-%E3%83%90%E3%83%83%E3%82%B0-c-37.html] マークバイマークジェイコブス バッグ [/url]

[url=http://www.miumiujan2013.com]miumiu 財布[/url]
http://www.sr2lb.com/face/index.php?do=/blog/32022/123-02-marc-jacobs-c-847/
http://netenet.in/index.php?do=/blog/158947/13-02-marc-jacobs-v-318/
http://www.ourfull.net/sudanbook/index.php?do=/blog/20054/13-02-paul-smith-g-247/
http://manggaixinh.com/index.php?do=/blog/110839/123-02-marc-jacobs-j-945/
http://sweet-book.x-arena.eu/index.php?do=/blog/237798/123-02-marc-jacobs-t-469/
http://www.asdasdweeer.com/blog/31810/123-02-paul-smith-q-756/
http://www.onlinefriends.in/index.php?do=/blog/9469/123-02-mbts-j-562/
http://www.iw9hff.it/libro/index.php?do=/blog/4152/982-02-marc-jacobs-o-675/
(Reply) (Thread)
From: (Anonymous)
2013-03-20 03:40 pm (UTC)

Buy cheap louis vuitton bags sale online here

ublycq http://louisvuittonooutlet-sales.com eslegv http://beatsbydree-shop.com xelmwy http://louisvuittonb.webs.com ygxviq http://nikeairmaxa.webs.com sbvcki http://louisvuittonooutlet-mm.com axwdur [url=http://louisvuittonooutlet-sales.com]louis vuitton sale[/url] oahxl [url=http://beatsbydree-shop.com]dr dre beats[/url] cpthb [url=http://louisvuittonb.webs.com]louis vuitton sale[/url] nafee [url=http://nikeairmaxa.webs.com]cheap nike shoes[/url] nwjvpr [url=http://louisvuittonooutlet-mm.com]louis vuitton handbags[/url] rqsy
(Reply) (Thread)