After the Cybernetic Totalism talk, the Emergent Epistemology Salon wanted to hunt around for something brilliantly new in the ballpark of "general reasoning" that could actually be implemented. To that end we looped back to something we talked about back in May of 2007 and met again on on Jan 13, 2008 to talk about...
Manifesto for an Evolutionary Economics of Intelligence
Eric B. Baum, 1998
PARTIAL ABSTRACT: We address the problem of reinforcement learning in ultra-complex environments. Such environments will require a modular approach. The modules must solve subproblems, and must collaborate on solution of the overall problem. However a collection of rational agents will only collaborate if appropriate structure is imposed. We give a result, analogous to the First Theorem of Welfare Economics, that shows how to impose such structure. That is, we describe how to use economic principles to assign credit and ensure that a collection of rational (but possibly computationally limited) agents will collaborate on reinforcement learning. Conversely, we survey catastrophic failure modes that can be expected in distributed learning systems, and empirically have occurred in biological evolution, real economics, and artificial intelligence programs, when such structure was not enforced.
We conjecture that simulated economies can evolve to reinforcement learn in complex environments in feasible time scales, starting from a collection of agents which have little knowledge and hence are *not* rational. We support this with two implementations of learning models based on these principles.
Compared to the previous discussion on this paper, we were more focused on the algorithm itself instead of on the broader claims about the relevance of economics to artificial intelligence. We were also more focused on a general theme the group has been following - the ways that biases in an optimizing process are (or are not) suited to the particularities of of given learning problem.
We covered some of the same ideas related to the oddness that a given code fragment within the system needed both domain knowledge and "business sense" in order to survive. Brilliant insights that are foolishly sold for less than their CPU costs might be deleted and at the same time, the potential for "market charlatanism" might introduce hiccups in the general system's ability to learn. By analogy to the real world, is it easier to invent an economically viable fusion technology or to defraud investors with a business that falsely claims to have an angle on such technology?
We also talked about the reasons real economies are so useful - they aggregate information from many contexts and agents into a single data point (price) that can be broadcast to all contexts to help agents in those contexts solve their local problems more effectively. It's not entirely clear how well the analogy from real economies maps into the idea of a general learning algorithm. You already have a bunch of agents in the real world. And there's already all kinds of structure (physical distance and varied resources and so on) in the world. The scope of the agents is already restricted to what they have at hand and the expensive problem that economics solves is "getting enough of the right information to all the dispersed agents". In a computer, with *random access* memory, the hard part is discovering and supporting structure in giant masses of data the first place. It seemed that economic inspiration might be a virtue in the physical world due the necessities imposed by the physical world. Perhaps something more cleanly mathematical would be better inside a computer?
Finally, there was discussion around efforts to re-implement the systems described in the paper and how different re-implementation choices might improve or hurt the performance.
Showing posts with label Society. Show all posts
Showing posts with label Society. Show all posts
Monday, April 28, 2008
Thursday, December 6, 2007
Law's Order
It's been a while since I posted anything, but that's mostly for lack of time to post rather than for lacking of meetings worth posting about. We held a salon on Sunday, October 14th 2007 to discuss the a book by David Friedman (son of Milton):
Law's Order
(The above link goes to the full online text, alternately try Amazon or Google Books)
From the Google Books blurb: "Suppose legislators propose that armed robbers receive life imprisonment. Editorial pages applaud them for getting tough on crime. Constitutional lawyers raise the issue of cruel and unusual punishment. Legal philosophers ponder questions of justness. An economist, on the other hand, observes that making the punishment for armed robbery the same as that for murder encourages muggers to kill their victims. This is the cut-to-the-chase quality that makes economics not only applicable to the interpretation of law, but beneficial to its crafting. Drawing on numerous commonsense examples, in addition to his extensive knowledge of Chicago-school economics, David D. Friedman offers a spirited defense of the economic view of law. He clarifies the relationship between law and economics in clear prose that is friendly to students, lawyers, and lay readers without sacrificing the intellectual heft of the ideas presented. Friedman is the ideal spokesman for an approach to law that is controversial not because it overturns the conclusions of traditional legal scholars--it can be used to advocate a surprising variety of political positions, including both sides of such contentious issues as capital punishment--but rather because it alters the very nature of their arguments. For example, rather than viewing landlord-tenant law as a matter of favoring landlords over tenants or tenants over landlords, an economic analysis makes clear that a bad law injures both groups in the long run."
The book covered basic economic concepts such as economic efficiency, externalities, and Coase's Therem. The author's honesty about the way the concept of "property" involves a substantial amount of work and thinking to get right is charming and intellectually productive.
When something is owned, in a rather deep sense what's owned in is not simply "a thing" but a complicated bundle of rights related to the thing. With "my land", for example, there are: the right to build on land, the right to not have certain things built on neighboring land, the right to control the movement of physical objects through airspace above the land, the right to the minerals beneath the surface, the right to have it supported by neighboring land, the right to make loud noises on the edge of the land, and so on and so forth.
Once bundling of rights is acknowledged to be going on in potentially arbitrary ways it opens up the discussion to questions about how rights relating to different things should be bundled, who should initially own the bundles, what sort of transfer schemes should hold. Mr. Friedman takes a position on what should be happening but it's rather complicated. Chapter 5 has a "spaghetti diagram" showing a variety of possible initial assignments of different kinds of rights, further ramified by the relative costs and benefits that accrue to various outcomes after rights have been renegotiated by various means (up front purchase contract, after the fact court dispute, etc).
The "thing that should happen" isn't a single way of doing things but a situationally sensitive rule that requires estimation of the costs and benefits parties on each side of an allocation of rights faces, and further guesses about who is likely to be able to see how many of the costs and benefits (the parties involved, the courts, etc), recognition of the number of people owning various rights and how many people any particular agent would have to negotiate with in order to get anything useful, and estimates of transaction costs (like the cost of making all these estimates) to boot.
If the initial assignment of rights is done poorly, various game theoretic barriers to collective action can arise. Given the complexity of the decision, this is not necessarily a conclusion that inspires happiness and hope. Mr. Friedman discusses institutions for working through these issues, including an examination of the claim that the best institution for achieving economically efficient outcomes in the long run is common law.
Our discussion of the book ranged rather widely. One of the juiciest veins of thought we found was in the question of bundling rights in novel ways and trying to understand how they might be rebundled by the market over time. For example, the idea of "salesright" (inspired by copyright) was a sort of "horrifying or amazing" concept that fell out of the discussions. Sales right would be "the right to a sale given certain propaganda efforts". If one company advertised at you, and you ended up buying something in their industry (when you wouldn't otherwise have done so) but you buy something from one of their competitors... in some sense the competitor has "stolen a sale" that "rightfully" belonged to the company that paid for the advertisement. (And you thought patents and copyright were bad :-P)
Another theme we examined (that Mr. Friedman mostly ignored) was the similarity between the questions of rights bundling and what, in in modern philosophy, is known as the Goodman's new problem of induction.
Law's Order
(The above link goes to the full online text, alternately try Amazon or Google Books)
From the Google Books blurb: "Suppose legislators propose that armed robbers receive life imprisonment. Editorial pages applaud them for getting tough on crime. Constitutional lawyers raise the issue of cruel and unusual punishment. Legal philosophers ponder questions of justness. An economist, on the other hand, observes that making the punishment for armed robbery the same as that for murder encourages muggers to kill their victims. This is the cut-to-the-chase quality that makes economics not only applicable to the interpretation of law, but beneficial to its crafting. Drawing on numerous commonsense examples, in addition to his extensive knowledge of Chicago-school economics, David D. Friedman offers a spirited defense of the economic view of law. He clarifies the relationship between law and economics in clear prose that is friendly to students, lawyers, and lay readers without sacrificing the intellectual heft of the ideas presented. Friedman is the ideal spokesman for an approach to law that is controversial not because it overturns the conclusions of traditional legal scholars--it can be used to advocate a surprising variety of political positions, including both sides of such contentious issues as capital punishment--but rather because it alters the very nature of their arguments. For example, rather than viewing landlord-tenant law as a matter of favoring landlords over tenants or tenants over landlords, an economic analysis makes clear that a bad law injures both groups in the long run."
The book covered basic economic concepts such as economic efficiency, externalities, and Coase's Therem. The author's honesty about the way the concept of "property" involves a substantial amount of work and thinking to get right is charming and intellectually productive.
When something is owned, in a rather deep sense what's owned in is not simply "a thing" but a complicated bundle of rights related to the thing. With "my land", for example, there are: the right to build on land, the right to not have certain things built on neighboring land, the right to control the movement of physical objects through airspace above the land, the right to the minerals beneath the surface, the right to have it supported by neighboring land, the right to make loud noises on the edge of the land, and so on and so forth.
Once bundling of rights is acknowledged to be going on in potentially arbitrary ways it opens up the discussion to questions about how rights relating to different things should be bundled, who should initially own the bundles, what sort of transfer schemes should hold. Mr. Friedman takes a position on what should be happening but it's rather complicated. Chapter 5 has a "spaghetti diagram" showing a variety of possible initial assignments of different kinds of rights, further ramified by the relative costs and benefits that accrue to various outcomes after rights have been renegotiated by various means (up front purchase contract, after the fact court dispute, etc).
The "thing that should happen" isn't a single way of doing things but a situationally sensitive rule that requires estimation of the costs and benefits parties on each side of an allocation of rights faces, and further guesses about who is likely to be able to see how many of the costs and benefits (the parties involved, the courts, etc), recognition of the number of people owning various rights and how many people any particular agent would have to negotiate with in order to get anything useful, and estimates of transaction costs (like the cost of making all these estimates) to boot.
If the initial assignment of rights is done poorly, various game theoretic barriers to collective action can arise. Given the complexity of the decision, this is not necessarily a conclusion that inspires happiness and hope. Mr. Friedman discusses institutions for working through these issues, including an examination of the claim that the best institution for achieving economically efficient outcomes in the long run is common law.
Our discussion of the book ranged rather widely. One of the juiciest veins of thought we found was in the question of bundling rights in novel ways and trying to understand how they might be rebundled by the market over time. For example, the idea of "salesright" (inspired by copyright) was a sort of "horrifying or amazing" concept that fell out of the discussions. Sales right would be "the right to a sale given certain propaganda efforts". If one company advertised at you, and you ended up buying something in their industry (when you wouldn't otherwise have done so) but you buy something from one of their competitors... in some sense the competitor has "stolen a sale" that "rightfully" belonged to the company that paid for the advertisement. (And you thought patents and copyright were bad :-P)
Another theme we examined (that Mr. Friedman mostly ignored) was the similarity between the questions of rights bundling and what, in in modern philosophy, is known as the Goodman's new problem of induction.
Friday, June 8, 2007
Games as Symbolic Bottlenecks
As UCSD's quarter winds down, the Emergent Epistemology Salon met on June 3, 2007 and talked about the ideas in the final paper of Justin, one of our members. The conversation was about "games" taking the idea very broadly to include actual games like chess but (fuzzy definition ahead!) to also include nearly any interaction that can be simplified to the point that non-human participation starts being feasible. Games offer their participants "symbolic bottlenecks".
Three snippets from a very early draft of Justin's writing:
Groups of humans give rise to a vast array of organizational patterns found in no other system in the natural world. This has, of late, given rise to the suggestion that humans implement "group intentionality" in one way or another.
...
Chess is primarily a person-person interaction taking place over a board in physical space. It can also be played through the mail, by phone, over computers, or simply between persons calling out moves. Meaning is created during a chess game by the rule-constrained manipulation of pieces on a board, however these are instantiated. The game of chess organizes human behavior in complex and interesting ways across space and time.
The last century saw the rise of computerized chess engines. Computers, though hideously inefficient and unable to do lots of other things, now kick ass at chess.
...
Traditionally, language games are viewed as giving rise to exclusively human-human interactions. Certainly human-human interactions form a large part of our game-playing activity. But there is nothing essential to the structure of most games that requires a human opponent/participant. And even if this was a requirement, groups of humans can and do participate in games all the time.
Maybe more importantly, humans care about and invest in games (consider the parliaments of the world, religions, etc). If we find ourselves struggling in a game with entities that are not obviously reducible to one agent (the phone company, NIMBY, Quebec), or entities that are non-human (the phone company's automated devil box), we care, because the moves in the game matter to us.
There is a sense in which our investiture in games opens the door to intervention by entities other than individual humans.
If we are committed to the belief that many meaningful human activities are best understood as game-like interactions, and we discover non-human entities that can play our games with us (or even impose them upon us), what are we to make of the activity that results from our interaction with these entities? Is it meaningful in the same way human-human interaction is meaningful? What does it say about the other party? What does it say about us?
Three snippets from a very early draft of Justin's writing:
Groups of humans give rise to a vast array of organizational patterns found in no other system in the natural world. This has, of late, given rise to the suggestion that humans implement "group intentionality" in one way or another.
...
Chess is primarily a person-person interaction taking place over a board in physical space. It can also be played through the mail, by phone, over computers, or simply between persons calling out moves. Meaning is created during a chess game by the rule-constrained manipulation of pieces on a board, however these are instantiated. The game of chess organizes human behavior in complex and interesting ways across space and time.
The last century saw the rise of computerized chess engines. Computers, though hideously inefficient and unable to do lots of other things, now kick ass at chess.
...
Traditionally, language games are viewed as giving rise to exclusively human-human interactions. Certainly human-human interactions form a large part of our game-playing activity. But there is nothing essential to the structure of most games that requires a human opponent/participant. And even if this was a requirement, groups of humans can and do participate in games all the time.
Maybe more importantly, humans care about and invest in games (consider the parliaments of the world, religions, etc). If we find ourselves struggling in a game with entities that are not obviously reducible to one agent (the phone company, NIMBY, Quebec), or entities that are non-human (the phone company's automated devil box), we care, because the moves in the game matter to us.
There is a sense in which our investiture in games opens the door to intervention by entities other than individual humans.
If we are committed to the belief that many meaningful human activities are best understood as game-like interactions, and we discover non-human entities that can play our games with us (or even impose them upon us), what are we to make of the activity that results from our interaction with these entities? Is it meaningful in the same way human-human interaction is meaningful? What does it say about the other party? What does it say about us?
Sunday, May 27, 2007
The Wisdom Economy
The second meeting wasn't exactly two weeks later... But on April 22, 2007 the EES met to talk about something we lumped under the label of the "Wisdom Economy" (though that term doesn't speak very well to the algorithmic angle we were focusing on). These were the readings for the meeting:
TOOL: The Open Opinion Layer
http://www.firstmonday.org/issues/issue7_7/masum/
2002, Hassan Masum
Shared opinions drive society: what we read, how we vote, and where we shop are all heavily influenced by the choices of others. However, the cost in time and money to systematically share opinions remains high, while the actual performance history of opinion generators is often not tracked. This article explores the development of a distributed open opinion layer, which is given the generic name of TOOL. Similar to the evolution of network protocols as an underlying layer for many computational tasks, we suggest that TOOL has the potential to become a common substrate upon which many scientific, commercial, and social activities will be based. Valuation decisions are ubiquitous in human interaction and thought itself. Incorporating information valuation into a computational layer will be as significant a step forward as our current communication and information retrieval layers.
Automated Collaborative Filtering and Semantic Transports
http://www.lucifer.com/~sasha/articles/ACF.html
1997, Alexander Chislenko
This essay focuses on the conceptualization of the issues, comparisons of current technological developments to other historical/evolutionary processes, future of automated collaboration and its implications for economic and social development of the world, and suggestions of what we may want to pursue and avoid. Explanations of the workings of the technology and analysis of the current market are not my purpose here, although some explanations and examples may be appropriate.
TOOL: The Open Opinion Layer
http://www.firstmonday.org
2002, Hassan Masum
Shared opinions drive society: what we read, how we vote, and where we shop are all heavily influenced by the choices of others. However, the cost in time and money to systematically share opinions remains high, while the actual performance history of opinion generators is often not tracked. This article explores the development of a distributed open opinion layer, which is given the generic name of TOOL. Similar to the evolution of network protocols as an underlying layer for many computational tasks, we suggest that TOOL has the potential to become a common substrate upon which many scientific, commercial, and social activities will be based. Valuation decisions are ubiquitous in human interaction and thought itself. Incorporating information valuation into a computational layer will be as significant a step forward as our current communication and information retrieval layers.
Automated Collaborative Filtering and Semantic Transports
http://www.lucifer.com/~sasha
1997, Alexander Chislenko
This essay focuses on the conceptualization of the issues, comparisons of current technological developments to other historical/evolutionary processes, future of automated collaboration and its implications for economic and social development of the world, and suggestions of what we may want to pursue and avoid. Explanations of the workings of the technology and analysis of the current market are not my purpose here, although some explanations and examples may be appropriate.
A.I. as a Positive and Negative Factor in Global Risk
Starting out with some back history :-)
Anna and I (more her than me, she's the inspired one :-)) schemed up something we ended up calling the Emergent Epistemology Salon (EES) for lack of a less painfully trendy name. The first get together was back on March 25, 2007 and as things have bounced along we kept telling ourselves it would help us write stuff and work up a "big picture" if we had a place to post thoughts and links and whatnot.
So, I'm going to start a tag called "Discussion" and every time we meet (that I have time for it) I'll post a link to what we talked about under that tag. That should ensure at least one post to the blog every two weeks or so... Here was the first thing we talked about back in March:
Artificial Intelligence as a Positive and Negative Factor in Global Risk
Eliezer Yudkowsky, 2006
An essay on the dangers and benefits of (succeeding at) building a general artificial intelligence. Of general theoretical interest is the discussion of the human reasoning biases that seem to lead many people to radically over estimate the degree to which they "understand intelligence".
--
I'm still trying to figure out what the right dynamics for "the blog version of the salon" should be. The meeting was pretty good and two of us (me and another person) wrote up a summary of where the discussion went. Those summaries might be worth posting? Or not? Or maybe it would be invasive of private conversation? I think I'll ask first and then post my summary here as a comment unless someone vetoes the idea.
Anna and I (more her than me, she's the inspired one :-)) schemed up something we ended up calling the Emergent Epistemology Salon (EES) for lack of a less painfully trendy name. The first get together was back on March 25, 2007 and as things have bounced along we kept telling ourselves it would help us write stuff and work up a "big picture" if we had a place to post thoughts and links and whatnot.
So, I'm going to start a tag called "Discussion" and every time we meet (that I have time for it) I'll post a link to what we talked about under that tag. That should ensure at least one post to the blog every two weeks or so... Here was the first thing we talked about back in March:
Artificial Intelligence as a Positive and Negative Factor in Global Risk
Eliezer Yudkowsky, 2006
An essay on the dangers and benefits of (succeeding at) building a general artificial intelligence. Of general theoretical interest is the discussion of the human reasoning biases that seem to lead many people to radically over estimate the degree to which they "understand intelligence".
--
I'm still trying to figure out what the right dynamics for "the blog version of the salon" should be. The meeting was pretty good and two of us (me and another person) wrote up a summary of where the discussion went. Those summaries might be worth posting? Or not? Or maybe it would be invasive of private conversation? I think I'll ask first and then post my summary here as a comment unless someone vetoes the idea.
Subscribe to:
Posts (Atom)