Where do institutions come from? And what kinds of social forces are at work to stabilize them once they are up and running? These are questions that historical institutionalists like Kathleen Thelen have considered in substantial depth (link, link, link). But the rational-choice paradigm has also offered some answers to these questions as well. The basic idea presented by the RCT paradigm is that institutions are the result of purposive agents coping with existential problems, forming alliances, and pursuing their interests in a rational way. James Coleman is one of the exponents of this approach in Foundations of Social Theory, where he treats institutions and norms as coordinated and mutually reinforcing patterns of individual behavior (link).
An actor-centered theory of institutions requires a substantial amount of boot-strapping: we need to have an account of how a set of rules and practices could have emerged from the purposive but often conflictual activities of individuals, and we need a similar account of how those rules are stabilized and enforced by individuals who have no inherent interest in the stability of the rules within which they act. Further, we need to take account of well-known conflicts between private and public benefits, short-term and long-term benefits, and intended and unintended benefits. Rational-choice theorists since Mancur Olson in The Logic of Collective Action: Public Goods and the Theory of Groups have made it clear that we cannot explain social outcomes on the basis of the collective benefits that they provide; rather, we need to show how those arrangements result from relatively myopic, relatively self-interested actors with bounded ability to foresee consequences.
Ken Shepsle is a leading advocate for a rational-choice theory of institutions within political science. He offers an exposition of his thinking in his contribution to The Oxford Handbook of Political Institutions (link). He distinguishes between institutions as exogenous and institutions as endogenous. The first conception takes the rules and practices of an institution as fixed and external to the individuals who operate within them, while the second looks at the rules and practices as being the net result of the intentions and actions of those individuals themselves. On the second view, it is open to the individuals within an activity to attempt to change the rules; and one set of rules will perhaps have better results for one set of interests than another. So the choice of rules in an activity is not a matter of indifference to the participants. (For example, untenured faculty might undertake a campaign to change the way the university evaluates teaching in the context of the tenure process, or to change the relative weights assigned to teaching and research.) Shepsle also distinguishes between structured and unstructured institutions -- a distinction that other authors characterize as "formal/informal". The distinction has to do with the degree to which the rules of the activity are codified and reinforced by strong external pressures. Shepsle encompasses various informal solutions to collective action problems under the rubric of unstructured institutions -- fluid solutions to a transient problem.
This description of institutions begins to frame the problem, but it doesn't go very far. In particular, it doesn't provide much insight into the dynamics of conflict over rule-setting among parties with different interests in a process. Other scholars have pushed the analysis further.
French sociologists Crozier and Friedberg address this problem in Actors and Systems: The Politics of Collective Action (1980 [1977]). Their premise is that actors within organizations have substantially more agency and freedom than they are generally afforded by orthodox organization theory, and we can best understand the workings and evolution of the organization as (partially) the result of the strategic actions of the participants (instead of understanding the conduct of the participants as a function of the rules of the organization). They look at institutions as solutions to collective action problems -- tasks or performances that allow attainment of a goal that is of interest to a broad public, but for which there are no antecedent private incentives for cooperation. Organized solutions to collective problems -- of which organizations are key examples -- do not emerge spontaneously; instead, "they consist of nothing other than solutions, always specific, that relatively autonomous actors have created, invented, established, with their particular resources and capacities, to solve these challenges for collective action" (15). And Crozier and Friedberg emphasize the inherent contingency of these particular solutions; there are always alternative solutions, neither better nor worse. This is a rational-choice analysis, though couched in sociological terms rather than economists' terms. (Here is a more extensive discussion of Crozier and Friedberg; link.)
Jack Knight brings conflict and power into the rational-choice analysis of the emergence of institutions in Institutions and Social Conflict
.I argue that the emphasis on collective benefits in theories of social institutions fails to capture crucial features of institutional development and change. I further argue that our explanations should invoke the distributional effects of such institutions and the conflict inherent in those effects. This requires an investigation of those factors that determine how these distributional conflicts are resolved. (13-14)
Institutions are not created to constrain groups or societies in an effort to avoid suboptimal outcomes but, rather, are the by-product of substantive conflicts over the distributions inherent in social outcomes. (40)Knight believes that we need to have microfoundations for the ways in which institutions emerge and behave (14), and he seeks those mechanisms in the workings of rational choices by the participants within the field of interaction within which the institution emerges.
Actors choose their strategies under various circumstances. In some situations individuals regard the rest of their environment, including the actions of others, as given. They calculate their optimal strategy within the constraints of fixed parameters.... But actors are often confronted by situations characterized by an interdependence between other actors and themselves.... Under these circumstances individuals must choose strategically by incorporating the expectations of the actions of others into their own decision making. (17)This implies, in particular, that we should not expect socially optimal or efficient outcomes in the emergence of institutions; rather, we should expect institutions that differentially favor the interests of some groups and disfavor those of other groups -- even if the social total is lower than a more egalitarian arrangement.
I conclude that social efficiency cannot provide the substantive content of institutional rules. Rational self-interested actors will not be the initiators of such rules if they diminish their own utility. Therefore rational-choice explanations of social institutions based on gains in social efficiency fail as long as they are grounded in the intentions of social actors. (34)Knight's work explicitly refutes the occasional Panglossian (or Smithian) assumptions sometimes associated with rational choice theory and micro-economics: the idea that individually rational action leads to a collectively efficient outcome (the invisible hand). This may be true in the context of certain kinds of markets; but it is not generally true in the social and political world. And Knight shows in detail how the assumption fails in the case of institutional emergence and ongoing workings.
Rational choice theory is one particular and specialized version of actor-centered social science (link). It differs from other approaches in the very narrow assumptions it makes about the actor's particular form of agency; it assumes narrow economic rationality rather than a broader conception of agency or practical rationality (link). What seems clear to me is that we need to take an actor-centered approach if we want to understand institutions -- either their emergence or their continuing functioning and change. So the approach taken by rational-choice theorists is ontologically correct. If RCT fails to provide an adequate analysis of institutions, it is because the underlying theory of agency is fundamentally unrealistic about human actors.
