Bayesian regret

In game theory, Bayesian regret is the average difference between the average utility of a strategy and the ideal utility. To be precise in some context, it requires two further definitions: the domain of the average and the ideal for comparison. A similar concept is "voter satisfaction efficiency" (also called "expected voter satisfaction", or "social utility efficiency"), which is the same number divided by the ideal utility.

Social choice theory

In social choice theory, Bayesian regret is the average difference in social utility between the chosen candidate and the best candidate. It is only measurable if it is possible to know the voters' true numerical utility for each candidate – that is, in Monte Carlo simulations of virtual elections. The term Bayesian is somewhat a misnomer, really meaning only "average probabilistic"; there is no standard or objective way to create distributions of voters and candidates.

The Bayesian regret concept was recognized as useful (and used) for comparing single-winner voting systems by Bordley and Merrill, and it also was invented independently by R. J. Weber. Bordley attributed it (and the whole idea of the usefulness of "social" utility, that is, summed over all people in the population) to John Harsanyi in 1955. Note that some of these authors instead used the name "social utility efficiency" which is the same thing as Bayesian Regret up to a linear transformation. However, those earlier studies are superseded by Smith, as they included a narrower range of voting systems.

Such simulations depend heavily on assumptions. The one assumption which particularly gets in the way of comparing voting systems is how many voters will vote tactically under each voting system. Thus, the simulation will at best only give a "best" regret number (if all voters vote honestly) and a "worst" (all voters voting tactically in a situation dominated by two preexisting parties) for every system in every set of scenarios. The real utility presumably lies in between these extremes.

Smith's simulation, over a variety of scenarios, showed that range voting had the lowest regrets for either fully honest or fully tactical voters. Voting systems whose regret score with honest voting fell between these numbers (and thus might be better by this measure if they led to less strategy) include Borda count and various forms of Condorcet voting. Voting systems whose best score fell at or below the worst score for range voting were instant-runoff voting, plurality voting, approval voting and various random-ballot options included for completion.

Economics

This term has been used to compare a random buy-and-hold strategy to professional traders' records. This same concept has received numerous different names, as the New York Times notes:

"In 1957, for example, a statistician named James Hanna called his theorem Bayesian Regret. He had been preceded by David Blackwell, also a statistician, who called his theorem Controlled Random Walks. Other, later papers had titles like 'On Pseudo Games', 'How to Play an Unknown Game', 'Universal Coding' and 'Universal Portfolios'".^[1]

Machine learning

Refers to the average difference between an algorithm's performance and ideal performance, for instance on a classification task. Note that ideal performance is not perfect since the "true" classifications may be somewhat intermixed in all perceivable dimensions.^[2]

See also

• Loss function
• Regret (decision theory)

References