research-article

Eliciting Predictions and Recommendations for Decision Making

Authors:

Michael Ruberry,

Victor ShnayderAuthors Info & Claims

ACM Transactions on Economics and Computation (TEAC), Volume 2, Issue 2

Article No.: 6, Pages 1 - 27

https://doi.org/10.1145/2556271

Published: 01 June 2014 Publication History

Abstract

When making a decision, a decision maker selects one of several possible actions and hopes to achieve a desirable outcome. To make a better decision, the decision maker often asks experts for advice. In this article, we consider two methods of acquiring advice for decision making. We begin with a method where one or more experts predict the effect of each action and the decision maker then selects an action based on the predictions. We characterize strictly proper decision making, where experts have an incentive to accurately reveal their beliefs about the outcome of each action. However, strictly proper decision making requires the decision maker use a completely mixed strategy to choose an action. To address this limitation, we consider a second method where the decision maker asks a single expert to recommend an action. We show that it is possible to elicit the decision maker’s most preferred action for a broad class of preferences of the decision maker, including when the decision maker is an expected value maximizer.

References

[1]

Jacob Abernethy, Yiling Chen, and Jennifer Wortman Vaughan. 2013. Efficient market making via convex optimization, and a connection to online learning. ACM Trans. Econ. Comp. 1, 2 (May), 1--39.

Digital Library

[2]

José C. R. Alcantud and Carlos Rodríguez-Palmero. 1999. Characterization of the existence of semicontinuous weak utilities. J. Math. Econ. 32, 4 (Dec.), 503--509.

[3]

Joyce E. Berg and Thomas A. Rietz. 2003. Prediction markets as decision support systems. Inform. Syst. Frontier 5, 79--93.

Digital Library

[4]

Craig Boutilier. 2012. Eliciting forecasts from self-interested experts: Scoring rules for decision markets. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 737--744.

Digital Library

[5]

Glenn W. Brier. 1950. Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78, 1, 1--3.

[6]

Yiling Chen, Stanko Dimitrov, Rahual Sami, Daniel M. Reeves, David M. Pennock, Robin D. Hanson, Lance Fortnow, and Rica Gonen. 2010. Gaming prediction markets: Equilibrium strategies with a market maker. Algorithmica 58, 4, 930--969.

Digital Library

[7]

Yiling Chen, Ian Kash, Mike Ruberry, and Victor Shnayder. 2011. Decision markets with good incentives. In Proceedings of the 7th International Conference on Internet and Network Economics (WINE). 72--83.

Digital Library

[8]

Yiling Chen and Ian A. Kash. 2011. Information elicitation for decision making. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 175--182.

Digital Library

[9]

Yiling Chen and David M. Pennock. 2007. A utility framework for bounded-loss market makers. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI). 49--56.

[10]

Yiling Chen, Mike Ruberry, and Jenn Wortman Vaughan. 2012. Designing informative securities. In Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (UAI). 185--195.

[11]

Yiling Chen and Jennifer Wortman Vaughan. 2010. A new understanding of prediction markets via no-regret learning. In Proceedings of the 11th ACM Conference on Electronic Commerce (EC). 189--198.

Digital Library

[12]

Robert Forsythe, Forrest Nelson, George R. Neumann, and Jack Wright. 1992. Anatomy of an experimental political stock market. Amer. Econ. Rev. 82, 5, 1142--1161.

[13]

Tilmann Gneiting and Adrian E. Raftery. 2007. Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102, 477, 359--378.

[14]

Robin Hanson. 1999. Decision markets. IEEE Intell. Syst. 14, 3, 16--19.

Digital Library

[15]

Robin D. Hanson. 2003. Combinatorial information market design. Inform. Syst. Frontiers 5, 1, 107--119.

Digital Library

[16]

Robin D. Hanson. 2007. Logarithmic market scoring rules for modular combinatorial information aggregation. J. Predict. Markets 1, 1, 1--15.

[17]

Arlo D. Hendrickson and Robert J. Buehler. 1971. Proper scores for probability forecasters. Ann. Math. Statist. 42, 6, 1916--1921.

[18]

Krishnamur Iyer, Ramesh Johari, and Ciamac C. Moallemi. 2010. Information aggregation in smooth markets. In Proceedings of the 11th ACM Conference on Electronic Commerce (EC). 199--205.

Digital Library

[19]

Bernard Mangold, Mike Dooley, Rael Dornfest, Gary W. Flake, Havi Hoffman, Tejaswi Kasturi, and David M. Pennock. 2005. The tech buzz game. IEEE Comput. 38, 7, 94--97.

Digital Library

[20]

John McCarthy. 1956. Measures of the value of information. Proc. Nat. Acad. Sciences USA 42, 9, 654--655.

[21]

Kent Osband. 1989. Optimal forecasting incentives. J. Polit. Econ. 97, 5, 1091--1112.

[22]

Michael Ostrovsky. 2012. Information aggregation in dynamic markets with strategic traders. Econometrica 80, 6, 2595--2648.

[23]

Abraham Othman and Tuomas Sandholm. 2010a. Automated market making in the large: The Gates Hillman prediction market. In Proceedings of the 11th ACM Conference on Electronic Commerce (EC). 367--376.

Digital Library

[24]

Abraham Othman and Tuomas Sandholm. 2010b. Decision rules and decision markets. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 625--632.

Digital Library

[25]

Abraham Othman, Tuomas Sandholm, David M. Pennock, and Daniel M. Reeves. 2010. A practical liquidity-sensitive automated market maker. In Proceedings of the 11th ACM Conference on Electronic Commerce (EC). 377--386.

Digital Library

[26]

David M. Pennock. 2004. A dynamic pari-mutuel market for hedging, wagering, and information aggregation. In Proceedings of the 5th ACM Conference on Electronic Commerce (EC). 170--179.

Digital Library

[27]

Josep E. Peris and Begoa Subiza. 1995. A weak utility function for acyclic preferences. Econ. Lett. 48, 1, 21--24.

[28]

Charles R. Plott, Jorgen Wit, and Winston C. Yang. 2003. Parimutuel betting markets as information aggregation devices: Experimental results. Econ. Theory 22, 2, 311--351.

[29]

Leonard J. Savage. 1971. Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66, 336, 783--801.

[30]

Peng Shi, Vincent Conitzer, and Mingyu Guo. 2009. Prediction mechanisms that do not incentivize undesirable actions. In Proceedings of the 5th International Workshop on Internet and Network Economics (WINE). 89--100.

Digital Library

Cited By

Wang WPfeiffer TDastani MSichman JAlechina NDignum V(2024)Decision Market Based Learning for Multi-agent Contextual Bandit ProblemsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663223(2549-2551)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663223
Oesterheld CTreutlein JCooper EHudson REvans RShpitser I(2023)Incentivizing honest performative predictions with proper scoring rulesProceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence10.5555/3625834.3625981(1564-1574)Online publication date: 31-Jul-2023
https://dl.acm.org/doi/10.5555/3625834.3625981
Wang WPfeiffer T(2021)Securities Based Decision MarketsDistributed Artificial Intelligence10.1007/978-3-030-94662-3_6(79-92)Online publication date: 17-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-030-94662-3_6
Show More Cited By

Index Terms

Eliciting Predictions and Recommendations for Decision Making
1. Applied computing
  1. Law, social and behavioral sciences
2. Theory of computation

Recommendations

Decision rules and decision markets
AAMAS '10: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1

We explore settings where a principal must make a decision about which action to take to achieve a desired outcome. The principal elicits the probability of achieving the outcome by following each action from a self-interested (but decision-agnostic) ...
Information elicitation for decision making
AAMAS '11: The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 1

Proper scoring rules, particularly when used as the basis for a prediction market, are powerful tools for eliciting and aggregating beliefs about events such as the likely outcome of an election or sporting event. Such scoring rules incentivize a single ...
The design of AHPEC in web-based decision support system for making decision
AIC'10/BEBI'10: Proceedings of the 10th WSEAS international conference on applied informatics and communications, and 3rd WSEAS international conference on Biomedical electronics and biomedical informatics

This paper presents the architecture of the WDSS using AHPEC for making decision. The WDSS-AHPEC involves client, user interface, server, web server, database, and DSS model. The DSS model in this study is using AHPEC method. In applying AHPEC, it ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Economics and Computation

ACM Transactions on Economics and Computation Volume 2, Issue 2

June 2014

98 pages

ISSN:2167-8375

EISSN:2167-8383

DOI:10.1145/2632259

Editors:
Vincent Conitzer
Duke University
,
Preston McAfee
Google

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2014

Accepted: 01 September 2013

Revised: 01 April 2013

Received: 01 August 2012

Published in TEAC Volume 2, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Division of Computing and Communication Foundations

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang WPfeiffer TDastani MSichman JAlechina NDignum V(2024)Decision Market Based Learning for Multi-agent Contextual Bandit ProblemsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663223(2549-2551)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663223
Oesterheld CTreutlein JCooper EHudson REvans RShpitser I(2023)Incentivizing honest performative predictions with proper scoring rulesProceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence10.5555/3625834.3625981(1564-1574)Online publication date: 31-Jul-2023
https://dl.acm.org/doi/10.5555/3625834.3625981
Wang WPfeiffer T(2021)Securities Based Decision MarketsDistributed Artificial Intelligence10.1007/978-3-030-94662-3_6(79-92)Online publication date: 17-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-030-94662-3_6
Wang WEl Fallah Seghrouchni ASukthankar GAn BYorke-Smith N(2020)Implementing Securities Based Decision Markets with Stochastic Decision RulesProceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3398761.3399136(2231-2233)Online publication date: 5-May-2020
https://dl.acm.org/doi/10.5555/3398761.3399136
Oesterheld CConitzer V(2020)Minimum-Regret Contracts for Principal-Expert ProblemsWeb and Internet Economics10.1007/978-3-030-64946-3_30(430-443)Online publication date: 6-Dec-2020
https://doi.org/10.1007/978-3-030-64946-3_30
Dreber APfeiffer TAlmenberg JIsaksson SWilson BChen YNosek BJohannesson M(2015)Using prediction markets to estimate the reproducibility of scientific researchProceedings of the National Academy of Sciences10.1073/pnas.1516179112112:50(15343-15347)Online publication date: 9-Nov-2015
https://doi.org/10.1073/pnas.1516179112

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents