poster

Aligning Vertical Collection Relevance with User Intent

Authors:

Ke Zhou,

Thomas Demeester,

Dong Nguyen,

Djoerd Hiemstra,

Dolf TrieschniggAuthors Info & Claims

CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

Pages 1915 - 1918

https://doi.org/10.1145/2661829.2661941

Published: 03 November 2014 Publication History

Get Access

Abstract

Selecting and aggregating different types of content from multiple vertical search engines is becoming popular in web search. The user vertical intent, the verticals the user expects to be relevant for a particular information need, might not correspond to the vertical collection relevance, the verticals containing the most relevant content. In this work we propose different approaches to define the set of relevant verticals based on document judgments. We correlate the collection-based relevant verticals obtained from these approaches to the real user vertical intent, and show that they can be aligned relatively well. The set of relevant verticals defined by those approaches could therefore serve as an approximate but reliable ground-truth for evaluating vertical selection, avoiding the need for collecting explicit user vertical intent, and vice versa.

References

[1]

J. Arguello, F. Diaz, J. Callan, and J.-F. Crespo. Sources of evidence for vertical selection. In SIGIR'09.

Digital Library

Google Scholar

[2]

T. Demeester, D. Trieschnigg, D. Nguyen, D. Hiemstra. Overview of the TREC 2013 Federated Web Search Track. In TREC'13.

Google Scholar

[3]

L. Gravano, H. Garcia-Molina and A. Tomasic. Precision and recall of GlOSS estimators for database discovery. In IEEE Parallel and Distributed Information Systems, 1994.

Digital Library

Google Scholar

[4]

J. Kekalainen and J. Kalervo. Using graded relevance assessments in IR evaluation. In JASIST'02.

Digital Library

Google Scholar

[5]

M. Sanderson, M. L. Paramita, P. Clough, and E. Kanoulas. Do user preferences and evaluation measures line up? In SIGIR'10.

Digital Library

Google Scholar

[6]

S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors affecting click-through behavior in aggregated search interfaces. In CIKM'10.

Digital Library

Google Scholar

[7]

A. Powell and J. French. Comparing the performance of collection selection algorithms. In ACM TOIS'03.

Digital Library

Google Scholar

[8]

K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Evaluating reward and risk for vertical selection. In CIKM'12.

Digital Library

Google Scholar

[9]

K. Zhou, R. Cummins, M. Lalmas and J.M. Jose. Which Vertical Search Engines are Relevant? In WWW'13.

Digital Library

Google Scholar

Cited By

View all

Achsas SNfaoui E(2022)Academic Aggregated Search Approach Based on BERT Language Model2022 2nd International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET)10.1109/IRASET52964.2022.9737888(1-9)Online publication date: 3-Mar-2022
https://doi.org/10.1109/IRASET52964.2022.9737888
Arguello J(2018)Aggregated SearchFoundations and Trends in Information Retrieval10.1561/150000005210:5(365-502)Online publication date: 14-Dec-2018
https://dl.acm.org/doi/10.1561/1500000052
Kadotami YYoshida YFujita SSakai TKamps JKanoulas Ede Rijke MFang HYilmaz E(2017)Mobile Vertical Ranking based on Preference GraphsProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3121050.3121082(225-228)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1145/3121050.3121082
Show More Cited By

Index Terms

Aligning Vertical Collection Relevance with User Intent
1. Information systems
  1. Information retrieval

Recommendations

Evaluating intuitiveness of vertical-aware click models
SIGIR '14: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval

Modeling user behavior on a search engine result page is important for understanding the users and supporting simulation experiments. As result pages become more complex, click models evolve as well in order to capture additional aspects of user ...
The 2nd workshop on Vertical Search Relevance at WSDM 2015
WSDM '15: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining

As the web information exponentially grows and the needs of users become more specific, traditional general web search engines are not able to perfectly satisfy the nowadays user requirement. Vertical search engines have emerged in various domains, ...
The influence of commercial intent of search results on their perceived relevance
iConference '11: Proceedings of the 2011 iConference

We carried out a retrieval effectiveness test on the three major web search engines (i.e., Google, Microsoft and Yahoo). In addition to relevance judgments, we classified the results according to their commercial intent and whether or not they carried ...

Comments

Information & Contributors

Information

Published In

CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

November 2014

2152 pages

ISBN:9781450325981

DOI:10.1145/2661829

General Chairs:
Jianzhong Li
Harbin Inst. of Technology
,
X. Sean Wang
Fudan University
,
Program Chairs:
Minos Garofalakis
Technical University of Crete, Greece
,
Ian Soboroff
National Institute of Standards, USA
,
Torsten Suel
New York University, USA
,
Min Wang
Google Research, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIKM '14

Sponsor:

CIKM '14: 2014 ACM Conference on Information and Knowledge Management

November 3 - 7, 2014

Shanghai, China

Acceptance Rates

CIKM '14 Paper Acceptance Rate 175 of 838 submissions, 21%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
128
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Achsas SNfaoui E(2022)Academic Aggregated Search Approach Based on BERT Language Model2022 2nd International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET)10.1109/IRASET52964.2022.9737888(1-9)Online publication date: 3-Mar-2022
https://doi.org/10.1109/IRASET52964.2022.9737888
Arguello J(2018)Aggregated SearchFoundations and Trends in Information Retrieval10.1561/150000005210:5(365-502)Online publication date: 14-Dec-2018
https://dl.acm.org/doi/10.1561/1500000052
Kadotami YYoshida YFujita SSakai TKamps JKanoulas Ede Rijke MFang HYilmaz E(2017)Mobile Vertical Ranking based on Preference GraphsProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3121050.3121082(225-228)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1145/3121050.3121082
Chuklin ASchuth AZhou KRijke M(2015)A Comparative Analysis of Interleaving Methods for Aggregated SearchACM Transactions on Information Systems10.1145/266812033:2(1-38)Online publication date: 17-Feb-2015
https://dl.acm.org/doi/10.1145/2668120

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Evaluating intuitiveness of vertical-aware click models

The 2nd workshop on Vertical Search Relevance at WSDM 2015

The influence of commercial intent of search results on their perceived relevance