2022

doi dblp
Crowd_Frame: A Simple and Complete Framework to Deploy Complex Crowdsourcing Tasks Off-the-shelf
Michael Soprano | Kevin Roitero | Francesco Bombassei De Bona | Stefano Mizzaro
WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21 - 25, 2022

pdf dblp
BUM at CheckThat!-2022: A Composite Deep Learning Approach to Fake News Detection using Evidence Retrieval
David La Barbera | Kevin Roitero | Joel Mackenzie | Damiano Spina | Gianluca Demartini | Stefano Mizzaro
Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, Bologna, Italy, September 5th - to - 8th, 2022

doi dblp
Preferences on a Budget: Prioritizing Document Pairs when Crowdsourcing Relevance Judgments
Kevin Roitero | Alessandro Checco | Stefano Mizzaro | Gianluca Demartini
WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022

2021

doi dblp
The many dimensions of truthfulness: Crowdsourcing misinformation assessments on a multidimensional scale
Michael Soprano | Kevin Roitero | David La Barbera | Davide Ceolin | Damiano Spina | Stefano Mizzaro | Gianluca Demartini
2021 Volume 58 Issue 6

doi dblp
On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation
Kevin Roitero | Eddy Maddalena | Stefano Mizzaro | Falk Scholer
2021 Volume 58 Issue 6

2020

doi dblp
Cheap IR evaluation: fewer topics, no relevance judgements, and crowdsourced assessments
Kevin Roitero
2020 Volume 54 Issue 2

doi dblp
Fewer topics? A million topics? Both?! On topics subsets in test collections
Kevin Roitero | J. Shane Culpepper | Mark Sanderson | Falk Scholer | Stefano Mizzaro
2020 Volume 23 Issue 1

doi dblp
Effectiveness evaluation without human relevance judgments: A systematic analysis of existing methods and of their combinations
Kevin Roitero | Andrea Brunello | Giuseppe Serra | Stefano Mizzaro
2020 Volume 57 Issue 2

pdf dblp
Twitter goes to the Doctor: Detecting Medical Tweets using Machine Learning and BERT
Kevin Roitero | Cristian Bozzato | Vincenzo Della Mea | Stefano Mizzaro | Giuseppe Serra
Proceedings of the Workshop on Semantic Indexing and Information Retrieval for Health from heterogeneous content types and languages co-located with 42nd European Conference on Information Retrieval, SIIRH@ECIR 2020, Lisbon, Portugal, April 14, 2020

doi dblp
Crowd Worker Strategies in Relevance Judgment Tasks
Lei Han | Eddy Maddalena | Alessandro Checco | Cristina Sarasua | Ujwal Gadiraju | Kevin Roitero | Gianluca Demartini
WSDM '20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020

doi dblp
Can The Crowd Identify Misinformation Objectively?: The Effects of Judgment Scale and Assessor's Background
Kevin Roitero | Michael Soprano | Shaoyang Fan | Damiano Spina | Stefano Mizzaro | Gianluca Demartini
Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020

doi dblp
The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?
Kevin Roitero | Michael Soprano | Beatrice Portelli | Damiano Spina | Vincenzo Della Mea | Giuseppe Serra | Stefano Mizzaro | Gianluca Demartini
CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020

doi dblp
Leveraging Behavioral Heterogeneity Across Markets for Cross-Market Training of Recommender Systems
Kevin Roitero | Ben Carterette | Rishabh Mehrotra | Mounia Lalmas
Companion of The 2020 Web Conference 2020, Taipei, Taiwan, April 20-24, 2020

doi dblp
Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias
David La Barbera | Kevin Roitero | Gianluca Demartini | Stefano Mizzaro | Damiano Spina
Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II

2019

pdf dblp
HITS Hits Readersourcing: Validating Peer Review Alternatives Using Network Analysis
Michael Soprano | Kevin Roitero | Stefano Mizzaro
Proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2019) co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019), Paris, France, July 25, 2019

pdf dblp
Bias and Fairness in Effectiveness Evaluation by Means of Network Analysis and Mixture Models
Kevin Roitero | Stefano Mizzaro | Michael Soprano
Proceedings of the 10th Italian Information Retrieval Workshop, Padova, Italy, September 16-18, 2019

doi dblp
All Those Wasted Hours: On Task Abandonment in Crowdsourcing
Lei Han | Kevin Roitero | Ujwal Gadiraju | Cristina Sarasua | Alessandro Checco | Eddy Maddalena | Gianluca Demartini
Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019

doi dblp
On Topic Difficulty in IR Evaluation: The Effect of Systems, Corpora, and System Components
Fabio Zampieri | Kevin Roitero | J. Shane Culpepper | Oren Kurland | Stefano Mizzaro
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019

doi dblp
Towards Stochastic Simulations of Relevance Profiles
Kevin Roitero | Andrea Brunello | Julián Urbano | Stefano Mizzaro
Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019

doi dblp
On Transforming Relevance Scales
Lei Han | Kevin Roitero | Eddy Maddalena | Stefano Mizzaro | Gianluca Demartini
Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019

2018

pdf dblp
How Many Truth Levels? Six? One Hundred? Even More? Validating Truthfulness of Statements via Crowdsourcing
Kevin Roitero | Gianluca Demartini | Stefano Mizzaro | Damiano Spina
Proceedings of the CIKM 2018 Workshops co-located with 27th ACM International Conference on Information and Knowledge Management (CIKM 2018), Torino, Italy, October 22, 2018

doi dblp
CHEERS: CHeap & Engineered Evaluation of Retrieval Systems
Kevin Roitero
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
Query Performance Prediction and Effectiveness Evaluation Without Relevance Judgments: Two Sides of the Same Coin
Stefano Mizzaro | Josiane Mothe | Kevin Roitero | Md Zia Ullah
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
Effectiveness Evaluation with a Subset of Topics: A Practical Approach
Kevin Roitero | Michael Soprano | Stefano Mizzaro
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
IRevalOO: An Object Oriented Framework for Retrieval Evaluation
Kevin Roitero | Eddy Maddalena | Yannick Ponte | Stefano Mizzaro
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
On Fine-Grained Relevance Scales
Kevin Roitero | Eddy Maddalena | Gianluca Demartini | Stefano Mizzaro
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

2017

pdf dblp
Economic Evaluation of Recommender Systems: A Proposal
Kevin Roitero | Stefano Mizzaro | Giuseppe Serra
Proceedings of the 8th Italian Information Retrieval Workshop, Lugano, Switzerland, June 05-07, 2017

doi dblp
Considering Assessor Agreement in IR Evaluation
Eddy Maddalena | Kevin Roitero | Gianluca Demartini | Stefano Mizzaro
Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2017, Amsterdam, The Netherlands, October 1-4, 2017

doi dblp
Do Easy Topics Predict Effectiveness Better Than Difficult Topics?
Kevin Roitero | Eddy Maddalena | Stefano Mizzaro
Advances in Information Retrieval - 39th European Conference on IR Research, ECIR 2017, Aberdeen, UK, April 8-13, 2017, Proceedings

2016

pdf dblp
Improving the Efficiency of Retrieval Effectiveness Evaluation: Finding a Few Good Topics with Clustering?
Kevin Roitero | Stefano Mizzaro
Proceedings of the 7th Italian Information Retrieval Workshop, Venezia, Italy, May 30-31, 2016

Search