2022

pdf dblp
BUM at CheckThat!-2022: A Composite Deep Learning Approach to Fake News Detection using Evidence Retrieval
David La Barbera | Kevin Roitero | Joel Mackenzie | Damiano Spina | Gianluca Demartini | Stefano Mizzaro
Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, Bologna, Italy, September 5th - to - 8th, 2022

doi dblp
Preferences on a Budget: Prioritizing Document Pairs when Crowdsourcing Relevance Judgments
Kevin Roitero | Alessandro Checco | Stefano Mizzaro | Gianluca Demartini
WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022

doi dblp
Crowd_Frame: A Simple and Complete Framework to Deploy Complex Crowdsourcing Tasks Off-the-shelf
Michael Soprano | Kevin Roitero | Francesco Bombassei De Bona | Stefano Mizzaro
WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21 - 25, 2022

2021

doi dblp
The many dimensions of truthfulness: Crowdsourcing misinformation assessments on a multidimensional scale
Michael Soprano | Kevin Roitero | David La Barbera | Davide Ceolin | Damiano Spina | Stefano Mizzaro | Gianluca Demartini
2021 Volume 58 Issue 6

doi dblp
On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation
Kevin Roitero | Eddy Maddalena | Stefano Mizzaro | Falk Scholer
2021 Volume 58 Issue 6

2020

doi dblp
Effectiveness evaluation without human relevance judgments: A systematic analysis of existing methods and of their combinations
Kevin Roitero | Andrea Brunello | Giuseppe Serra | Stefano Mizzaro
2020 Volume 57 Issue 2

doi dblp
Cheap IR evaluation: fewer topics, no relevance judgements, and crowdsourced assessments
Kevin Roitero
2020 Volume 54 Issue 2

doi dblp
Fewer topics? A million topics? Both?! On topics subsets in test collections
Kevin Roitero | J. Shane Culpepper | Mark Sanderson | Falk Scholer | Stefano Mizzaro
2020 Volume 23 Issue 1

pdf dblp
Twitter goes to the Doctor: Detecting Medical Tweets using Machine Learning and BERT
Kevin Roitero | Cristian Bozzato | Vincenzo Della Mea | Stefano Mizzaro | Giuseppe Serra
Proceedings of the Workshop on Semantic Indexing and Information Retrieval for Health from heterogeneous content types and languages co-located with 42nd European Conference on Information Retrieval, SIIRH@ECIR 2020, Lisbon, Portugal, April 14, 2020

doi dblp
Leveraging Behavioral Heterogeneity Across Markets for Cross-Market Training of Recommender Systems
Kevin Roitero | Ben Carterette | Rishabh Mehrotra | Mounia Lalmas
Companion of The 2020 Web Conference 2020, Taipei, Taiwan, April 20-24, 2020

doi dblp
Crowd Worker Strategies in Relevance Judgment Tasks
Lei Han | Eddy Maddalena | Alessandro Checco | Cristina Sarasua | Ujwal Gadiraju | Kevin Roitero | Gianluca Demartini
WSDM '20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020

doi dblp
Can The Crowd Identify Misinformation Objectively?: The Effects of Judgment Scale and Assessor's Background
Kevin Roitero | Michael Soprano | Shaoyang Fan | Damiano Spina | Stefano Mizzaro | Gianluca Demartini
Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020

doi dblp
The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?
Kevin Roitero | Michael Soprano | Beatrice Portelli | Damiano Spina | Vincenzo Della Mea | Giuseppe Serra | Stefano Mizzaro | Gianluca Demartini
CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020

doi dblp
Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias
David La Barbera | Kevin Roitero | Gianluca Demartini | Stefano Mizzaro | Damiano Spina
Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II

2019

pdf dblp
Bias and Fairness in Effectiveness Evaluation by Means of Network Analysis and Mixture Models
Kevin Roitero | Stefano Mizzaro | Michael Soprano
Proceedings of the 10th Italian Information Retrieval Workshop, Padova, Italy, September 16-18, 2019

pdf dblp
HITS Hits Readersourcing: Validating Peer Review Alternatives Using Network Analysis
Michael Soprano | Kevin Roitero | Stefano Mizzaro
Proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2019) co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019), Paris, France, July 25, 2019

doi dblp
All Those Wasted Hours: On Task Abandonment in Crowdsourcing
Lei Han | Kevin Roitero | Ujwal Gadiraju | Cristina Sarasua | Alessandro Checco | Eddy Maddalena | Gianluca Demartini
Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019

doi dblp
On Topic Difficulty in IR Evaluation: The Effect of Systems, Corpora, and System Components
Fabio Zampieri | Kevin Roitero | J. Shane Culpepper | Oren Kurland | Stefano Mizzaro
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019

doi dblp
Towards Stochastic Simulations of Relevance Profiles
Kevin Roitero | Andrea Brunello | Julián Urbano | Stefano Mizzaro
Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019

doi dblp
On Transforming Relevance Scales
Lei Han | Kevin Roitero | Eddy Maddalena | Stefano Mizzaro | Gianluca Demartini
Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019

2018

pdf dblp
How Many Truth Levels? Six? One Hundred? Even More? Validating Truthfulness of Statements via Crowdsourcing
Kevin Roitero | Gianluca Demartini | Stefano Mizzaro | Damiano Spina
Proceedings of the CIKM 2018 Workshops co-located with 27th ACM International Conference on Information and Knowledge Management (CIKM 2018), Torino, Italy, October 22, 2018

doi dblp
CHEERS: CHeap & Engineered Evaluation of Retrieval Systems
Kevin Roitero
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
Query Performance Prediction and Effectiveness Evaluation Without Relevance Judgments: Two Sides of the Same Coin
Stefano Mizzaro | Josiane Mothe | Kevin Roitero | Md Zia Ullah
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
Effectiveness Evaluation with a Subset of Topics: A Practical Approach
Kevin Roitero | Michael Soprano | Stefano Mizzaro
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
IRevalOO: An Object Oriented Framework for Retrieval Evaluation
Kevin Roitero | Eddy Maddalena | Yannick Ponte | Stefano Mizzaro
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

doi dblp
On Fine-Grained Relevance Scales
Kevin Roitero | Eddy Maddalena | Gianluca Demartini | Stefano Mizzaro
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018

2017

pdf dblp
Economic Evaluation of Recommender Systems: A Proposal
Kevin Roitero | Stefano Mizzaro | Giuseppe Serra
Proceedings of the 8th Italian Information Retrieval Workshop, Lugano, Switzerland, June 05-07, 2017

doi dblp
Considering Assessor Agreement in IR Evaluation
Eddy Maddalena | Kevin Roitero | Gianluca Demartini | Stefano Mizzaro
Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2017, Amsterdam, The Netherlands, October 1-4, 2017

doi dblp
Do Easy Topics Predict Effectiveness Better Than Difficult Topics?
Kevin Roitero | Eddy Maddalena | Stefano Mizzaro
Advances in Information Retrieval - 39th European Conference on IR Research, ECIR 2017, Aberdeen, UK, April 8-13, 2017, Proceedings

2016

pdf dblp
Improving the Efficiency of Retrieval Effectiveness Evaluation: Finding a Few Good Topics with Clustering?
Kevin Roitero | Stefano Mizzaro
Proceedings of the 7th Italian Information Retrieval Workshop, Venezia, Italy, May 30-31, 2016