Rajeev Verma

University of Amsterdam | UVA · Faculty of Science

We study the statistical properties of learning to defer (L2D) to multiple experts. In particular, we address the open problems of deriving a consistent surrogate loss, confidence calibration, and principled ensembling of experts. Firstly, we derive two consistent surrogates -- one based on a softmax parameterization, the other on a one-vs-all (OvA...

The learning to defer (L2D) framework has the potential to make AI systems safer. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar & Sontag's (...

Peer-review process is fraught with issues like bias, inconsistencies, arbitrariness, non-committal weak rejects, etc. However, it is anticipated that the peer reviews provide constructive feedback to the authors against some aspects of the paper such as Motivation/Impact, Soundness/Correctness, Novelty, Substance, etc. A good review is expected to...

In the originally published version of chapter 88 the acknowledgement statement was erroneously omitted. The acknowledgement statement has been added to the chapter.

Deciding the appropriateness of a manuscript to the aims and scope of a journal is very important in the first stage of peer review. Editors should be confident about the article’s suitability to the intended journal to further channel its progress through the steps in the review process. However, not all sections in a research article are equally...

Editorial pre-screening is the first step in academic peer review. The deluge of research papers and the huge amount of submissions being made to journals these days makes editorial decision a very challenging task. The current work attempts to investigate certain impact factors that may have a role in the editorial decision making process. The pro...