1 code implementation • 25 Nov 2022 • Takuma Udagawa, Haruka Kiyohara, Yusuke Narita, Yuta Saito, Kei Tateno
Although many estimators have been developed, there is no single estimator that dominates the others, because the estimators' accuracy can vary greatly depending on a given OPE task such as the evaluation policy, number of actions, and noise level.
no code implementations • 17 Sep 2021 • Yuta Saito, Takuma Udagawa, Kei Tateno
As proof of concept, we use our procedure to select the best estimator to evaluate coupon treatment policies on a real-world online content delivery service.
2 code implementations • 31 Aug 2021 • Yuta Saito, Takuma Udagawa, Haruka Kiyohara, Kazuki Mogi, Yusuke Narita, Kei Tateno
Unfortunately, identifying a reliable estimator from results reported in research papers is often difficult because the current experimental procedure evaluates and compares the estimators' performance on a narrow set of hyperparameters and evaluation policies.