今天读了ICML12'一篇有趣的论文《Machine Learning that Matters》作者 Kiri Wagstaff。

下面是这篇论文的摘要：

Much of current machine learning (ML) research has lost its connection to problems of import to the larger world of science and society. From this perspective, there exist glaring limitations in the data sets we investigate, the metrics we employ for evaluation, and the degree to which results are communicated back to their originating domains. What changes are needed to how we conduct research to increase the impact that ML has? We present six Impact Challenges to explicitly focus the field’s energy and attention, and we discuss existing obstacles that must be addressed. We aim to inspire ongoing discussion and focus on ML that matters.

    作者在文中指出了目前机器学习研究太过于注重测试数据，如UCI等，而忽略了数据的实际应用领域；同时，目前使用的性能评价指标像AUC、ROC曲线，完全忽略了数据本身的应用背景，提供的是一个数值上的对比，很难保证这个数值在实际应用中有任何意义。例如在植物学领域，80%的准确度也许是一个很不错的结果了，但是99%的准确度显示某类蘑菇是无毒的，也许我们也不敢吃这类蘑菇。性能评价应该结合具体的应用背景才能有其实际的价值。

    作者指出，一个真正实用的机器学习算法或系统应该如下图所示