Treatments for psychiatric disorders are only as effective as the precision with which we administer them. We have treatments that work; we just cannot always accurately predict who they are going to work for and why. In this article, we discuss how big data can help identify robust, reproducible and generalizable predictors of treatment response in psychiatry. Specifically, we focus on how machine-learning approaches can facilitate a move beyond discovery studies and toward model validation. We will highlight some recent exemplary studies in this area, describe how one can assess the merits of studies reporting treatment biomarkers, and discuss what we consider to be best practice for prediction research in psychiatry.