A major challenge for crop research in the 21st century is how to predict crop performance as a function of genetic architecture. Advances in “next generation” DNA sequencing have greatly improved genotyping efficiency and reduced genotyping costs. Methods for characterizing plant traits (phenotypes), however, have much progressed more slowly over the past 30 years, and constraints in phenotyping capability limit our ability to dissect the genetics of quantitative traits, especially those related to harvestable yield and stress tolerance. As a case in point, mapping populations for major crops may consist of 20 or more families, each represented by as many as 200 lines, necessitating field trials with over 20,000 plots at a single location. Investing in the resources and labor needed to quantify even a few agronomic traits for linkage with genetic markers in such massive populations is currently impractical for most breeding programs. Herein, we define key criteria, experimental approaches, equipment and data analysis tools required for robust, high-throughput field-based phenotyping (FBP). The focus is on simultaneous proximal sensing for spectral reflectance, canopy temperature, and plant architecture where a vehicle carrying replicated sets of sensors records data on multiple plots, with the potential to record data throughout the crop life cycle. The potential to assess traits, such as adaptations to water deficits or acute heat stress, several times during a single diurnal cycle is especially valuable for quantifying stress recovery. Simulation modeling and related tools can help estimate physiological traits such as canopy conductance and rooting capacity. Many of the underlying techniques and requisite instruments are available and in use for precision crop management. Further innovations are required to better integrate the functions of multiple instruments and to ensure efficient, robust analysis of the large volumes of data that are anticipated. A complement to the core proximal sensing is high-throughput phenotyping of specific traits such as nutrient status, seed composition, and other biochemical characteristics, as well as underground root architecture. The ability to “ground truth” results with conventional measurements is also necessary. The development of new sensors and imaging systems undoubtedly will continue to improve our ability to phenotype very large experiments or breeding nurseries, with the core FBP abilities achievable through strong interdisciplinary efforts that assemble and adapt existing technologies in novel ways.