Conference PaperPDF Available

Automatic grape bunch detection in vineyards based on affordable 3D phenotyping using a consumer webcam

Abstract and Figures

This work presents a methodology for 3-D phenotyping of vineyards based on images captured by a low cost high-definition webcamera. A novel software application integrated visual odometry and multiple-view stereo components to create dense and accurate three-dimensional points clouds for vines, properly transformed to millimeter scale. Geometrical and color features of the points were employed by a classification procedure that reached 93% of accuracy on detecting points belonging to grapes. Individual bunches were automatically delimited and their volumes estimated. The sum of the estimated volumes per vine presented a coefficient of correlation of R = 0.99 to the real grape weight observed in each vine after harvesting.
Content may be subject to copyright.

... Santos et al., 2020), animais ou sintomas de doenças e pragas (Ferentinos, 2018;Barbedo, 2019). A partir de imagens capturadas por equipes de campo ou obtidas por câmeras acopladas em tratores, implementos, robôs ou drones, um monitoramento constante e eficiente pode ser realizado: a busca por anomalias na cultura ou na criação; a avaliação de variabilidade espacial da cultura para intervenção, segundo os preceitos da agricultura de precisão; e a atuação autônoma por máquinas e implementos. ...
... Como discutido anteriormente, diversos fatores dificultam o processo de detecção, de oclusão por folhas e galhos a problemas de foco da câmera e iluminação (Figura 3). Em algumas culturas, os frutos apresentam ainda uma diversidade de formatos, compacticidade e orientação, como no caso da viticultura (Santos et al., 2020). Apesar de algum sucesso obtido com outras técnicas de aprendizado de máquina (Gongal et al., 2015), a detecção de frutos realmente ganhou força recentemente, com os avanços em redes neurais convolutivas (Sa et al., 2016;Bargoti;Underwood, 2017;Kamilaris;Prenafeta-Boldú, 2018 (Redmon et al., 2016;Farhadi, 2018). ...
... A Figura 6 exibe um exemplo da detecção de frutos em uma imagem de laranjeira obtida em campo. Santos et al. (2020) mostraram que, para uvas em viticultura, culturas que apresentam grande variabilidade em forma, cor, tamanho e compacidade, os cachos podem ser detectados e segmentados com o uso de arquiteturas como Mask-RNN e YOLO. Os autores produziram uma nova ferramenta de anotação capaz agilizar o processo de associação de pixels a frutos, discriminando exatamente quais pixels pertencem a quais cachos. ...
Chapter
Full-text available
Em uma definição simples e abrangente, visão computacional é o campo da inteligência artificial dedicado à extração de informações a partir de imagens digitais. No contexto da agricultura digital, a visão computacional pode ser empregada na detecção de doenças e pragas, na estimação de safra e na avaliação não invasiva de atributos como qualidade, aparência e volume, além de ser componente essencial em sistemas robóticos agrícolas. Segundo Duckett et al. (2018), a robótica de campo deve viabilizar uma nova gama de equipamentos agrícolas: máquinas pequenas e inteligentes capazes de reduzir desperdício e impacto ambiental1 e proporcionar viabilidade econômica, aumentando assim a sustentabilidade dos alimentos. Ainda segundo Duckett et al. (2018), há um potencial considerável no aumento da janela de oportunidades para intervenções, por exemplo, em operação em solos úmidos, operação noturna e monitoramento constante da lavoura. Uma classe de problemas abordados pela visão computacional são os problemas ditos perceptuais: a detecção e a classificação de padrões, nas imagens, que são associados a um objeto de interesse, como frutos (Sa et al., 2016; 1 Devido ao uso comedido e inteligente de defensivos ou simplesmente à intervenção mecânica: a remoção física de pragas por manipuladores.
... High-resolution 3D scans of grapes and vines have also been achieved using multiple RGB images captured from different positions using structure from motion photogrammetry techniques [11][12][13]. This method can be used with inexpensive equipment [14] and data collection can be automated by mounting cameras on platforms such as robots or drones [15]. However, generating photogrammetry scans requires significant computation load and time. ...
... Improved results could be obtained by merging multiple RGB-D camera scans taken at a range of positions and angles relative to the grapes. This could be achieved using SLAM or a similar point cloud alignment technique [14]. This should then make the RGB-D camera scans more comparable to the photogrammetry scans. ...
Article
Full-text available
This work investigates the performance of five depth cameras in relation to their potential for grape yield estimation. The technologies used by these cameras include structured light (Kinect V1), active infrared stereoscopy (RealSense D415), time of flight (Kinect V2 and Kinect Azure), and LiDAR (Intel L515). To evaluate their suitability for grape yield estimation, a range of factors were investigated including their performance in and out of direct sunlight, their ability to accurately measure the shape of the grapes, and their potential to facilitate counting and sizing of individual berries. The depth cameras’ performance was benchmarked using high-resolution photogrammetry scans. All the cameras except the Kinect V1 were able to operate in direct sunlight. Indoors, the RealSense D415 camera provided the most accurate depth scans of grape bunches, with a 2 mm average depth error relative to photogrammetric scans. However, its performance was reduced in direct sunlight. The time of flight and LiDAR cameras provided depth scans of grapes that had about an 8 mm depth bias. Furthermore, the individual berries manifested in the scans as pointed shape distortions. This led to an underestimation of berry sizes when applying the RANSAC sphere fitting but may help with the detection of individual berries with more advanced algorithms. Applying an opaque coating to the surface of the grapes reduced the observed distance bias and shape distortion. This indicated that these are likely caused by the cameras’ transmitted light experiencing diffused scattering within the grapes. More work is needed to investigate if this distortion can be used for enhanced measurement of grape properties such as ripeness and berry size.
... Oriundas do campo da visão computacional (Hartley & Zisserman, 2003;Triggs et al., 2000) e da robótica (Scaramuzza & Fraundorfer, 2011) , essas técnicas permitem que modelos tridimensionais sejam construídos a partir de apenas um conjunto de imagens sobrepostas dos objetos de interesse, virtualmente transformando um câmera comum em um scanner 3-D. Inicialmente utilizadas na reconstrução 3-D de plantas de pequeno porte em vasos (Kumar et al., 2014;Lou et al., 2014;Santos & de Oliveira, 2012;Santos & Rodrigues, 2016) e no campo (Jay et al., 2015), essas técnicas se mostraram capazes também de capturar a estrutura tridimensional de plantas de maior porte no campo, como por exemplo videiras (Santos et al., 2017). Neste trabalho, apresentamos uma metodologia para a produção de modelos tridimensionais de cafeeiros com o uso de técnicas de visão computacional. ...
... As imagens foram obtidas em campo com uma câmera C920 (Logitech Inc., Lausanne, Suíça), comumente encontrada no mercado brasileiro, e conectada a um computador notebook. Na captura das imagens, foi utilizado o programa 3dmcap 1 , software livre (GPLv3) desenvolvido pela Embrapa Informática Agropecuária e introduzido porSantos et al. (2017). O software realiza uma seleção automática de imagens a partir da sequência de vídeo produzida pela câmera, de modo que seja possível encontrar um número suficiente de correspondências entre as imagens necessárias ao pipeline de structure-from-motion. ...
Conference Paper
Full-text available
RESUMO: O registro da arquitetura de uma planta, isto é, a descrição da topologia e da geometria de sua estrutura, é empregado em estudos funcionais-estruturais variados como intercepção solar, interações bióticas e abióticas e dispersão de defensivos, entre outros. Porém, a realização desse registro é extremamente laboriosa, especialmente em plantas de maior porte e complexidade como o cafeeiro, o que restringe sua adoção e, consequentemente, o volume de dados disponíveis a tais estudos. Neste trabalho, mostramos que técnicas de visão computacional podem ser empregadas na construção de modelos tridimensionais de cafeeiros com o uso de uma única câmera comum e software adequado. Esses resultados indicam a possibilidade de automação do registro in silico da estrutura tridimensional de cafeeiros, permitindo um maior volume na obtenção de dados da arquitetura das plantas. Tal metodologia de aquisição de dados tridimensionais está ao alcance de qualquer grupo de pesquisa, devido ao baixo custo do equipamento necessário e implementações em software livre. Exemplos de reconstruções tridimensionais para oito cafeeiros adultos estão disponíveis em uma base de dados pública. PALAVRAS-CHAVE: reconstrução 3-D, arquitetura de planta, FSPM, visão computacional ABSTRACT: The representation of a plant architecture, that is, the description of its structure's topology and geometry, is employed in various functional-structural studies such as solar interception, biotic and abiotic interactions, and pesticide dispersion, among others. However, this registration is extremely laborious, especially for larger and more complex plants such as coffee, which restricts its adoption and, consequently, the volume of data available to such studies. In this work, we show that computer vision techniques can be employed in the construction of three-dimensional coffee tree models using a single common camera and suitable software. These results indicate the possibility of automating the in silico representation of three-dimensional coffee trees structure, allowing a larger volume on obtaining plant architecture data. Such a three-dimensional data acquisition methodology is within the reach of any research group, due to the low cost of required equipment and open source implementations. Examples of three-dimensional reconstructions for eight adult coffee trees are available in a public database. KEY WORDS: 3-D reconstruction, plant architecture, FSPM, computer vision INTRODUÇÃO Modelos funcionais-estruturais de plantas (functional-structural plant models-FSPM) unem representações da estrutura tridimensional da planta a modelos de suas funções fisiológicas. Esses modelos se mostram úteis a diversos estudos em ciência de plantas: interceptação solar, troca de gases, difusão de pesticidas, poda e interações entre plantas e agentes biológicos (Vos et al., 2010). Modelos tridimensionais de plantas podem ser pensados como uma representação in silico da arquitetura da planta, permitindo aos pesquisadores avaliar detalhadamente seu estado passado e, ao longo do tempo, estudar a dinâmica de seu desenvolvimento e resposta ao ambiente. Um exemplo é o trabalho recente desenvolvido por Rakocevic et al. (2018), que empregou modelos funcionais-estruturais para avaliar alterações em plantas de café submetidas a taxas elevadas de CO 2. A representação da estrutura tridimensional deve registrar tanto a topologia da planta (como as várias estruturas da planta estão conectadas entre si) quanto sua geometria (forma, dimensões e orientação no espaço) (Vos et al., 2010). A construção dessas representações necessita de um registro acurado de dados topológicos e geométricos, comumente codificados na forma de um grafo em árvore multiescalar (multiscale tree graph-MTG), como proposto por Godin & Caraglio (1998). Uma vez que a arquitetura é registrada na forma de MTGs, ela pode ser analisada com o auxílio de software específico como o AMAPmod (Godin et al., 1999) e visualizada na forma de mock-ups tridimensionais (3-D), como apresentado por Matsunaga et al. (2016) para o caso do cafeieros. Para a obtenção desses dados, rastreadores tridimensionais como o FastTrack (Polhemus Inc., Cochester, VT, USA) são comumente empregados no registro da posição tridimensional de pontos de interesse na arquitetura vegetal (Costes et al., 2008; Sinoquet & Rivet, 1997), posteriormente transcritos manualmente pelo pesquisador na forma de MTGs. Infelizmente, esse processo de aquisição de dados é lento, trabalhoso, custoso (devido ao valor elevado dos rastreadores) e deve ser realizado em campo. Essas
... Neste cenário, com um número crescente de dados (big data) tornou-se possível implementar e aprimorar técnicas de Deep Learning, tais como as redes neurais convolutivas: ResNet [6], Rede Eficiente [7], entre outros modelos que tem mostrado resultados expressivos, principalmente para a seleção de características, uma etapa de crucial importância para o bom desempenho da etapa de classificação. Esses modelos vêm apresentando bons resultados em diversas áreas [8], bem como na tomografia computadorizada para a triagem e diagnóstico da COVID-19 [9]. ...
Conference Paper
Full-text available
A inteligência artificial, mais especificadamente, os algoritmos de aprendizagem profundo e técnicas da teoria de conjuntos Fuzzy tem sido propostos em análises de imagens. Deste modo, as aplicações de aprendizado profundo em imagens médicas pulmonares, tem obtido avanços para detecção de doenças, levando a ensaios clínicos promissores no que diz respeito à triagem e diagnóstico de pacientes com o Covid-19. Assim, imagens de Tomografia Computadorizada (TC), Radiografia Computadoriza (RC) e Ultrassonografia tem sido usadas por especialistas como ferramentas para diagnóstico da Covid-19. No entanto, devido ao número elevado de pacientes e baixa quantidade de profissionais especializados torna-se necessário um sistema de detecção por imagens para auxílio aos profissionais da saúde. Neste cenário, a inteligência computacional, mas especialmente modelos de Aprendizado Profundo do tipo Convolutional Neural Network (CNN) e métodos de classificação não-supervisionados, entre estes em destaque o método K-Means e Fuzzy C-Means, foram utilizados para a classificação de imagens de Covid-19. Para esta finalidade, foi utilizada uma base de dados disponibilizada pela Universidade de São Paulo, contendo 4173 imagens. Os resultados demonstraram a importância da seleção de características através de redes convolucionais. Na aprendizagem não-supervisionada observou-se poucas diferenças para a clusterização utilizando os métodos k-Means e Fuzzy C-Means.
... It solves the camera pose and scene geometry estimation simultaneously, employing only image matching and bundle adjustment (Triggs et al., 2000), and finding three-dimensional structure by the motion of a single camera around the scene (or from a set of independent cameras). In a previous work (Santos et al., 2017), we showed that SfM can recover vine structure from image sequences on vineyards. It is an interesting alternative to integrate image data from different camera poses registering the same structures in space. ...
Article
Agricultural applications such as yield prediction, precision agriculture and automated harvesting need systems able to infer the crop state from low-cost sensing devices. Proximal sensing using affordable cameras combined with computer vision has seen a promising alternative, strengthened after the advent of convolutional neural networks (CNNs) as an alternative for challenging pattern recognition problems in natural images. Considering fruit growing monitoring and automation, a fundamental problem is the detection, segmentation and counting of individual fruits in orchards. Here we show that for wine grapes, a crop presenting large variability in shape, color, size and compactness, grape clusters can be successfully detected, segmented and tracked using state-of-the-art CNNs. In a test set containing 408 grape clusters from images taken on a trellis-system based vineyard, we have reached an -score up to 0.91 for instance segmentation, a fine separation of each cluster from other structures in the image that allows a more accurate assessment of fruit size and shape. We have also shown as clusters can be identified and tracked along video sequences recording orchard rows. We also present a public dataset containing grape clusters properly annotated in 300 images and a novel annotation methodology for segmentation of complex objects in natural images. The presented pipeline for annotation, training, evaluation and tracking of agricultural patterns in images can be replicated for different crops and production systems. It can be employed in the development of sensing components for several agricultural and environmental applications.
... It solves the camera pose and scene geometry estimation simultaneously, employing only image matching and bundle adjustment (Triggs et al., 2000), and finding three-dimensional structure by the motion of a single camera around the scene (or from a set of independent cameras). In a previous work (Santos et al., 2017), we showed that SfM can recover vine structure from image sequences on vineyards. It is an interesting alternative to integrate image data from different camera poses registering the same structures in space. ...
Preprint
Full-text available
Agricultural applications as yield prediction, precision agriculture and automated harvesting need systems able to infer the culture state from low cost sensing devices. Proximal sensing using affordable cameras combined to computer vision have seen a promising alternative, strengthened after the advent of convolutional neural networks (CNNs) as an alternative for challenging pattern recognition problems in natural images. Considering fruit growing monitoring and automation, a fundamental problem is the detection, segmentation and counting of individual fruits in orchards. Here we show that for grape wines, a crop presenting large variability in shape, color, size and compactness, grape clusters can be successfully detected, segmented and tracked using state-of-the-art CNNs. In a dataset containing 408 grape clusters from images taken on field, we have reached a F 1-score up to 0.91 for instance segmentation, a fine separation of each cluster from other structures in the image that allows a more accurate assessment of fruit size and shape. We have also shown as clusters can be identified and tracked along video sequences recording orchards rows. We also present a public dataset containing grape clusters properly annotated in 300 images and a novel annotation methodology for segmentation of complex objects in natural images. The presented pipeline for annotation, training, evaluation and tracking of agricultural patterns in images can be replicated for different crops and production systems. It can be employed on the development of sensing components for several agricultural and environmental applications.
... It solves the camera pose and scene geometry estimation simultaneously, employing only image matching and bundle adjustment [52], and finding three-dimensional structure by the motion of a single camera around the scene (or from a set of independent cameras). In a previous work [46], we showed that SfM is able to recovery vine structure from image sequences on vineyards. It is an interesting alternative to integrate image data from different camera poses registering the same structures in space. ...
Preprint
Full-text available
Agricultural applications as yield prediction, precision agriculture and automated harvesting need systems able to infer the culture state from low cost sensing devices. Proximal sensing using affordable cameras combined to computer vision have seen a promising alternative, strengthened after the advent of convolutional neural networks (CNNs) as an alternative for challenging pattern recognition problems in natural images. Considering fruit growing monitoring and automation, a fundamental problem is the detection, segmentation and counting of individual fruits in orchards. Here we show that for grape wines, a crop presenting large variability in shape, color, size and compactness, grape clusters can be successfully detected, segmented and tracked using state-of-the-art CNNs. In a dataset containing 408 grape clusters from images taken on field, we have reached a F1-score up to 0.91 for instance segmentation, a fine separation of each cluster from other structures in the image that allows a more accurate assessment of fruit size and shape. We have also shown as clusters can be identified and tracked along video sequences recording orchards rows. We also present a public dataset containing grape clusters properly annotated in 300 images and a novel annotation methodology for segmentation of complex objects in natural images. The presented pipeline for annotation, training, evaluation and tracking of agricultural patterns in images can be replicated for different crops and production systems. It can be employed on the development of sensing components for several agricultural and environmental applications.
Article
Full-text available
Brazilian fruit culture has a great influence on the social and economic sector in the most diverse regions of the country, generating employment and income in the exercise of its activities. As it is an activity carried out most often in a manual and conventional manner, fruit culture has a great potential for technological growth, especially when adopting the concepts applied by precision agriculture on the crops of grains, fibers and energy, creating a new segment, Precision Fruit Farming. The present work aims to carry out a bibliographic review on the main trends that have emerged in the last five years on Precision Fruit growing, highlighting its future perspectives and the history of technological evolution. 83 articles were analyzed, classified in different perennial cultures and applications, such as machine learning, remote sensing, robotics, using UAV to obtain different vegetation indexes, among others.
Article
Full-text available
In viticulture, phenotypic data are traditionally collected directly in the field via visual and manual means by an experienced person. This approach is time consuming, subjective and prone to human errors. In recent years, research therefore has focused strongly on developing automated and non-invasive sensor-based methods to increase data acquisition speed, enhance measurement accuracy and objectivity and to reduce labor costs. While many 2D methods based on image processing have been proposed for field phenotyping, only a few 3D solutions are found in the literature. A track-driven vehicle consisting of a camera system, a real-time-kinematic GPS system for positioning, as well as hardware for vehicle control, image storage and acquisition is used to visually capture a whole vine row canopy with georeferenced RGB images. In the first post-processing step, these images were used within a multi-view-stereo software to reconstruct a textured 3D point cloud of the whole grapevine row. A classification algorithm is then used in the second step to automatically classify the raw point cloud data into the semantic plant components, grape bunches and canopy. In the third step, phenotypic data for the semantic objects is gathered using the classification results obtaining the quantity of grape bunches, berries and the berry diameter.
Article
Full-text available
The three-dimensional reconstruction of plants using computer vision methods is a promising alternative to non-destructive metrology in plant phenotyping. However, diversity in plants form and size, different surrounding environments (laboratory, greenhouse or field), and occlusions impose challenging issues. We propose the use of state-of-the-art methods for visual odometry to accurately recover camera pose and preliminary three-dimensional models on image acquisition time. Specimens of maize and sunflower were imaged using a single free-moving camera and a software tool with visual odometry capabilities. Multiple-view stereo was employed to produce dense point clouds sampling the plant surfaces. The produced three-dimensional models are accurate snapshots of the shoot state and plant measurements can be recovered in a non-invasive way. The results show a free-moving low-resolution camera is able to handle occlusions and variations in plant size and form, allowing the reconstruction of different species, and specimens in different stages of development. It is also a cheap and flexible method, suitable for different phenotyping needs. Plant traits were computed from the point clouds and compared to manually measured reference, showing millimeter accuracy. All data, including images, camera calibration, pose, and three-dimensional models are publicly available.
Article
Full-text available
Background: Plant organ segmentation from 3D point clouds is a relevant task for plant phenotyping and plant growth observation. Automated solutions are required to increase the efficiency of recent high-throughput plant phenotyping pipelines. However, plant geometrical properties vary with time, among observation scales and different plant types. The main objective of the present research is to develop a fully automated, fast and reliable data driven approach for plant organ segmentation. Results: The automated segmentation of plant organs using unsupervised, clustering methods is crucial in cases where the goal is to get fast insights into the data or no labeled data is available or costly to achieve. For this we propose and compare data driven approaches that are easy-to-realize and make the use of standard algorithms possible. Since normalized histograms, acquired from 3D point clouds, can be seen as samples from a probability simplex, we propose to map the data from the simplex space into Euclidean space using Aitchisons log ratio transformation, or into the positive quadrant of the unit sphere using square root transformation. This, in turn, paves the way to a wide range of commonly used analysis techniques that are based on measuring the similarities between data points using Euclidean distance. We investigate the performance of the resulting approaches in the practical context of grouping 3D point clouds and demonstrate empirically that they lead to clustering results with high accuracy for monocotyledonous and dicotyledonous plant species with diverse shoot architecture. Conclusion: An automated segmentation of 3D point clouds is demonstrated in the present work. Within seconds first insights into plant data can be deviated - even from non-labelled data. This approach is applicable to different plant species with high accuracy. The analysis cascade can be implemented in future high-throughput phenotyping scenarios and will support the evaluation of the performance of different plant genotypes exposed to stress or in different environmental scenarios.
Article
Full-text available
Background: The demand for high-throughput and objective phenotyping in plant research has been increasing during the last years due to large experimental sites. Sensor-based, non-invasive and automated processes are needed to overcome the phenotypic bottleneck, which limits data volumes on account of manual evaluations. A major challenge for sensor-based phenotyping in vineyards is the distinction between the grapevine in the foreground and the field in the background - this is especially the case for red-green-blue (RGB) images, where similar color distributions occur both in the foreground plant and in the field and background plants. However, RGB cameras are a suitable tool in the field because they provide high-resolution data at fast acquisition rates with robustness to outdoor illumination. Results: This study presents a method to segment the phenotypic classes 'leaf', 'stem', 'grape' and 'background' in RGB images that were taken with a standard consumer camera in vineyards. Background subtraction is achieved by taking two images of each plant for depth reconstruction. The color information is furthermore used to distinguish the leaves from stem and grapes in the foreground. The presented approach allows for objective computation of phenotypic traits like 3D leaf surface areas and fruit-to-leaf ratios. The method has been successfully applied to objective assessment of growth habits of new breeding lines. To this end, leaf areas of two breeding lines were monitored and compared with traditional cultivars. A statistical analysis of the method shows a significant (p <0.001) determination coefficient R (2)= 0.93 and root-mean-square error of 3.0%. Conclusions: The presented approach allows for non-invasive, fast and objective assessment of plant growth. The main contributions of this study are 1) the robust segmentation of RGB images taken from a standard consumer camera directly in the field, 2) in particular, the robust background subtraction via reconstruction of dense depth maps, and 3) phenotypic applications to monitoring of plant growth and computation of fruit-to-leaf ratios in 3D. This advance provides a promising tool for high-throughput, automated image acquisition, e.g., for field robots.
Article
Full-text available
This paper presents ORB-SLAM, a feature-based monocular SLAM system that operates in real time, in small and large, indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.
Conference Paper
Full-text available
We propose a semi-direct monocular visual odom-etry algorithm that is precise, robust, and faster than current state-of-the-art methods. The semi-direct approach eliminates the need of costly feature extraction and robust matching techniques for motion estimation. Our algorithm operates directly on pixel intensities, which results in subpixel precision at high frame-rates. A probabilistic mapping method that explicitly models outlier measurements is used to estimate 3D points, which results in fewer outliers and more reliable points. Precise and high frame-rate motion estimation brings increased robustness in scenes of little, repetitive, and high-frequency texture. The algorithm is applied to micro-aerial-vehicle state-estimation in GPS-denied environments and runs at 55 frames per second on the onboard embedded computer and at more than 300 frames per second on a consumer laptop. We call our approach SVO (Semi-direct Visual Odometry) and release our implementation as open-source software.
Article
Full-text available
Laserscanning recently has become a powerful and common method for plant parameterization andplant growth observation on nearly every scale range. However, 3D measurements with highaccuracy, spatial resolution and speed result in a multitude of points that require processing andanalysis. The primary objective of this research has been to establish a reliable and fast technique forhigh throughput phenotyping using differentiation, segmentation and classification of single plantsby a fully automated system. In this report, we introduce a technique for automated classification ofpoint clouds of plants and present the applicability for plant parameterization. A surface feature histogram based approach from the field of robotics was adapted to close-uplaserscans of plants. Local geometric point features describe class characteristics, which were usedto distinguish among different plant organs. This approach has been proven and tested on severalplant species. Grapevine stems and leaves were classified with an accuracy of up to 98%. Theproposed method was successfully transferred to 3D-laserscans of wheat plants for yield estimation.Wheat ears were separated with an accuracy of 96% from other plant organs. Subsequently, the earvolume was calculated and correlated to the ear weight, the kernel weights and the number ofkernels. Furthermore the impact of the data resolution was evaluated considering point to pointdistances between 0:3 and 4:0 mm with respect to the classification accuracy. We introduced an approach using surface feature histograms for automated plant organparameterization. Highly reliable classification results of about 96% for the separation of grapevineand wheat organs have been obtained. This approach was found to be independent of the point topoint distance and applicable to multiple plant species. Its reliability, flexibility and its high order ofautomation make this method well suited for the demands of high throughput phenotyping.
Article
With increasing demand to support and accelerate progress in breeding for novel traits, the plant research community faces the need to accurately measure increasingly large numbers of plants and plant parameters. The goal is to provide quantitative analyses of plant structure and function relevant for traits that help plants better adapt to lowinput agriculture and resource-limited environments. We provide an overview of the inherently multidisciplinary research in plant phenotyping, focusing on traits that will assist in selecting genotypes with increased resource use efficiency. We highlight opportunities and challenges for integrating noninvasive or minimally invasive technologies into screening protocols to characterize plant responses to environmental challenges for both controlled and field experimentation. Although technology evolves rapidly, parallel efforts are still required because large-scale phenotyping demands accurate reporting of at least a minimum set of information concerning experimental protocols, data management schemas, and integration with modeling. The journey toward systematic plant phenotyping has only just begun. Expected final online publication date for the Annual Review of Plant Biology Volume 64 is April 29, 2013. Please see http://www.annualreviews.org/catalog/pubdates.aspx for revised estimates.