Stalk Width Estimation
The goal of this project was to detect and size stalks in Sorghum plants. Hence, the objective was to develop an algorithm that can work in real time and is easily generalized for sizing different plants.
Fig.1 (a) Detected Stalks
Fig.1 (b) Estimated Width
The algorithm needed to run in real time and had to be generalizable across different crops. Hence a Deep Learninig based data driven approach was used.
A VGG-16 based Faster-RCNN model was trained to detect stalks.
We used around 300 images with almost 1400 total bounding boxes.
The resulting detections are shown in Fig.2
Fig. 2 This figure shows the detected stalks
But, once the stalks are detected, how to size them? For that :
These stalks were then used as proposals for A Fully Convolutional Network, that Semantically Segmented them.
Thus, masks generated from Semantic Segmentation were used to fit ellipses using SVD.
Fig.3 This figure shows the resulting semantically segmented stalk.
All measurements were in pixels. To get a metric estimate, depth-maps were computed using calibrated stereo pair.
The red sensor in the right of Fig.4 is the custom stereo sensor used in our lab.
This sensor uses strong flashes and low exposure time to get rid of background, thus making uniform data for deep-learning pipelines to learn faster.
Fig.4 This figure shows the resulting disparity map after applying SGBM over rectified stereo images.
The length of minor axis gave the width for the proposed stalk, as shown in Fig.5. This width was in pixels. To get a metric estimate, the depth map was used to get the real world coordinates of the minor axis.
Fig.5 This figure shows the length of the minor axis in metric units. This is the resulting stalk-width.
Process Flow Diagram And Results
The resulting architecture, shown in Fig. 6 was faster to train than the architectures semantically segment the whole image, because Faster-RCNN was used to generate Regions of Interests for the FCN.
The pipeline was measure stalk-width with 2.2 mm absolute error, taking human-measurements as reference.
The pipeline was 220 times faster than human going through the same amount of data.
Fig.6 This figure shows the system-pipeline for stalk width estimation. At first the stalks are detected using Faster-RCNN. The detections are sent trough a F.C.N. for semantic segmentation. Then ellipses are fit to the resulting masks. The length of minor axis in metric units then gives us the estimated width.
Once, stalk-widths are computed for the entire data-set, a Geo-registered heat-map was made indicating regions with low and high stalk-width.
This data was collected in Clemson, South-Carolina.
Fig.7. This image shows the heat-map for stalk-widths across the entire field.
As shown by Fig.8, this architecture can be easily used for different phenotypes. This particular examples shows the use of this architecture for detecting and sizing Panicles.
Fig.8. (a) and (b) show the results of using the same architecture for detecting Panicles and (c) and (d) show the resulting size estimates.