Our applications of Computer Vision strategies to Agricolture (Olive trees plantations)

 

overviewAbstract — Recent years have seen an increase of computer vision techniques applied to agriculture, especially to fruit detection and yield forecasting. However, in the case of olive orchards little work has been done. Olives are one relevant agricultural production of Southern Europe and North Africa, but olive trees present specific challenges that make the problem of yield estimation with computer vision techniques more challenging with respect to typical horticultural productions.

Despite this difficulty, the economic advantage of yield estimation for this type of cultivations makes the problem worth of consideration, and recent advancements, in particular in the field of segmentation with Deep Neural Networks, show promising results. 

The aim of this work is twofold: first, we want to assess the capabilities of state-of-the-art techniques, based on RGB sensors, with respect to the problem of detecting olive flowers and fruits (called drupes). Secondly, we devise Encoder-Decoder Convolutional Neural Network models to perform image segmentation and improve precision and recall performance to ease the yield estimation task. To compare the approaches and validate the proposed approach we also gather datasets of different olive tree images and videos, collected in different illumination conditions and containing different cultivars (varieties).  


Description

Olives are a relevant agricultural product in North Africa, Southern Europe and Middle East, especially for olive oil production. The automation of existing olive oil production pipelines or the application of precision farming protocols to olive orchards is of great economic interest and poses challenging open questions, mainly related to monitoring and forecasting of production.

In this context, computer vision techniques, eventually using autonomous robotic platforms, could be used to measure the inflorescence (and thus the pollination) in early spring and drupe development during summer, in order to allow for a more precise treatments planning and to forecast production. Current state-of-the-art vision system for fruit detection and yield estimation, and eventually harvesting, have been tested recently with success on cultivation big, round and generally not green fruits. In addition, many computer vision based systems for agricultural applications are tested under controlled light conditions (i.e. artificial illumination and absence of shadows) or rely on expensive Infra-Red sensors. 

However, for olive flowers and fruits, many of the above considerations do not hold. In fact, olive flowers and fruits are small, tightly grouped and have the same color and very similar texture to tree leaves. In addition, olive trees plantations are usually located on difficult terrain, with high slopes, such as cliffs an hillsides, thus making the use of illumination control strategies infeasible. For these reasons, the development of robust and cost-effective computer vision based solutions for olive production is of great interest. In this project, we aim to address this topic, exploring different techniques and algorithms.

Our main contributions summarizes in the following points:

  1. We collect two datasets of olive flowers and fruits RGB images taken under natural illumination conditions.

  2. We train a Conditional Random Field (CRF) and a Support Vector Machine (SVM) on the basis of handcrafted feature, namely HOG, LBP and RGB, to discriminate between fruit/flowers and non-fruits/flowers pixels in each image

  3. We adapt and train a state-of-the-art segmentation Convolutional Neural Network (CNN) for pixel-wise segmentation of RGB images

 Network Architecture

CNN based approaches are variants of the popular SegNet architecture. The original network architecture consists of a sequence of four encoding and decoding layers followed by a pixel-wise classification layer.

We use the original SegNet architecture trained with our olive flowers and drupes dataset (OliveSegNet, OSN), and we set also reduced variants of it:

  1. Small-OliveSegNet (Small-OSN), where the number of channels in each convolutional layer is reduced from the original value of 64 to 16.

  2. Tiny-OliveSegNet (Tiny-OSN), that presents a further reduction of the number of channels in each convolutional layer, to a total of 8 channels.

  3. Ripped-OliveSegNet (Ripped-OSN), which has convolutions of 64 channels and where the last coder and the first decoder layers are removed.

 

olive seg

 

 Results

 We tested our approaches on the collected flowers and fruits datasets. Here we show some preliminary results both quantitative and qualitative. The variants of the stat-of-the-art segmentation CNN we devise outperform classical

state-of-the-art techniques based on handcrafted features, improving performances by up to 25.09% precision and 14.74% recall.

 
Flowers images

The flowers of the olive trees are small, grouped and light-yellow colored. It is usually quite difficult to distinguish them from the foliage and, in general, the background. Our preliminary results on the flower images dataset are encouraging, especially those of the Segmentation CNN architectures we analysed. 

Also from a qualitative point of view it is observed that the output of OSN is less noisy and provides more accurate segmentation with respect to CRF and, more definitely, to SVM.

 
 
 F1 Score 
 Precision [%] 
 Recall [%] 
 Accuracy [%] 
CRF
0.0859
73.97
4.56
93.83
SVM
0.3974
20.89
58.16 
83.33 
OSN
0.3824
35.53
50.86 
91.88 
Small-OSN
0.3815
29.37
69.71 
88.28 
Tiny-OSN
0.4058
31.53
70.98 
88.87 
Ripped-OSN
0.4004
32.63
72.95 
86.89 

 

1 flower
gt 1 flower
pred 1 svm flower
prediction 1 flower CRF
prediction 1 flower cnn
 Test image
 Ground truth
 SVM result
 CRF result
 OSN result

 

 

FRUITS IMAGES

The fruits of the olive trees have texture, color and size very similar to the leaves. For this reason it is very challenging to detect them from a RGB image. However, our preliminary results on the fruits images dataset are encouraging, especially those of the Segmentation CNN architectures we analysed. 

Also from a qualitative point of view it is observed that the output of OSN is less noisy and provides more accurate segmentation with respect to CRF and, more definitely, to SVM.

 

   F1 Score   Precision [%]   Recall [%]   Accuracy [%] 
CRF  0.3070  20.86 58.18  83.31 
SVM 0.2744  17.93  58.45  72.28 
OSN 0.3962 43.02 47.59 90.81 
Small-OSN 0.4012  34.80  59.11  88.99
Tiny-OSN 0.3595  30.22  58.42   86.01
Ripped-OSN 0.3634  32.35  62.92  83.53 



2 fruit gt 2 fruit pred 2 svm fruit prediction 2 fruit CRF prediction 2 fruit cnn
 Test image  Ground truth  SVM result  CRF result  OSN result

 

 

Download

We will release the code used forour experiments, as well as the collected datasets.


Related Publications and Citations

Coming Soon