See as a Bee: UV Sensor for Aerial Strawberry Crop Monitoring

Precision agriculture aims to use technological tools for the agro-food sector to increase productivity, cut labor costs, and reduce the use of resources. This work takes inspiration from bees vision to design a remote sensing system tailored to incorporate UV-reflectance into a flower detector. We demonstrate how this approach can provide feature-rich images for deep learning strawberry flower detection and we apply it to a scalable, yet cost effective aerial monitoring robotic system in the field. We also compare the performance of our UV-G-B image detector with a similar work that utilizes RGB images.

The development of digital cameras, using CCD and CMOS sensors, more powerful computers, and object detection algorithms have been key to precision agriculture.Today, neural network algorithms combined with RGB or hyperspectral camera data has given rise to vegetation indices that can measure soil and plant health, crop growth, and nutrient requirements [6].It is also showing great potential for weed and disease detection, input requirements, and crop yield estimation [2].
In order to support artificial pollination, the robotic system first need to detect and localise the target flowers.This feature also provides data for crop counting or flower counting for yield estimation. Manual stand counts are labor intensive and prone to human error.By using a UAV mounted camera and AI software, automated, non-subjective yield estimation can be achieved at a fraction of the cost and effort.Most of the existing remote sensing systems are based on human vision (RGB); mimicking farmers visually observing their fields.However, flowering plants have coevolved for millennia to interact with insects whose vision spans the UV-G-B range [7].Pollinating insects such a honey bees (Apis sp.) can distinguish crop species and cultivars from one another based on floral patterning undetectable to human vision [7], against a complex background, and while airborne.This is due to the greater contrast seen in the UV-G-B spectrum than the RGB.Our hypothesis is that the UV-G-B spectrum would be better suited to UAV platforms for flower detection as it would mimic pollinator vision and detect intended plant cues for aerial pollinators.
This paper presents a biologically inspired UV-G-B camera (Sec.III), which mimics a natural pollinators' vision, for detecting crop flowers using state-of-the-art object detection algorithms tuned to the task (Sec.IV.We then validate the technology with a complete deployment onboard a UAV in a strawberry field (Sec.V), and show that our detector yields better results compared to another similar deployment which used RGB images.

II. RELATED WORK
From huge phenotyping platforms such as Field Scanalyzer [8], to unmanned ground and aerial vehicles (UGVs and UAVs), many platforms are currently used for gathering field data.There are already several options of ground vehicles available for sensing and phenotyping platforms [9], [10].Nevertheless, recent years have seen a tendency towards the use of UAV for precision agriculture [11], [12].Depending on the application, some aerial platforms offer a better set of features than others.Fixed wing platforms have greater flight time and more payload capacity, however, it is difficult to arXiv:2210.16923v1 [cs.RO] 30 Oct 2022 get higher quality data due to limited hovering capabilities.Blimps offer the advantage of simpler operations but it becomes difficult to operate in harsh outdoor conditions.Rotor copters have better hovering capabilities and thus offer a better chance to capture higher quality imagery.However, these platforms have limited flight times [13].It also presents a very comprehensive review of the advantages and disadvantages of different types of platforms for applications in the domain of agriculture.As for artificial pollinators, UGV based robotic systems have been already proposed for Kiwi fruit pollination [14], [15] and used in poly-tunnels [16], [17].
The most frequently used remote sensors for precision agriculture applications are visible light sensors (RGB).These light weight, inexpensive sensors replicate the human vision range (400-700nm) when capturing images and they benefit from the largest literature on object detection algorithms.Detecting flowers and fruits using these sensors have led to different types of algorithms, such as flower contour detection through color and edge detection [18], and spectral spatial methods for e.g.detection of tomatoes [19].However, their main disadvantage is the inability to analyze any parameter outside the visual spectrum [2].Other spectral bands offer valuable data about the crops, such as the state of chlorophyll, which reflects near infrared radiation (NIR).Naturally, when a plant becomes diseased or stressed, the amount of chlorophyll reduces -resulting in a an overall changed spectrum within this range.Studies have shown that UAV based NIR imagery can be used to accurately detect disease and water stress in crops [20], [21].
As such, the use of multispectral cameras in agriculture is growing in popularity.With a Visible Near-Infrared (Vis-NIR) camera, a robotic system can detect powdery mildew in asymptomatic squash plant with 89% accuracy under field conditions [22].Similar sensor can also serve to study water stress in apple orchards showing a strong correlation (R² = 0,9975) between temperature measured on ground targets and estimates made from aerial images [23].Stumph et al. [24] used UV LED lights on a UAV with an RGB camera to induce fluorescence in tree dwelling insects and they achieved detection precision as high as 80%.Thus the complete plant reflection spectra, acquired using multispectral and hyperspectral cameras, is key to monitoring several important plant health and growth attributes.
However, these cameras are expensive, heavy and their wide spectrum is mostly underused for specific crop monitoring tasks.Taking inspiration from biological pollinating agents such as honey bees, a spectrum of near-UV, Green, and blue light is used to perceive and identify flowers [7].The lack of development of such sensors may come form the lack of studies exploring the use of UV floral reflectance of plants; let alone crops.Spectral reflectance studies have reported down to 300nm but most species in reflectance databases, are native species not agricultural crops [27].Furthermore, UV cameras and lenses in these studies were bulky and very expensive; leading to even fewer field studies [28].Fig. 1: Spectral Sensitivity of Western Honey Bee compared to Human vision range.Bee Uv, B, and G are represented by the dotted black, blue and green lines respectively.Human sensitivity for Red, Green, and Blue correspond to the solid line colors [25], [26].
This project aims at developing an efficient, biologically inspired flower detection system, which leverages the UV-G-B spectrum using a powerful yet cost effective drone setup.

III. UV-G-B SENSOR DESIGN
The design of our robotic system was tested on a representative of the Rosacea family of crops; the strawberry.Canada produces 24,185 metric tons of strawberries annually; making it the 5th most valued fruiting crop for Canada in 2020 [29].Unlike Rosacea orchard species (e.g., apple,) some commercial strawberry cultivars are everbearing; flowering throughout the growing season.This, along with its compact habit and flowers held on stocks above the foliage, made it an ideal crop to use for aerial imagery in this study.Strawberry flowers are mainly pollinated by various bee species; primarily Apis mellifera, or western honey bees, in agricultural settings [30].Like humans, bees perceive in 3 light ranges.While humans see from 400-700nm, from blue to red (RGB), honey bees see from 300-650nm, near-UV to green [7].Both sensitivity curves are plot in fig. 1.Most digital cameras are designed to capture images in the human visual spectrum, e.g.RGB.This is achieved by coating the sensor with a Bayer filter which is a pattern of red, green and blue filters across the photodiode matrix [31].
In order to create a camera which can capture images similar to pollinator vision, the light reaching the sensor must be restricted to the UVGB range.Also, to acquire usable data, the aerial vision system have to provide a ground sampling distance (GSD) comparable to existing flower detection aerial imagery; ideally in the range of 0.7 to 3 cm/pixel [32]- [34].GSD is computed with: where H is the flight altitude (m), c, the Focal length (mm), and λ, the camera sensor pixel size.

A. UV-G-B Camera Design
We designed light weight, affordable, and drone mountable camera after the vision range of most insect pollinators: 300-650nm [7].
The sensor is based on a MaxMax XNite USB8M-M [35], which provides a video capture resolution of 3264x2448.Considering a flight height of 3m, we can expect a GSD = 1.16cm/pixel.These cameras retail for 500 USD and are highly customizable to adapt to the specific needs of the application.The X-Nite camera is most comparable to a GoPro [36] in terms of size and weight.See Table I   Unlike the GoPro, the X-Nite has a USB cable which allows the camera to be connected directly to a computer for image capture and continuous power draw.In order for the camera range to extend to the UV range (300-400nm), the optical window, or ICF, was replaced with Schott WG280 glass which transmits light down to 280nm [37].The glass of the lense was also replaced with Schott WG280 glass and had no laminated filter installed.To restrict the upper range of the spectrum, the X-NiteBP1 filter from Max Max was chosen.It has 80-100% transmission between 300-650nm; ideal for imitating insect vision in a monochrome image.Figure 2 show the effect of the lens and filter to the sensor's spectrum sensitivity.The camera's CMOS sensor had its Bayer filter and micro lens removed; increasing transmission down to 300nm.Removal of the Bayer filter creates a single-channel monochrome sensor.Inherently, monochrome sensors can achieve higher resolution, faster processing times or frame rates, and be stored as smaller files than their RGB counterparts.It also means smaller processing time as colored images have to be processed over three dimensions.The Bayer filter itself can reduce the optical resolution of the system [38].Since our use case requires the removal of the red filter, the Bayer filter was completely removed to optimize the camera's performance.

B. Camera Characterization
As commercially available reflectance standards reflect UV (320-400 nm) poorly, we created an appropriate reflectance standard following the work of [39].Five standards were made from varying proportions of medical-grade MgO, Plaster of Paris, and activated carbon [39]; we also added a sixth standard consisting of a black UV-absorbing plastic.Each standard's (1-6) spectral reflectance (% R) in the range 200nm -700nm was measured with a Perkin Elmer's Lambda 850 UV-VIS spectrophotometer.Figure 4 shows the resulting spectrogram.The intercept for each curve denotes the consistent % R each standard will emit across varying lighting conditions.Images of the standards were then taken with our camera and an average pixel value for each sample was attained (avg. of 10 pixels).Fig. 3 shows the strong linear relationship R 2 = 0.9821 between % R and the pixel value our sensor produces.On competition data sets Convolutional Neural Networks (CNN) have proven themselves to be the best approach for target identification and classification due to their ability to extract increasingly complex visual features through its Fig. 4: Spectrogram of reflectance standard samples with spectralon as a control hierarchical structure [40].In regards to strawberry flower counting, Faster Region-based Convolutional Neural Networks (RCNN) have shown best results (86.1%, 86.4%, and 86% accuracy respectively from [33], [41], and [42]).In an aerial application [33] recently employed Faster R-CNN for strawberry flower detection with a DJI phantom 4 pro and attained mAP= 0.772 [33].Given their flight altitude of 3m and mixed training data set of strawberry cultivars this is the most recent comparable work to our current study.
However, Faster RCNN often has a longer processing time [43] as compared to Yolo.Furthermore, Immaneni et al. [44] tested YOLOV4 [45] drone images from a strawberry field and achieved a better accuracy (91.95% at 14.6 FPS).Related species such as Pear flowers, have also shown promising results with Yolo (mAP of 94%) [46].

A. UV-G-B Strawberry Flower Dataset
Since no image datasets of UVGB images are available, we create an original dataset using various strawberry cultivars.All datasets generated from this study are openly available on the Roboflow platform and can be accessed throught the IEEE dataport.F. x ananassa 'Seascape' and 'Fort Laramie' are commonly recommended commercial everbearing cultivars for central and east coast Canadian provinces [47].Therefore, they were included along with F. x ananassa 'Hecker'; which is an older everbearing cultivar which was popular with commercial growers for decades prior to 'Seascape' [47].F. Vesca is a North American woodland native grown by breeders for its genetic attributes and specialty fruit growers for wine.All bare root plants were procured from a local Quebec nursery.
Images were captured under sunny conditions between 11am and 1pm within the 300-630nm.The camera was stationed 6cm from an open flower at a 90-degree overhead angle.Images were captured with 640x480 resolution (focal length= 3.6mm, GSD= 0.023 cm/pixel).An exterp of the dataset is shown in fig.5.Roboflow Inc. [48] was used for data management and bounding box image labeling.Images were Preprocessed for auto orientation and resized into a square shape, 416x416, for detection algorithm compatibility.The following augmentations of the dataset were then applied: horizontal and vertical flip, rotate ±90-degrees, rotate ±15-degrees, sheer ±15-degrees vertical and horizontal, noise 5%, and blur 5px.The resulting dataset consisted of 284 UV-G-B images.
We based our code implementation on publicly available script for both YoloV5 [49] and Faster R-CNN [50].Pretrained weights for both algorithms were initially taken from MS COCO2017 [51].We compared the algorithms using mAP@0.5,True positive (TP), False positive (FP), and False negative (FN) metrics.YoloV5s version was chosen due to the small training dataset in this study and ran for 1000 epochs.Faster R-CNN was run with inception V2 for 20000 steps.Training/validation/test split was 40%/40%/20%.Table II shows the results for the trained algorithms.YoloV5s showed overall higher performance than Faster R-CNN and was therefore used for field testing of the systemn.

V. FIELD DEPLOYMENT
To validate the usefulness of our solution, we deployed our sensor on a commercial UAV and flew over a local strawberry field.

A. Aerial System implementation
We selected the UAV platform as to maximise the payload capacity while minimizing its cost and size.Spiri Mu [52] stood out as the best option compared to other commercially available devices.Table III details the comparison over the potential options we considered.The Mu is powered by an Nvidia's TX2, powerful enough for heavy onboard image processing.To mount our camera and interface it to the onboard computer, we replaced the original underbelly of the Mu with an in-lab 3D printed attachment which could house the mounting for the camera.Moreover, the original landing gear of the drone was replaced by wooden dowels to make it taller in order to accommodate the additional sensor.Figure 6 shows the modified drone in the field ready for take off.
The drone is connected to Qgroundcontrol (QGC) for calibrating onboard sensors and monitoring mission parameters during flight.The onboard system runs on Linux with ROS preinstalled which simplify the software integration of our sensors, and provide us with tools to record and transmit data efficiently.Our ROS camera driver is based on common ROS packages to fetch the USB camera feed and convert it to a ROS image topic.

B. Field deployment setup
Field flights were conducted at Pepinerie F. Fortier near Princeville, Qc. containing white-flowering 'Seascape' and 'Albion' cultivars.To keep consistent with the training dataset, video was captured under sunny conditions from 11am-1pm.altitude of 3m reduced the effects of rotor downdraft on plants and produced a GSD=1.16cm/pix.Spanning 3 crop rows, 88 frames were captured and 2295 flowers were counted by a single reviewer.Frames were tiled and resized to 768x768 pixels to reduce processing time.

C. Flower detection on UAV images
Once aerial images were attained, the flower resolution was much lower than for the training dataset.YoloV5 and Faster R-CNN was therefore trained on the same image dataset as before but at a lower resolution, 96x96 pixels, to increase TP detection.Inferences were run with 0.51 confidence threshold on a Tesla P100-PCIE -16 GB in Google colab [53].Table II

D. Results
Although TP detection proportion was low (37.1 percent) for YoloV5 ,overall, 97 percent of flowers were accurately detected (n=1218 of 1243 in dataset).However, the FP detections (n=2042,62.2 percent) far out numbered the TP leading  to a lower proportion.33.3% (n=680) of FP detections were accounted for ripening or developing fruits 7bb.A similar result was attained with Faster R-CNN ( 17.6 %).YoloV5 showed overall best performance for our system.As our algorithm was not trained with strawberry fruits, and they exhibited similar spectral properties as the flowers, this lead to increased FP detections (see for instance Fig. 7).[33] conducted a similar flower detection study on strawberries using a Phantom 4 Pro to capture aerial images in RGB.
Table IV compares their results with ours.[33] used Faster R-CNN with Resnet50 trained on Imagenet at a flight altitude of 3m.
Our system has a higher mAP with Faster R-CNN as compared with their study, however, overall detection was low.Our YoloV5 algorithm vastly outperformed both Faster R-CNN algorithms in terms of training time (0.3 vs 4.5-5.5 hours) and image processing (0.0083 vs 1.86-8.872FPS).mAP was also significantly Higher for both YoloV5 ad Faster R-CNN on our monochrome images [33] RGB images(mAp=0.951vs mAP= 0.772).

VI. DISCUSSION
Farmers, like the ones from our test field, re-plant yearly for best harvest results.A tool for flower detection would need to be robust to changing varieties and cultivars and be able to provide consistent results.The test field was randomly planted with both 'Seascape' and 'Albion' cultivars.Although 'Seascape' was included in our initial training dataset 'Albion' was not.Our algorithm was able to detect novel cultivar flowers as strawberry flowers.This indicates that a limited database of strawberry cultivar flowers could be sufficient for developing remote sensing tools and would  Schaefer, McGraw and Catoni [57] explored the effects of fruit color variation as signals of dietary reward.They found that fruits rich in anthocyanins, a plant antioxidant, are black or UV-reflecting and are in higher concentrations in ripe fruits.In strawberry fruits, anthocyanin concentration increases with fruit maturity [58], [59].As our camera perceives the UV but not Red spectral range ripening strawberries appear bright white in our frames.When at similar size and circularity these fruits are mislabeled as flowers by our algorithm.
Remaining FP detections can be attributed to solar reflection on leaves, runners and flower stocks.Theses are expamples of the complexity of our field setting.Including null images of background foliage in initial training would also increase overall algorithm robustness.However, no FP were attributed to weed species which were visible within and between crop rows.These included: Fleabane, lambsquarter, crabgrass, purslane, and cow vetch.Some of these plants were in flower however the algorithm could distinguish between these and strawberry blossoms.This aspect may be explored in future research.A significant limitation to this research was the aerial resolution of the flowers.

VII. CONCLUSION
In this paper we've described and discussed the development of a cost-effective, light weight, UV-sensitive camera, used with a promising aerial platform.We performed extensive field experimentation to gather high quality imagery data and demonstrated its usability feeding it to two state-of-theart object detection algorithms.The improved results over previous similar experiments show that our system is highly scalable, as the cost of the system is low, and that the UV spectrum can provide valuable information about the crops' flowers.
The development of this sensor and the choice of the aerial platform opens up the opportunities for a lot of potential work in the future.The development of a flower detector is the first step towards the bigger goals, such as estimating crop yields and creating a functional, biologically inspired robotic pollinator.Currently, using ROS onboard of Spiri Mu drone, the GPS coordinates are being recorded as well as rostopics.In the future, these GPS coordinates can be used in conjunction with the images to create a global map of the field and to direct harvest efforts.Additionally, developing automatic row cropping methodologies would help in getting better results with the object detection algorithms.Furtur iterations of this study will also include strawberry fruits at various developmental stages in the initial training process.Our sensor design shows a linear relationship between percent reflectance and pixel value between 300 and 650 nm opening up many possibilities for it's use even beyond agriculture.Although in this study we designed a filter which allowed 300-650 nm, the sensor can easily be modified into a UV-only camera with the addition of another filter such as the Xnite330 from Maxmax.Our design can be ordered to specification; and, the sensor transmission spectra is provided upon request.Lastly, our monochrome sensor design does not allow for the use of contrast between chroma channels as RGB sensors or animal eyes do.Furtur sensor iterations should explore removing only the red color of the Bayer filter.This would retain the green and blue channels and replace the red channel with UV using our existing camera body design.Computer analysis for 3 channel images could be carried out routinely as the proportion and placement of the three channels on the sensor diode are unchanged.Thus, a number of enhancements to this platform can improve the overall performance of the system and can help us make significant contributions to accelerate the research in this area.Agriculture has, and always will be, a sector for innovation.We openly make our datasets and trained algorithms available in hopes of furthering scientific pursuits.

Fig. 3 :
Fig. 3: Relation between % Reflectance of standard samples and pixel value produced by our camera sensor.

Fig. 5 :
Fig. 5: From left to right: strawberry cultivars 'Fort Laramie', 'Hecker', 'Seascape', and wild strawberry sp.vesca.Top row images are taken with RGB camera; bottom row images are with UVGB camera.The far right image shows Yolo detection of a strawberry flower on the training dataset.

Fig. 6 :
Fig. 6: Our Spiri Mu prototype with the near-UV camera sensor pointing down and extended legs.

Fig. 7 :
Fig. 7: Detection examples from aerial images of TP (a) detections of open flowers and FP (b) from ripening fruits

TABLE I :
Our camera model contrasted with a comparable camera on the market

TABLE II :
Resulting detection from UVGB trained YOLOV5 and Faster R-CNN on training dataset at 416x416 resolution, and, on aerial images at 96x96 resolution and table IV show the results of our trained YOLOV5 on the aerial images.

TABLE IV :
[33]UVGB camera detection results on aerial images compared to those from[33]