YOLO: A FUSION APPROACH TO GEOLOCATING NATURAL RESOURCES 

Technology

​​​

"You Only Look Once" Computer Vison Model

AI or deep learning allows researchers to process tens of thousands of images. YOLO, which stands for “you only look once” is a state-of-the-art computer vision model designed to detect and classify objects in imagery. Although a relatively new technology, UAS thermal imaging data and red, green, blue (RGB) camera images have successfully been integrated into YOLO for effective wildlife detection. We plan to leverage our UAS-based monitoring efforts with new data acquisitions to develop UAS data collection protocols for thermal and visual camera image data to optimize YOLO detection of raptor nests in near real-time.

​

YOLO has two key advantages; it is fast, and it has been shown to be accurate in a variety of classification tasks. Wildlife detection approaches in UAS imagery can use a segmentation approach to group similar pixels together that have a characteristic of shape, texture, color, and size to the target object, but these approaches are time consuming and computationally intensive. Other object detection models use a brute force moving window approach which can also be slow and ineffective – in contrast, YOLO applies a single forward pass neural network to the entire image but is still robust relative to varying image size and location.​

​

Sensors

We aim to address one of the major shortcomings of YOLO by incorporating leading edge LiDAR and camera fusion to increase performance and object detection with precise geographic location, especially in 3D environments. This fusion provides an additional opportunity for collecting precise metrics on object size and relative position. To leverage the power of these toolkits for natural resource observations in remote and complex terrain environments, data acquisition with UAS is a logical choice.​

​

Uncrewed Aerial Systems (UAS)

We will utilize a multirotor platform equipped with sensors for visual and thermal imagery for centimeter-level data acquisition. A Mission Control system is responsible for developing, planning, and executing autonomous UAS missions. Missions are programmed through defined waypoints, which are uploaded to the UAS, allowing for fully automated flight operations tailored to specific data collection objectives. Our flights will be parallel to cliff faces and maintained at a minimum distance of 50 m away. During the flight mission, the UAS will maintain a steady speed and not pause along the route or move in closer to any nest sites. By flying in a steady pattern and direction without changes in speed, our experience has indicated minimal disturbance to occupied eagle nests. Further, onsite eagle observers will be in communication with the pilot to report any evidence of potential adverse responses such as flushing from the nest or eagle approach of a UAS. Flight operators set up a survey-grade GPS base station at the flight site to collect static positioning data, which is later used in Post-Processing Kinematic (PPK) workflows. This enhances the geospatial accuracy of the imagery and sensor data collected during flight, ensuring alignment with ground truth coordinates. This station will be set up in a position that ensures the UAS stays in line-of sight of the operator at all times. 

​​

Processing

To overcome the challenge of field based near real-time processing and detection of natural resource observations, computational solutions such as the Microsoft Azure Stack Edge Pro R can be used. This system offers a rugged, physical device, and a local web user interface for use in harsh environments with graphical processing unit (GPU) capabilities and processing power to facilitate data pre-processing and AI inferencing with YOLO algorithms for field based operations with the option to securely transfer data collected to the cloud (i.e. Azure Government) for further computation or archival purposes. Researchers are already leveraging similar tools by uploading thousands of images from wildlife cameras and aerial imagery for cloud-based analysis using deep learning tools for animal detection in images. Yet these tools lack spatial awareness and positioning of observations in 3D space. Our proposed solution would overcome this limitation with the use of a spatially aware UAS camera/LiDAR sensor and a YOLO workflow to identify natural resources in near real-time with geolocation information. This solution is useful for land managers to rapidly identify potential conflicts onsite and raise awareness of wildlife presence.

​

Validation

In addition to model development, we will validate our technology and protocols by surveying in areas previously unsampled during model development. By incorporating testing in novel environments, we can adapt our technique to emerging challenges. This will help to ensure that technology and protocols developed in this project will be transferable across different landscapes and can be generalized to other taxa.​​​