We broke down the process of developing a geospatial AI solution for the detection of roof faces into the following phases:

Selection and import of a suitable roof face dataset

Creation of an appropriate segmentation config file using geolibs

 for parallel training of multiple models

Tracking of experiments’ progress and comparison between multiple experiments through 

We selected the Roof Information Dataset(RID)[1]: a standard remote-sensing dataset of roof faces and superstructures containing 1880 annotated roofs. The images cover an area of 1.5 km2, excluding overlap, and 4.9 km2 in total. The roof segment annotations include 4520 polygons, and their classes were derived by its azimuths. Lee et al. selected 16 azimuth classes, for example, south, south–south–west, and south–west, plus a class for flat roofs and one for trees. Usually, roof orientations are biased towards north, south, east, and west for architectural reasons. Therefore, we provided code to classify roof segments in three different ways.

As the first stage of solution generation, we imported this dataset into GeoEngine platform 

 for visualization and exploration purposes, and generated an export for model training in later stages.

: our proprietary geospatial AI library that acts as a common interface between multiple open-mm-lab libraries (e.g. 

, etc), specialized towards geospatial AI solutions’ development. 

 provides an interactive interface for configuration file generation, wherein we only need to specify our registered organization, GeoEngine project, export version, and public URL of the base mm configuration file from GitHub. Below is an example of how we interface with 

 creates a config file based on our URL: 

https://github.com/open-mmlab/mmsegmentation/blob/main/configs/segformer/segformer_mit-b0_8xb1-160k_cityscapes-1024x1024.py

 in the current directory, provides instructions to download the export version of our choice, and instructions to train our model. We make minute changes to our config file to be able to properly ingest our dataset and set the environment for training.

 to take in the config file path as an argument and initiate training. For our specific use case, we invoked the following command:

This initiates training in our environment. For the roof face solution, we instantiated multiple EC2 instances, set up 

, used its interactive interface to generate configuration files for various model types, and ran multiple experiments in parallel.

 is integrated with engine CLI, a library that interfaces with the GeoEngine platform to track the progress of experiments. We configured engine configs in our individual config files to track and visualize experiment statistics in the platform 

. GeoEngine also allows us to select multiple experiments for comparison.

We tabulate the results of our experiments below:

Based on the table above, the experiment 

 has the best mIoU and mAcc values of 55.57 and 68.42 respectively, so we select this experiment to be deployed as a roof face solution.

[1]: Krapf, S., Bogenrieder, L., Netzler, F., Balke, G., and Lienkamp, M. 2022. RID: Roof Information Dataset for Computer Vision-Based Photovoltaic Potential Assessment. *Remote Sensing, 14*.

Let's explore the step-by-step process of developing a solution to detect roof faces in a low-code no-code manner using geolibs: our geospatial AI library.

A geospatial AI solution to detect roof faces

Uploading files is one of the most basic features of web development. However, it is not without its complexities. Designing a reliable and fast upload feature for a large number of files is still challenging. The naive upload process sends bytes from a client to a server, which receives said bytes and writes them into a file. Essentially, this still remains the same regardless of the number of files or their size needed to upload. You start to run into issues with resource overload, namely CPU, RAM, and network when the number of files and their size increases.

The front end makes all the requests using the client’s resources. Sequential requests for all the files would be very slow, whereas simultaneous (concurrent) requests to upload would overwhelm the client’s resources. In the case of file I/O, reading them one by one is fine and slow, whereas loading too many crashes the client.

The same goes for the server as well. Sequentially serving upload acknowledgments is slow, whereas simultaneously receiving too many upload requests can crash the server as well, It’s more capable than a client’s computer, but there is a limit.

Then there is the expectation around the speed of upload. Say you have 100Mbps network bandwidth, to upload 100 files 10MB each, which would ideally take 

. But in reality, for each file, there has to be an IO operation and request, and response operations, which takes CPU time. This inflates the total time to upload a single file by a lot (can take up to a full minute per file as well), Numbers may vary but it could be as large as 40-60% CPU time and the rest over the network. The network bandwidth is only partially utilized. This only increases as the file size grows as well as the number of files.

We cannot ignore the possibility of network failure, RAM out-of-memory error or similar issues, that can interrupt the upload. Upload features for large data must provide reliability as well and not just speed, as the process takes time anyway. Re-doing a large upload is usually not a good option,

At last, there is the issue of the maximum size of payload a request can carry. If your requests are routed by Cloudflare, then 100MB is the limit.

The most obvious choice to improve upload speed is not to send requests sequentially and send them concurrently from UI. The CPU usage per file may look like a lot of overhead, but improving our network utilization by concurrent requests can improve the upload times by a lot. Another addition would be to chunk the files into a manageable payload size (e.g. 1MB). While this is counterintuitive to our discussion and the number of requests and responses by order, it offers two main benefits.

The first would be a clear view of the progress of the upload, this can be reliably tracked with acknowledgments for each chunk. With the payload much smaller, the CPU processing time for each request and response decreases substantially, and the payload can be sent over much more frequently than a large request. This isn’t a new solution and is referred to as a multipart upload process. Concurrently sending file chunks turns out to be reliable and very fast. On the client side, just by managing the maximum number of concurrent requests and file I/Os, we can prevent freezes and crashes as well. A naive approach like a hard limit on the number of requests or files loaded into memory also works.

Uploading a large number of files over the network is a challenge. The author provides an opinion piece on the insights gained while designing a reliable file upload feature.

Challenges in Building a Reliable Upload Feature

Wildfires, whether started by nature or human actions, can leave destruction in their wake. In 2020, the U.S. witnessed over 10 million acres consumed by flames, resulting in significant financial costs. Managing these disasters efficiently requires a deep understanding of high-risk areas and effective resource allocation.

In the realm of geospatial AI, where technology takes on the task of identifying wildfire-affected areas, Siamese UNet emerges as a standout solution. This research was done by our team at Granular. Today, we'll explore Siamese UNet's vital role in addressing the aftermath of wildfires and its promising implications for the future of disaster management.

Siamese UNet, an evolution of the UNet framework, introduces a critical innovation to enhance its change detection capabilities. This enhancement comes in the form of an attention mechanism, a sophisticated feature that enables the model to selectively focus on pertinent regions within the images. By doing so, it can prioritize the extraction of informative features, ultimately leading to a substantial improvement in the overall performance of change detection algorithms.

As you can see in the above image, the network structure consists of several building blocks, each incorporating convolutional layers, batch normalization, and ReLU activations. These convolutional operations employ 3x3 filters with a stride and padding of 1. It's important to note that the encoding process is executed independently for both pre-fire and post-fire images. Subsequently, the decoder takes the final features extracted from the encoder and initiates the upsampling process to restore the original dimensions.

All experiments in this study were conducted using the 

dataset, which comprises Sentinel-2A satellite images capturing forest fires in California. These images contain twelve channels, encompassing data from the visible spectrum, infrared, and ultrablue. A total of 1,766 images were utilized, partitioned into training and validation sets, with a hidden test set employed to establish final results.

To enhance model performance, data augmentation techniques from the Albumentations Library were applied. This involved combinations of horizontal/vertical flips, random crops, grid distortion, optical distortion, and elastic transform.

The focal loss function, known for its effectiveness in training architectures, was chosen over traditional cross-entropy loss. Focal loss places greater emphasis on challenging examples, ensuring continuous improvement in predictions, particularly for difficult cases.

The results display the best four predictions from the validation set. Each row showcases three columns: the first exhibits the pre-fire image, the second showcases the post-fire image, and the third presents the true and predicted masks. In the masks, white pixels represent the ground truth; green pixels indicate correct predictions, and red pixels denote false predictions.

Siamese UNet detects wildfire-affected areas with remarkable accuracy, offering a practical solution for disaster management.

Post Wildfire Burnt-up Detection on ChaBuD dataset

Detecting change through multi-image, multi-date remote sensing is essential for developing and understanding of global conditions. Despite recent advancements in remote sensing realized through deep learning, novel methods for accurate multi-image change detection remain unrealized. Recently, several promising methods have been proposed to address this topic, but a lack of publicly available data limits the methods that can be assessed. In particular, there exists limited work on categorizing the nature and status of change across an observation period.

Mou et al[1] proposed a deep patch-based architecture wherein bi-temporal patches are processed in parallel by a series of dilated convolutional layers generating features which are then fed to a recurrent sub-network to learn sequential information. In the end, fully connected layers are used to create the change prediction map. Although accuracy, such patch-based methods are very time-consuming since they need to process every single pixel of the image individually

A number of methods use transfer learning to overcome the problem of scarcity of training data through transfer learning by using pre-trained networks to generate pixel descriptors [2,3]

Daudt et al. [4] suggested three different fully convolutional Siamese networks based on the U-Net architecture [25] aiming to address this problem and detect accurately the regions with changes. This approach, however, lacks the appropriate modeling of the data’s temporal pattern.

Papadomanolaki et al. [5] proposed a convolutional LSTM-based approach that takes in five dates to detect changes in the first and last date’s images. This method is resilient to seasonal changes, cloud covers, and shadows

Daudt et al. [6] proposed a weakly supervised change detection method using guided anisotropic diffusion and iterative learning.

: OSCD [7] dataset is a bi-date change detection dataset comprising 24 Sentinel-2 image pairs of two dates from 24 different cities, mostly European. Seasonal change, cloud cover, and shadows are present in some of the images. In total, there are 1769 change polygons

: Papadomanolaki et al. [5] extended the OSCD dataset to 5 dates by introducing three intermediate dates to show that by having more dates their proposed method is resilient to false positives occurring due to seasonal changes, cloud covers, and shadows.

 LEVIR-CD [6] is a large-scale remote sensing building change detection dataset. LEVIR-CD consists of 637 very high-resolution Google Earth image patch pairs with a size of 1024 × 1024. These bitemporal images with a time span of 5 to 14 years have significant land-use changes, especially the construction growth.

: HRSCD [7] contains 291 coregistered image pairs of RGB aerial images from IGS’s BD ORTHO database. Pixel-level change and land cover annotations are provided, generated by rasterizing 50 cm resolution images from Urban Atlas 2006, Urban Atlas 2012, and Urban Atlas 2006-2012 maps. Change and no-change labels are available along with six land cover classes for pixel-level LULC.

 We select 504 different locations from 100 different cities maximizing dataset coverage of all geographic and urban types. For each of these 504 locations, we sample satellite images for five different dates such that the duration between two consecutive date-pairs are almost equal. An attempt was made to introduce different months to capture seasonal patterns and have different seasons present on different dates for each location. Image selection was further restricted to images with no more than 5% cloud cover.

 For our baseline, we used 3-channel RGB Images, but also examined the contributions of the near-infrared (NIR) channel. These images were enhanced with a separate, higher-resolution panchromatic (grayscale) channel to double the original resolution of the multispectral imagery (i.e., ”pan-sharpened”). Each location is a tile of size 8192 × 8192 with a resolution of 0.45m × 0.45m ground sample distance. The 16-bit pan-sharpened RGB-NIR pixel intensities were truncated at 3000 and then rescaled to an 8-bit range.

 A team of 20 annotators annotated the ground truth details in two phases. In the first stage, each location is assigned to an annotator to annotate all change polygons present in that location from first and last date images. In the second stage, each annotated polygon is used to crop out images from five date images. A buffer was introduced around the region of interest to ensure that annotators had a perspective on the surroundings, enabling them to annotate change type, change status, geographic type, and urban type.

The details of the dataset can be observed here: 

https://engine.granular.ai/organizations/granular/projects/631e0974b59aa3b615b0d29a/overview

. We also have a paper published on our dataset: 

https://openaccess.thecvf.com/content/CVPR2021W/EarthVision/papers/Verma_QFabric_Multi-Task_Change_Detection_Dataset_CVPRW_2021_paper.pdf

We split the RGB dataset into training, validation, and test sets in the ratio of 70:20:10 by randomly selecting cities. Patches of size 512 × 512 are used by taking a striding window with stride size 512, and 6 different mask files are generated for each city grid: a change type mask and 6 change status masks. We randomly applied augmentations: 90-degree rotations, X and Y flips, imagery zooming of up to 25%, and linear brightness adjustments of up to 50%, to training images. We ran our experiments for each of the problem types: Change Detection, Change Type Classification, Change Status Tracking, and Neighbourhood Classification.

PyTorch is employed to implement all networks. In order to manage our experiments we use Polyaxon on a Kubernetes cluster and use three computing nodes with eight V100 GPU each.

[1]: Lichao Mou, Lorenzo Bruzzone, and Xiao Xiang Zhu.Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery. CoRR, abs/1803.02642, 2018.

[2]: Arabi Mohammed El Amin, Qingjie Liu, and Yunhong Wang. Convolutional neural network features-based change detection in satellite images. In Xudong Jiang, Guojian Chen, Genci Capi, and Chiharu Ishll, editors, First International Workshop on Pattern Recognition, volume 10011, pages 181 – 186. International Society for Optics and Photonics, SPIE, 2016.

[3]: Ken Sakurada and Takayuki Okatani. Change detection from a street image pair using cnn features and superpixel segmentation. BMVC, pages 61.1–61.12, 2015.

[4]: Rodrigo Caye Daudt, Bertrand Le Saux, and Alexandre Boulch. Fully convolutional siamese networks for change detection. In ICIP, 2018.

[5]: M. Papadomanolaki, S. Verma, M. Vakalopoulou, S. Gupta, and K. Karantzalos. Detecting urban changes with recurrent neural networks from multitemporal sentinel-2 data. In IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, pages 214–217, 2019

[6]: Hao Chen and Zhenwei Shi. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sensing, 12(10), 2020.

[7]: Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, and Yann Gousseau. Multitask learning for large-scale semantic change detection. Computer Vision and Image Understanding, 187:102783, 2019.

[8]: Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, and Yann Gousseau. Urban change detection for multispectral earth observation using convolutional neural networks. In IEEE International Geoscience and Remote Sensing Symposium, IGARSS. IEEE, 2018.

Curation of an accurate temporal multi-task change type and status detection dataset.

QFabric: Multi-Task Change Detection Dataset

Recent developments in large-scale text and vision models have revolutionized downstream task solutions. These large-scale models are termed foundational models and their versatility can be attributed to the vast data requirements and high modeling capacity (large number of tunable parameters). An example would be the GPT, a large language model from OpenAI, which took the world by storm with its chat-based application ChatGPT. In our recently 

 2023 [1],  we look at the performance of such foundational models in one downstream task of image captioning using satellite imagery. The findings point towards subpar results (near-random performance).

Authors test the zero-shot performance of CLIP [2] and BLIP [3] language models, along with their image-encoder-based variants on remote-sensing datasets EuroSAT and BigEarthNet-S2 [4, 5]. EuroSAT is a Land User / Land cover dataset with 27k images and BigEarthNet-S2 large-scale multi-label dataset with 590,326 Sentinel-2 patches. More details of the experiments can be found in [1].

The above table shows the zero-shot performance of CLIP and BLIP with different backbone networks. ‘Standard’ refers to the out-of-the-box model and ‘Context’ refers to the fine-tuned model on geospatial datasets. It is worth noting that, while the standard model has subpar performance, adding geospatial context doesn’t necessarily improve performance and is subjective to the model and the dataset.

Akash Panigrahi, Sagar Verma, Matthieu Terris, Maria Vakalopoulou. Have Foundational Models Seen Satellite Images?. IGARSS 2023 - International Geoscience and Remote Sensing Symposium, IEEE, Jul 2023, Pasadena, United States. hal-04112634f

A. Radford, J. W. Kim, C. Hallacy, et al., “Learning transferable visual models from natural language supervision,” in ICML, 2021.

J. Li, D. Li, C. Xiong, and S. C. H. Hoi, “BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in ICML, 2022.

P. Helber, B. Bischke, A. R. Dengel, and D. Borth, “EuroSAT: A novel dataset and deep learning benchmark for land use and land cover classification,” IEEE JSTARS, vol. 12, pp. 2217–2226,2017.

G. Sumbul, M. Charfuelan, B. Demir, and V. Markl, “BigEarthNet: A large-scale benchmark archive for remote sensing image understanding,” IGARSS, pp. 5901–5904, 2019.

The shortcomings of large-scale text and vision models when used in conjunction with satellite images are discussed in this article. Published in IGARSS 2023.

Foundational models and satellite imagery

In the realm of property management and analysis, geospatial artificial intelligence (AI) solutions have emerged as powerful tools. Granular is at the forefront of this technological advancement, dedicated to crafting sophisticated solutions that redefine property assessment through data-driven insights. In this blog, we'll take a comprehensive look at our innovative geospatial AI solutions, focusing on roof detection, classification, and area estimation, while showcasing results from a satellite image.

Urban properties exhibit diverse architectural structures, each with its unique attributes. In this intricate landscape, our geospatial AI solutions excel in navigating the complexities of property analysis. By employing cutting-edge algorithms and machine learning techniques, we've developed a model that accurately detects and classifies various roof types, whether they are gabled, hipped, or flat. This capability allows us to unravel the intricate tapestry of urban roofing, contributing to a comprehensive understanding of architectural trends.

Our geospatial AI solution employs a sophisticated Vit model to detect roofs from satellite images. This technology can accurately identify roof contours even in densely populated urban environments, where the distinction between structures is intricate. The model's ability to decipher the outlines of different roof types contributes to a comprehensive understanding of the diverse roofing architecture.

Our roof classification model categorizes detected roofs into distinct types. This critical insight provides property managers, urban planners, and architects with a deeper understanding of the prevalent architectural trends. Whether it's the classic lines of gabled roofs or the contemporary simplicity of flat designs, our technology aids in unraveling the urban roofscape.

Accurate property valuation hinges on precise measurements, and our geospatial AI solution excels in this domain. By combining high-resolution satellite imagery with advanced machine learning algorithms, we can swiftly calculate roof areas. This quantitative data is invaluable for property assessments, insurance evaluations, and urban planning, fostering informed decision-making across the property sector.

To demonstrate the efficacy of our geospatial AI solutions, we applied them to a satellite image of 

. The results are striking—clearly defined roofs, accurately classified types, and precise area calculations. These visualizations offer stakeholders tangible insights into the potential applications of our technology, spanning property development, infrastructure planning, and beyond.

The evolution of property analysis is intimately tied to technological progress. Our geospatial AI solutions lead this evolution, reshaping the landscape of property assessment, management, and comprehension. By harnessing advanced algorithms and leveraging high-resolution satellite imagery, we empower property managers, insurers, and stakeholders to navigate the intricacies of property analysis. As the journey of property intelligence continues, our commitment stands strong—shaping a future where insights foster informed decisions and optimized property outcomes. Experience the potential of our geospatial AI solutions and step into a new era of property analysis.

Explore how our advanced geospatial AI solutions redefine property analysis, from accurate roof detection and damage assessment to precise measurements, enhancing decision-making and shaping the future of property management.

Advancing Property Insights with Geospatial AI: A Deep Dive into Roof Analysis

Recent progress in self-supervision shows that pre-training large neural networks on vast amounts of unsupervised data can increase generalization for downstream tasks. Such models, recently coined as foundation models, have been transformational to computer vision and natural language processing. In this work, we analyze the zero-shot performance of these models on standard remote sensing image classification datasets, in order to establish empirical evidence of whether these foundational models have seen satellite imagery during their training.

Zero-Shot Learning is a scenario wherein, at inference time, a model is expected to predict labels on samples not observed during training accurately. It is a measure of a model’s capability to comprehend visual concepts to be able to generalize over unseen data.

Language-vision models pre-trained on large datasets (

) are observed to have achieved tremendous success in zero-shot transfers to downstream tasks. The most popular foundation models in the public domain are:

 uses contrastive losses to maximize the similarity between image and text representations during joint training of image and text encoder

: uses the scale of the dataset to offset dataset noise to align image and text embeddings using contrastive losses

: minimises losses in dataset using a bootstrapping mechanism called CapFilt, which is used for training dual-encoder architecture

: recursively applies an efficient model in the data collection loop to curate a huge segmentation dataset with 11M images and over 1B masks.

In this work, we intend to analyze the zero-shot performance of state-of-the-art foundation models on remote-sensing data to establish whether these models have seen satellite imagery during their pre-training.

we leveraged the publicly available variants of foundation models, namely, BLIP and CLIP. These advanced models were employed on well-established remote-sensing benchmarks, EuroSAT and BigEarthNet-S2, to produce zero-shot results under two different settings:.

: In this configuration, we utilize standard prompts such as "an image of", "a view of", and so on, on the classification labels to generate text representations.

: In this mode, we use domain-specific prompts, for instance, "an aerial view of", "a satellite view of", and so forth, on the classification labels.

The rationale behind these settings, Standard and Context-based, is to investigate whether the application of geospatial context enhances the zero-shot performance of the foundational models. Any observed enhancement in performance would be a compelling sign that these models have been exposed to such images before.

Based on our results, the following key observations can be made:

CLIP has a near-random performance on BigEarthNet-S2 owing to large image-encoder activations for (almost) all classes that lead to many false positives.

Fine-tuned BLIP models have a better zero-shot performance on EuroSAT and BigEarthNet benchmarks than standard variants. Based on this, it can be safely concluded that fine-tuning standard benchmarks improves performance.

Zero-shot performance on EuroSAT improves with the addition of remote-sensing context for smaller CLIP variants like ResNet50, ResNet101, and ViT-B/32, and degrades for larger architectures like ViT-L/14 and EfficientNet-based scaled versions of ResNet50. No such visible patterns could be observed in CLIP’s performance on context-addition for BigEarthNet labels

Adding geospatial priors leads to a marked improvement in zero-shot performance for most of the BLIP variants on EuroSAT.

On BigEarthNet-S2, the addition of context leads to a degradation in performance for most of the BLIP variants.

In summary, our study presents an empirical examination of the zero-shot performance of pre-trained foundation models on standard remote-sensing datasets like EuroSAT and BigEarthNet-S2. Our findings reveal that fine-tuned BLIP variants outperform the standard version on these benchmarks, and incorporation of geospatial context during the inference stage can lead to mixed outcomes depending on the model and dataset selected.

[1] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in ICML, 2021.

[2] C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. V. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in ICML, 2021.

[3] J. Li, D. Li, C. Xiong, and S. C. H. Hoi, “BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in ICML, 2022.

[4] L. Yuan, D. Chen, Y.-L. Chen, N. C. F. Codella, X. Dai, J. Gao, H. Hu, X. Huang, B. Li, C. Li, C. Liu, M. Liu, Z. Liu, Y. Lu, Y. Shi, L. Wang,J. Wang, B. Xiao, Z. Xiao, J. Yang, M. Zeng, L. Zhou, and P. Zhang, “Florence: A new foundation model for computer vision,” ArXiv, vol. abs/2111.11432, 2021.

[5] A. Kirillov, E. Mintun, N. Ravi, et al., “Segment anything,” 2023.

[6] P. Helber, B. Bischke, A. R. Dengel, and D. Borth, “EuroSAT: A novel dataset and deep learning benchmark for land use and land cover classification,” IEEE JSTARS, vol. 12, pp. 2217–2226, 2017.

[7] G. Sumbul, M. Charfuelan, B. Demir, and V. Markl, “BigEarthNet: A large-scale benchmark archive for remote sensing image understanding,” IGARSS, pp. 5901–5904, 2019.

Analyze the zero-shot performance of state-of-the-art foundation models on satellite imagery

Zero-shot Performance of Foundational Models on Satellite Imagery

During residential construction or renovation, the detection of the presence and dimensions of the existing swimming pool is essential. This operation is currently performed by construction or renovation companies through on-site surveys, which can take several hours and cost hundreds of dollars per site. In this article, we intend to explore how solutions deployed in our MLOps platform, GeoEngine, specifically SAM-GEO, can be leveraged to automate this process at scale in a fraction of time and cost

Segment Geospatial or SAM-GEO relies on Language-Segment-Anything (LangSAM), a powerful foundation model that combines the power of instance segmentation and text prompts to generate masks for specific objects in images. It is observed to have decent performance on accurate segmentation of high-resolution satellite imagery.

To analyze the performance of the SAM-GEO model as regards localization and identification of swimming pools, we use the 

 dataset, which consists of 200 4K images and 3980 pool annotations from 8 different neighborhoods in the city of Belo Horizonte, Minas Gerais, Brazil.

We applied SAM-GEO on each of the BH-POOL images to generate pool masks. Figure 3 depicts some of the BH-POOL samples and respective SAM-GEO predictions:

We observe that SAM-GEO manages to identify pools in the BH-POOL dataset exceptionally well, barring a few false positives and negatives.

Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A., Lo, W.Y., Dollar, P., & Girshick, R. (2023). Segment Anything. arXiv:2304.02643.

Discover how the SAM-GEO model automates swimming pool detection and localization in residential areas at scale.

Detecting pools in urban areas using SAM-GEO

Urban planning is undergoing a significant transformation, with a growing emphasis on incorporating green spaces and sustainable elements into urban environments. 

 in urban spaces has emerged as a crucial tool for urban planners and landscape architects to achieve this balance between urbanization and environmental preservation. In this blog, we will explore the importance of tree detection in urban spaces and shed light on the powerful 

 model, which has proven to be an exceptional tool in this domain.

The Significance of Tree Detection in Urban Spaces

 Effective urban planning requires a comprehensive understanding of tree distribution and density within cities. By accurately detecting and mapping trees, urban planners can make informed decisions about the placement of new trees, parks, and green spaces. This not only enhances the aesthetic appeal of urban areas but also contributes to improved air quality, temperature regulation, and overall well-being of city dwellers.

 Tree detection in urban spaces enables environmental assessments that evaluate the health and quality of the urban ecosystem. By analyzing the distribution and characteristics of trees, environmentalists can monitor changes over time and develop strategies for conservation and environmental policies. Trees play a vital role in air purification, carbon sequestration, and mitigating the urban heat island effect, making their detection crucial for assessing the overall environmental impact of urban areas.

 Accurate tree detection relies on advanced models that can process large amounts of data efficiently. Among the various models available, YOLOv5 stands out as a reliable and efficient solution. It is a detection model that excels at identifying and localizing objects in images with remarkable precision and speed. This makes it a perfect fit for tree detection in urban spaces, where large-scale datasets need to be processed quickly.

The YOLOv5 model has gained significant attention in recent years due to its outstanding performance in object detection tasks. Its key features include:

 YOLOv5 strikes an excellent balance between accuracy and inference speed. It achieves high mAP (mean average precision) scores while providing real-time or near real-time predictions, making it ideal for our use case.

 YOLOv5 is built on the popular PyTorch framework, allowing for seamless integration into existing workflows. Its modular architecture facilitates easy customization and fine-tuning, enabling practitioners to adapt the model to specific requirements and datasets.

The dataset we used for training contains 33 UAV 

 images. Each image has a width of 4400 and a height of 6600.

 which is a wrapper library that integrates GeoEngine and MMLab libraries for GeoSpatial ML development.

, we can track the experiments shown below.

Take a look at the following images to see the outcomes of our inference on randomly selected samples.

YOLOv5 by Ultralytics (Version 7.0) [Computer software]. Jocher, G. (2020). https://doi.org/10.5281/zenodo.3908559

Europa: Increasing Accessibility of Geospatial Datasets

. H Shin, N Exe, U Dutta, TR Joshi, S Verma, S Gupta. 

International Geoscience and Remote Sensing Symposium

GeoEngine: A Platform for Production-Ready Geospatial Research

. S Verma, S Gupta, H Shin, A Panigrahi, S Goswami, S Pardeshi, N Exe, .. 

Computer Vision and Pattern Recognition Demo Track

The utilization of YOLOv5, a powerful object detection algorithm, in the field of urban planning.

Granular Blog

Try our platform for free