skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Towards Geospatial Knowledge Graph Infused Neuro-Symbolic AI for Remote Sensing Scene Understanding

Conference ·

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
DOE Contract Number:
Resource Relation:
Conference: International Geoscience and Remote Sensing Symposium 2023 (IGARSS) - Pasadena, California, United States of America - 7/16/2023 8:00:00 AM-7/21/2023 8:00:00 AM
Country of Publication:
United States

References (11)

WorldKG: A World-Scale Geographic Knowledge Graph conference October 2021
A Geospatial Ontological Model for Remote Sensing Scene Semantic Knowledge Mining for the Flood Disaster conference July 2018
Knowledge Graph Embedding: A Survey of Approaches and Applications journal December 2017
Semantics-Driven Remote Sensing Scene Understanding Framework for Grounded Spatio-Contextual Scene Descriptions journal January 2021
A survey on visual transfer learning using knowledge graphs journal April 2022
Learning Visual Models Using a Knowledge Graph as a Trainer book January 2021
A Comprehensive Survey of Graph Neural Networks for Knowledge Graphs journal January 2022
Semantics-Enabled Framework for Spatial Image Information Mining of Linked Earth Observation Data journal January 2017
Robust deep alignment network with remote sensing knowledge graph for zero-shot and generalized zero-shot remote sensing image scene classification journal September 2021
Neuro-symbolic approaches in artificial intelligence journal March 2022
Semantics-enabled framework for knowledge discovery from Earth observation data archives journal November 2005