The importance of remote sensing image analysis is ever increasing due to its ability to supply meaningful geographic information that informs local and global problems, such as measuring urban sprawl, mapping vegetation communities, monitoring the impacts of global climate change, and managing natural resources and urban planning. In this process of geo-object extraction, geographic object-based image analysis (GEOBIA) provides a method to identify real-world geographic objects from remotely sensed imagery. GEOBIA uses techniques analogous to those used by humans to perceive and distinguish geo-objects in imagery, usually acquired from satellite or airborne platforms. Experts use domain knowledge and measurement data extracted from remote sensing images for object-based analysis. This signifies a need for human involvement in the form of applying expert knowledge at the time of image object identification. The need for such human intervention acts as a barrier to the automation of GEOBIA processes. In this regard, knowledge representation techniques such as the use of ontologies provide possibilities for modeling expert knowledge in a manner that contributes to the further development of GEOBIA. In this chapter, we will discuss the importance of the human factors in GEOBIA. To this end, we will draw on literature from both GEOBIA and ontology use.