Gantavya - A Landmark Recognition System

Abstract

Landmarks are culturally, historically, and architecturally significant places that aid in navigation, foster community identity, and offer insight into a region's heritage and character. However, most people including tourists are unaware of the whereabouts of landmarks and their significance and a lot of time is wasted by people searching for details about the landmark. Landmark Recognition System aims to help people find their destination landmarks, related facts and information and guides them to their destination with its map functionality saving a lot of trouble. We opted to use object detection instead of classification, for which we studied various model architectures and chose to use YOLOv8, a state-of-the-art model at the time. To fine-tune the model, a custom dataset comprising landmarks images captured in person as well as scraped from the internet and labels for each image was created. The chosen model, YOLOv8 utilizes a hybrid backbone comprising CSPDarknet53 and PANet, employing cross-stage partial networks and path aggregation to efficiently detect objects with improved speed and accuracy. The backbone of this model acts as a feature extractor and sends the results of each kernel in the pyramid in the neck for further operations and finally, the head provides the final output of the model. A mobile application was developed by integrating the model with the Django backend that communicated to the React-Native frontend through RESTful API. The application takes an image as input from an authenticated user, predicts the landmark in the image and helps the user navigate to the place using maps. The results of the model developed, using AdamW optimizer and Cross Entropy Loss, when validated were found to be 97.7% box precision with 100% recall and the mAP scores were as 99.5% for mAP50 and 79.9% for mAP50-95 showing that the model developed is robust and accurate.

Full text






!"#$"#
"#%%"
"#
&"%%"'

(")&"
*%"#&%" +"(,
,
"($&%
 ,% -."/)". 0++"&%
1,"% 2#%" 3+",
*++,"%" **#,,4
+"+&


+"$&%"#&%"
   56 5 78 9696  86
:;6<8
6 = > 9   59 96 8
865 8 9  =  8 = 
86;


+"(,,
•9 786 >  56  7  7 6 = 8
: = 8 6  766  7  >  7 56
8759=;86=98868796
<  6 =  59  > 58 
5 866 = 6 ; 8  9
>>69?6?669?6
 96 6= 6? 9 6@6 :
6 = 58 A96 B @7 9
86C;
-

"($&%
•8 5D   5>  9=6 556
 956 58  6> 8 57 = >
6  = 6  5> 6
8;
0

,%
1
 9     >96 5
858@79
 ? 87 >9 9  
=96 8  5> = 8
9C55>;
#&


,%
2
#>75865
669 8  @7 9 
8?8787978
=96=9;
#7


."/)".
3

•.66*
E>25958=958F?
E759788865==
E5=
•%+8%65?(@56?8,99?GD
9@? +9 8> 9? 8 9@? %" +886 &59
H%&I? + 95? (988 95? %" +9687 &59 H%&&I? %"
8586&59H)#I?@89>%65?%6,9
,99
*
."/)".
&66

."/)".
**
6%&66 % 6
%
2-J - 0*
&66
%@9

9,8
<?&5?
6
G<6)65
7,;;
)7@#
+8
"5&
+%
*
+"&%
%,#,%"

%,"%"
•6#@6%
•@6 8 
966? 788 7 
95;
•&  ;K :6 = 8
 8  = 8
5 = 8 @9
@K = 8 @D  ""
=;
•&66@676
5::6;
*
+"&%

,"%"
,6&:9
  "" 6 7 =9 @ 9 = 9 ? 7 8 
:9;
• ;6 : :6 7  8  = 5 6
9@=6?58>6;
* 

*-
,"
%"

%< 0 L0 
6%< 0 L0 
=58 
(8<M 
"5< .
?
,9 ;-22?;3
,"%"
*0
,6
G55

=6M -
+M ***0*-
 *** 0*-
5 2;1)"+
+5M ;0
%= *;
&696M ;*2
+5 ; 
 ; 08
*1
,6)9
96
,"%"

#%"
*2

#%"
*3
,6
96

+",

"">2 6 7  78 D (  78 8 865 =
=96 +%? 9 78  > )?  @6
556 7 >65; 8 556 8 8 =667
=96
•$.98
•#5??#5+:6?&8+7
•+786
•K56>6@6,5
• + 78 6 6  >  5 
6
•>8==99=
•,9=55=?5>568

++,"%"
*

#,,4+"+&

9
%69?95D89=966>65886N
9=6   ; > >
86 6 O7? D?   >? 6 78
58 8A9 98   9 78 "5& 
+8? 7C>   @9 6 5@6 = 96 =
6;(56=86:5B
 6 9 K5? 9  @9  8
N?9?@66;8
 : >  @6@ 6 :
785@:=9;

#,,4+"+&

)99+5
•566@6>5=658@
9@66@65=56=;
•%  8@ 8 9   5> 6
=@96;
•  898 > 65  > 
>6855;
•6855786>+89
K57858>8

G/"#
 