Content uploaded by Roberto Saia
Author content
All content in this area was uploaded by Roberto Saia on Apr 27, 2016
Content may be subject to copyright.
PopularityDoesNotAlwaysMeanTriviality:Introduction
ofPopularityCriteriatoImprovetheAccuracyofa
RecommenderSystem
RobertoSaia*,LudovicoBoratto,SalvatoreCarta
UniversitàdiCagliari,DipartimentodiMatematicaeInformatica,Cagliari,Italy.
*Correspondingauthor.Tel.:+390706758755;email:roberto.saia@unica.it
ManuscriptsubmittedSeptember5,2015;acceptedDecember17,2015.
doi:10.17706/jcp.12.1.1‐9
Abstract:Themaingoalofarecommendersystemistoprovidesuggestions,bypredictingasetofitemsthat
mightinteresttheusers.Inthispaper,wewillfocusontherolethatthepopularityoftheitemscanplayin
therecommendationprocess.Themainideabehindthisworkisthatifanitemwithahighpredictedrating
forauserisverypopular,thisinformationaboutitspopularitycanbeeffectivelyemployedtoselecttheitems
to recommend. Indeed, by merging a high predicted rating with a high popularity, the effectiveness of the
producedrecommendationswouldincreasewithrespecttoacaseinwhichalesspopularitemissuggested.
The proposed strategy aims to employ in the recommendation process new criteria based on the items'
popularity,bymeasuring how much it is preferred by users. Throughapost‐processingapproach,weuse
thismetricto extendoneofthemostperformingstate‐of‐the‐artrecommendationtechniques, i.e., SVD++.
The effectiveness of this hybrid strategy of recommendation has been verified through a series of
experiments,whichshowstrongimprovementsintermsofaccuracyw.r.t.SVD++.
Keywords:Collaborativefiltering,algorithms,metrics.
1. Introduction
In order to provide effective suggestions in terms of the good or services offered by a company, a
recommendersystemplaysan essentialrole,sinceit is able tofiltertheuser preferences, suggesting them
onlytheitemsthatcouldbeinteresting.Theidentificationoftheseitemsisbasedonapredictiontask,than
infersthe interest of a user toward an item notyet evaluated, to deriveif it is worth recommending[1].
Most of the strategies used to generate the recommendations arebasedontheso‐calledCollaborative
Filtering(CF)approach[2]whichisbasedontheassumptionthatusershave similar preferences on an
item if they have already rated otheritemsinasimilarway[3].In recent years, the latentfactormodels
havebeenadoptedinCFapproacheswiththeaimtouncoverlatentcharacteristicsthatexplainthe
observedratings[4].Someofthemostcommonapproachesofthistypearethosethatexploittheneural
networks[5], the LatentDirichletAllocation[6],butespecially,thosethatexploitamodelinducedbythe
factorizationoftheuser‐itemratingsmatrix[7](i.e.,thematrixthatreportstheratingsgiventoitemsby
the users). Among these last approaches, the state of the art is represented by SVD++ [8], the Koren's
versionoftheSingularValueDecomposition(SVD)[9],whichexploitstheso‐calledlatentfactormodeland
presents good performance in terms of accuracy and scalability [7],[10].AlthoughSVD++provides
excellent performance, it does nottakeintoaccountthefactor of popularity of the items that are
Journal of Computers
1
Volume 12, Number 1, January 2017
recommended.Thismightleadtoanunderperformanceofarecommender system,incasethesamescore
is predicted for multiple items. Indeed, the system is not abletodiscriminatethemonthebasisoftheir
popularity, so there is the risk to recommend those unpopular, which are less likely to be preferred by
users.Thepopularityof the items isanaspectthathasbeen widely studied intherecommendersystems
literature.Whiletheirabilitytoidentifyitemsofpotentialinterest to users has been recognized, some
limitations have been highlighted. The most important of these is that the recommendations made
accordingtopopularitycriteria are trivial, anddonotbring considerable benefits neither tousers,norto
thosethatofferthemgoodsorservices.Thishappenswhenusingtheso‐callednonpersonalizedmodel[11],
a naive approach of recommendation that does not take into account the user preferences, because it
always recommends a fixed list with the most popular items, regardless of the target user. On the other
hand,however,recommendinglesspopularitemsaddsnovelty(andalsoserendipity)[12]totheusers,but
itusuallyisamoredifficulttasktoperform.
ThisworkaimsatimprovingtherecommendationsproducedbytheSDV++approach,byconsideringthe
items'popularity.Ourstrategyinvolvesabalanceduseofanindexofitempopularity,whichisbasedonthe
positivefeedbacksoftheusers.Thisindexisthenappliedwithintheboundariesofarecommendationlist,
generated through a state‐of‐the‐art approach based on the latentfactormodel(the so‐called SVD++
approach[8]),insteadofusingtheentiredataset.Thiswayofproceedingallowsustoexploitthepopularity
metricstoperforma finetuning of the recommendations generated byarecommendationstrategyatthe
stateoftheart,whichdoesnottakeintoaccounttheitempopularity,thusreducingthetrivialityofthefinal
result.Thecontributionsofourworkarethefollowing:
• DefinitionoftheDomainPopularityIndex(DPI),ametricabletoevaluate thepreferencesoftheusers
aboutanitem;
• Creationof thePBSVD++algorithm,whichextendsthecapabilitiesofSVD++,addingtoitthecapability
toevaluatetheitempopularity;
• Experimentation on three real‐world datasets, to evaluate the capability of popularity to increase the
numberofeffectiverecommendations,withrespecttoastate‐of‐the‐artapproachthatdoesnotemploy
it.
Intherestofthispaper,wefirstintroducetheliteraturerelatedwiththeproposedstrategy(Section2),
continuingtodefinetheadoptednotationandtheproblemdefinition (Section 3), the implementation
detailsofourproposal(Section4).Finally,wecompletethepaperwith the description of the performed
experiments(Section5),endingwithsomeconcludingremarksandfuturework(Section6).
2. RelatedWork
Thissectionpresentstwoconceptscloselyrelatedwithourwork.
NonpersonalizedModels.The recommender systems based on the so‐called nonpersonalizedmodel
[11],proposetoallusersthesamelistofrecommendations,withouttakingintoaccounttheirpreferences.
Thisstaticapproachisusuallybasedontwoalgorithms,thefirstofthem(TopPop),operatesbysuggesting
themostrateditems(i.e.,thosemostpopular),whilethesecond (MovieAvg), works by suggesting the
highestrateditems(i.e.,thosemostliked).Theexclusiveuseofthenon‐personalizedmodels leadstoward
the absence of two important characteristics that a recommender system should have, i.e., novelty and
serendipity [13]. Novelty occurs when a system is able to recommendunknownitemsthatausermight
have autonomously found, while the serendipity happens when it helps the user to find a surprisingly
interestingitemthatausermightnothaveotherwisefound,orifitisveryhardtofind.
LatentFactorModels.Thetypeofdatawithwhicharecommendationsystemoperatesis typically a
sparsematrixwheretherowsrepresenttheusers,andthecolumnsrepresenttheitems.Theentriesofthis
Journal of Computers
2
Volume 12, Number 1, January 2017
matrix are the interaction between users and items, in the form of ratings or purchases. The aim of a
recommendersystem istoinfer,foreachuser u,arankedlistofitems,and inliteraturemany ofthemare
focused on the rating prediction problem. The most effective strategiesinthisfieldexploittheso‐called
latentfactormodels, but especially, the matrixfactorization techniques [7]. Other CF ranking‐oriented
approachesthatextendthematrixfactorizationtechniques,havebeenrecentlyproposed,andmostofthem
usearankingorientedobjectivefunction,inordertolearnthelatentfactorsofusersanditems[14].SVD++
[8],theKoren'sversionoftheSingularValueDecomposition(SVD) [9],istoday consideredoneof thebest
strategies in terms of accuracy and scalability. In [16]‐[18], the problem of modeling semantically
correlateditemswastackled,buttheauthorsconsideratemporalcorrelationandnottheonebetweenthe
itemsandauserprofile.
3. NotationandProblemDefinition
Themathematicalnotationusedinthiswork,andtheproblemstatement,arerecalledinthefollowing.
3.1. Notation
Wearegivenasetofusers },,{ 1N
uuU …=,asetofitems },,{ 1M
iiI …
=
,andasetVofvaluesusedto
expresstheuserpreferences(e.g.,V=[1,5]orV={like,dislike}).Thesetofallpossiblepreferencesexpressed
by the users is a ternary relation
P
⊆
U
×
I
×
V
. We denote as
P
+
⊆P
the subset of preferences with a
positivevalue(i.e., {( , , ) }
P
uiv Pvvvlike
+=∈≥∨=
),where
v
indicatesthemean value(intheprevious
example,
v=3
). Moreover, we denote as {(,,)}IiIuivP
+
+
=∈∃ ∈ the set of items for which there is a
positive preference, and as ,(,,) , ,
iu
np u i v P i I u U
+
=∈∈∀∈
the number of positive preferences
expressedby allusersuforanitemi.Wealsodenoteas {(,,) }
u
I
iI uiv PuU=∈∃ ∈∧∈ thesetofitemsin
theprofileofauser u, and as
R
u
={u∈U
∧
R
⊆
I}
,thesetofitemsirecommendedtoauser u. The set of
itemsiwithouttheitemsalreadyevaluatedbytheuseru(i.e.,thosein
I
u
)isdenotedas
ˆ
I
u
⊆I
.
3.2. ProblemDefinition
Weconsiderthe function
f
:U×
I
→
V
,adopted to predict the ratings for the notevaluateditemswith
theSVD++recommendersystem.Ouraimistodefine,foreachitem,aDomainPopularityIndexDPI(i)that
represents the popularity of the item with respect to the othersinthedataset(intermsofpositive
evaluationsgivenbytheuserstoit).TheDomainPopularityIndexDPIofanitemnotevaluatedbyauser
willbeemployedtobuildascore
α
.Ourobjectiveistogeneratealistofrecommendeditem
i
*
suchthat:
*
ˆ
argmax ( , )
u
jI
ifuj
α
∈
=
+(1)
4. IntegratingPopularityintheRecommendationProcess:Algorithm
In this section, we present the steps made to generate the recommendations based on the proposed
Popularity‐basedSVD++(PBSVD++)strategy, startingfromthedefinitionof thepopularityindexemployed
bytheapproach,andendingwiththeimplementationofournovelalgorithm.
4.1. ItemsPopularityDefinition
Inthissection,we introduceandformalizethepopularityindexemployedofourapproach.Thevalueof
the Domain Popularity Index (DPI) for an item iI
∈
, with [0,1]DPI
∈
,
npi,U
represents the number of
positivepreferencesexpressedbyallusersUfortheitemi.Itiscalculatedasshowninequation2.
Journal of Computers
3
Volume 12, Number 1, January 2017
,
,
(, ) iU
jU
jI
np
DPI i U np
∈
=∑(2)
DPIisanimportantindicator,becauseitprovidesaglobalmeasure ofthepreferencesexpressedforan
itembyallusers.
4.2. PBSVD++Algorithm
WeexploittheDPIindexpreviouslypresented,inordertomodifytheresultoftheSVD++approach.The
indexisemployedinAlgorithm1,wherewebuildavalue
α
.Givenasetofrecommendations
R
u
,addressed
toauser
u∈U
,the final rating
ρ
i,u assignedtoeach item
i
∈
R
u
byour algorithm, is composedbythe
rating
i,u
calculated through the SVD++ approach, normalized in a continuousrangefrom0to1,and
denotedasSTD(i,u),addedtothe
α
(alsonormalizedinacontinuousrangefrom0to1),builtbyemploying
theDPIindex,asshowninEquation3.Thefinalratingassignedtoanitemisthenintherangefrom0to2.
,
,
,
(, )
rating (, )
with ( , ) and
rating ( , )
iu
iu
ju
jU jU
STD i u
DPI I U
STD i U DPI j U
ρ
α
α
∈∈
=+
==
∑∑
(3)
Thenewrating
ρ
i,u,assignedtoanitemiforauserutakes into account,inabalancedway,itsdomain
popularity, and this produces a substantial change in the canonical SVD++ ranking during the
recommendationprocess,changingtheperformanceoftherecommendersystem.Algorithm1implements
theoperationsdescribedabove.Ittakesasinputthetraining sets(usedby theSVD++approach,in step3,
tobuildthelatentfactormodel),theuseruto whom address therecommendations,andthenumber nof
these.
Algorithm1.PBSVD++
Input:s=Trainingset,u=User,n=Recommendations
Output:L=Listofnrecommendations
1. procedureGETPBSVD
R
ECS(
s
,u,n)
2. x=GetNumOfNotEvaluatedItems(u)
3. I=GetSvdRecs(s,u,x)
4.
t
=0
5. foreachiinIdo
6. if(SvdRating(i)+1)>SvdRating(i0)then
7. Ri
←
8. t+=GetD P I(i )
9. endi
f
10. endfor
11. foreachrinRdo
12. rating=(SvdRating(r)/SumAllSvdRatings(R))
13. α=GetDPI(r)/t
14. SetNewRating(r,rating+ α)
15. endfor
16. L=GetRecsDescOrdered(R,n)
17. ReturnL
18. endprocedure
After the number x of potential items to recommend to the user uhasbeenobtained(step2),we
calculatethrough thestandardSVD++approach,fortheuser u,aset Iofx recommendationsbasedon the
training set s (step 3). In the steps from 5 to 10, we select from I only the elements i that are possible
Journal of Computers
4
Volume 12, Number 1, January 2017
candidates for the recommendations based on the proposed approach. They are those items in which a
modificationof thescore,by addingtotheoriginalratingofSVD++thevalueof
α
(parametercalculated
inthestep 13,whosevalueisin the rangefrom0 to 1),couldaltertherank proposed bySVD++.Forthis
reason,thecandidatesareonlytheitemstowhich,addingatmost1,wegetavaluehigherthanthatofthe
itemwiththemaximumSVD++ score (i.e., the first element
i0
).Weuse this process also to calculate(in
step8)thesumoftheDPIweight,relatedtoalltheitems
i
∈
I
.StartingwiththissetRofcandidateitems,
inthestepsfrom11to17,wealtertheSVD++scoreofeachitem
i
∈
I
,following equation3,afterwhich
wereturnalistLofnrecommendations,composedbytheitemswiththehighestscores.
5. Experiments
Inthissection,afterthedefinition of the experimental environment and of the adopted datasets'
characteristics,wedescribethestrategyandmetricsused,concludingwiththepresentationanddiscussion
oftheexperimentalresults.
5.1. ExperimentalSetup
The environment for this work isbasedontheJavalanguage,withthesupportoftheApacheMahout
(https://mahout.apache.org)Java frameworktoimplementthestate‐of‐the‐artapproachthatwecompare
ournovelapproachwith.Inordertoevaluatetheproposedstrategy,weperformaseriesofexperimentson
three different real‐world datasets, which represent a quite standard benchmark in the context of the
recommender systems: the first one is the dataset Yahoo! Webscope R4
(http://webscope.sandbox.yahoo.com),whichcontainsalargeamountofdatarelatedtouserspreferences
expressedbytheYahoo!Moviescommunity;theotherstwoareextractedfromthedatasetMovielens10M
(http://grouplens.org/datasets/movielens/),composedbythedatacollected overvariousperiodsoftime,
ontheMovieLenswebsite.Thefirstsetofexperimentsprovidesageneraloverviewoftheresultsobtained
bycomparingtheperformance ofarecommendersystem,wherewehaveimplementedthenewPBSVD++
algorithm,with thoseofacanonicalsystembasedonthe SVD++algorithm.Thesecondsetofexperiments
showsinmoredetailthe resultspreviouslysummarized,analyzing them through theprecisionandrecall
metrics.
5.2. Datasets
In order to evaluate the proposed strategy, we perform a series of experiments on three different
real‐world datasets, extracted bytwoquitestandardbenchmarks in the context of the recommender
systems:Yahoo!WebscopeR4andMovielens10M.
Yahoo!Webscope(R4). This dataset contains a large amount of data related to users preferences
expressedontheYahoo!Moviescommunitythatareratedonthebaseoftwodifferentscales,from1to13
andfrom1to5 (weusethelatter).Thetrainingdataiscomposedby 7,642users,11,915movies/items,
and211,231ratings.Alltheusersinthetrainingsethaveratedatleast10itemsandallitemsareratedby
atleastone user.Thetestdata iscomposedby 2,309users,2,380items, and10,136ratings.Thereareno
testusers/itemsthatdonotalsoappearinthetrainingdata.Alltheusersinthetestsethaveratedatleast
oneitemandallitemshavebeenratedbyatleast oneuser.Theitemsare classifiedin20differentclasses
(genres),anditshouldbenotedthatanitemmaybeclassifiedwithmultipleclasses.
Movielens10M. The second dataset used in this work is composed by 71,567 users, 10,681
movies/items, and 10,000,054 ratings. It was extracted at random from MovieLens (a movie
recommendation website). All the users in the dataset had rated at least 20 movies, and each user is
representedby auniqueID.Theratingsoftheitemsare basedona5starscale,withhalfstarincrements.
Inthisdatasettheitemsareclassifiedin18differentclasses(moviegenres),andalsointhiscaseeachitem
Journal of Computers
5
Volume 12, Number 1, January 2017
maybeclassifiedwithmultipleclasses(genres).SincetheMovielens 10M dataset does not contain any
textualdescriptionofthe items, to obtainthisinformationweused a file provided by theWebscope(R4)
dataset,which containsamappingfromthe movieIDs usedinthe datasettothe correspondingmovieIDs
andtitlesusedintheMovieLensdataset.UsingthescriptprovidedwiththeMovielens10Mdataset,wesplit
upthewholedatasetintwodifferentdatasetswithexactly10ratingsperuserinthetestset.Bothtraining
setsare composedby69,878users, and9,301,274ratings,with10,667movies/itemsin thefirstone, and
10,676movies/itemsinthesecondone.Eachtestdatasetcontains69,878users,and698,780ratings,with
3,326 movies/items in the first one, and 5,724 movies/items in thesecondone.Fromeachofthese
datasets,wetakeinaccountasubsetof20,000users.
5.3. Strategy
We compare the proposed recommendation strategy with the state‐of‐the‐artapproachSVD++.The
Mahout framework, used to implement it, in addition to the training set requires two additional
parameters: the number of target features and the number of training steps to run. The first parameter
wouldbeequivalenttothenumberofinvolvedgenres,thuswehave set this value to 20 for the Yahoo
dataset, and to 18 for the Movielens datasets. Regarding the second parameter, we use the value 15, as
indicatedintheSVD++referencepaper[8].
5.4. Metrics
Here,wepresentthemetricsusedduringtheexperiments.
PrecisionandRecall.Theperformance measuresadoptedtoevaluateour approach,comparingtheset
ofrecommendations generatedbyourstrategyandthesetof thosegeneratebythecanonicalapproachof
recommendationwith therealuserpreferencesstored inthetestset, aretheprecisionandtherecall,and
metrics[15].GiventwosetsXuandZu,whereXu denotesthesetofrecommendationsperformedforauser
u, and Zu the set of the real choices of the user u in the test set, these metrics are defined as shown in
Equation4.
precision( , )
recall( , )
uu
uu
u
uu
uu
u
Z
X
XZ X
ZX
XZ Z
=
=
∩
∩(4)
MetricsEvaluation.Inordertocomparetheresultsofthetwoapproachesofrecommendation(i.e.,our
approachbasedonthePBSVD++algorithm,andthecanonicalnone, based on SVD++), we calculate the
previous metrics, presented in Equation 4, for each group of n performed recommendations (denoted as
@n,withn={2,4,…,20}),subtractingfromthevaluesobtainedbyourapproachthoseobtainedbySVD++.
Inthisway,apositivevaluedenotesthatourapproachimprovesthestandardone,whileanegativevalue
denotesthatourapproachworsensthestandardone.DenotingasXnthesetofn recommendations
generatedbyourstrategy,asYnthesetofnrecommendationsgeneratedbythecanonicalSVD++strategy,
andasZnthesetofnrealuserpreferencesstoredinthetestset,wedefinethemeasuresshowninEquation
5.
variation@ precision@ ( , ) precision@ ( , )
variation@ recall@ ( , ) recall@ ( , )
nn nn
nn nn
p
nnXZ nYZ
rnnXZnYZ
−= −
−= − (5)
5.5. ExperimentalResults
Journal of Computers
6
Volume 12, Number 1, January 2017
Here,wereporttheresultsoftheexperimentspresentedinSection5.1.
PerformanceOverviewandDetails.TheresultpresentedinFig.1showsthegeneralperformanceof
the proposed strategy, in the context of the three considered real‐world datasets. It indicates the
percentageoftimesinwhichwehavedonebetter,orhavedoneworsethanSVD++(respectively,BandW).
Theoverallresultsshowthegoodperformanceofourapproachwithallthreedatasets.
Fig.1.Resultsoverview.
Inthesecondsetofexperimentswecomparetheperformanceofarecommendersystemwherewehave
implementedthePBSVD++algorithm,withthoseofthecanonicrecommendersystem basedontheSVD++
algorithm.Weevaluatetheresultsintermsofpvariation@nandrvariation@n,asdescribedinSection5.4.
AswecanobserveinthegraphsinFig.2,theresultsarequitesimilarforallthethreeconsidereddatasets,
apartintherecall@n measure,whichreportsadifference between the YahooandMovielensresults.This
happensbecausethescriptprovidedbyMovielensplacesafixednumberofratingsperuserinthetestset
(10).ThisdoesnothappenintheYahoodataset,whichbuildsatestsetwithavariablenumberofitemsfor
eachuser.Sincetherecall@nmetrichasasdenominatorthenumberofitemsinthetestset,thisnumberis
fixedfortheMovielensdataset(hence,theresultsaremore“flat”),andvariablefortheYahoodataset(this
leadstothevariableresultsinFig.2(b)).
Theoverallresults, presented in Fig. 2,showthatourstrategyoutperforms the canonical one, usingall
metrics, except when we test the maximum number of recommendations (i.e., 20). This is an obvious
aspect,sincethealgorithmPBSVD++operatesinthedomainoftheSVD++recommendations,recalculating
their ratings: therefore, when we consider the entire domain, the results of SVD++, and PBSVD++, will
alwaysbeidentical.
(a) (b)
Fig.2.Experimentalresults.
Discussion.Theperformed experiments, presented in Section 5,provethat ourstrategy,basedonthe
novelPBSVD++algorithm,isabletoimprovetheresultsofacanonicalrecommendersystem basedonthe
SVD++algorithm.Aswecanobserve,thishappenswithanynumberof recommendations,exceptthecase
inwhichthemaximumnumberoftheseisgenerated,fortheobvious reason explained in the previous
section.Whenevaluatingtheseresults,wecanobservethatthemaximumvalueofpositivevariationfora
metricis1(whichrepresentsa100%improvementw.r.t.SVD++).Therefore,ourresultssuggestimportant
improvements,thinkingthattheNetflixprizewasbasedona10%improvementintermsofaccuracy.This
Journal of Computers
7
Volume 12, Number 1, January 2017
provesthatispossibletoimproveastate‐of‐the‐artapproachsuchasSVD++,byusingitsoutputasaninput
domain,inordertoperformafinetuningbasedonthepopularityoftheinvolveditems.
6. ConclusionsandFutureWork
Inthispaperweproposedanovelformofrecommendation,whichintegratedtheinformationaboutitem
popularityintoastate‐of‐the‐artapproach.Theperformedexperimentshaveshownboththevalidityofthe
adoptedindex,and itsabilitytoimprovethe performanceoftheSVD++approach. Infuturework, wewill
extendourapproach,byaddingnewmetricsabletoevaluatetheitempopularity,inthecontextofsystems
thatoperate within more than one domain ofgoods/services,trying to parameterize both the popularity
aspectofeachitem,andtheirinterconnectionsbetweendifferentoperativedomains.Wewillalsostudythe
introductionofothersmetricsofpopularity,e.g.,basedonthegeographicordemographicinformation.
Acknowledgment
ThisworkispartiallyfundedbyRegioneSardegnaunderprojectSocialGlue,throughPIA‐Pacchetti
Integrati di Agevolazione “Industria Artigianato e Servizi” (annualità 2010), and by MIUR PRIN 2010‐11
underproject“SecurityHorizons”.
References
[1] Ricci,F.F.,Rokach,L.,&Shapira,B.(2011).Introductiontorecommendersystemshandbook.InF.Ricci,
L.Rokach,B.Shapira,&P.B.Kantor.(Eds.),RecommenderSystemsHandbook(pp.1–35).Springer.
[2] Karypis, G. (November 5‐10, 2001). Evaluation of item‐based top‐n recommendation algorithms.
Proceedingsofthe2001ACMCIKMInternationalConferenceonInformationandKnowledge
Management(pp.247–254).Atlanta,Georgia.
[3] Su,X.&Khoshgoftaar,T.M.Asurveyofcollaborativefiltering techniques. Adv.ArtificialIntelligence,
2009.
[4] Koren,Y.,&Bell,R.M.(2011).Advancesincollaborativefiltering.InF.Ricci,L.Rokach,B.Shapira,&P.
B.Kantor.(Eds.),RecommenderSystemsHandbook(pp.145–186).Springer.
[5] Georgiev, K., & Nakov, P. (2013). A non‐iid framework for collaborative filtering with restricted
boltzmann machines. Proceedingsofthe30thInternationalConferenceonMachineLearning: Vol.28
(pp.1148–1156).
[6] Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latentdirichlet allocation. JournalofMachineLearning
Research,3,993–1022.
[7] Koren,Y.,Bell,R.M.,&Volinsky,C.(2009).Matrixfactorizationtechniquesforrecommendersystems.
IEEEComputer,42(8),30–37.
[8] Koren,Y.(2008). Factorization meetstheneighborhood:Amultifacetedcollaborative filtering model.
Proceedingsofthe14thACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,
LasVegas,Nevada,USA.
[9] Billsus, D., & Pazzani, M. J. (1998). Learning collaborative information filters. Proceedingsofthe
FifteenthInternationalConferenceonMachineLearning(pp.46–54).
[10] Bennett,J.,Elkan,C.,Liu,B.,Smyth,P.,&Tikk,D.(2007).Kddcupandworkshop2007.SIGKDDExplor.
Newsl.,9(2),51–52.
[11] Cremonesi, P., Koren, Y., & Turrin, R. (2010). Performance of recommender algorithms on top‐n
recommendationtasks.Proceedingsofthe2010ACMConferenceonRecommenderSystems(pp.39–46).
[12] Ge, M., Delgado‐Battenfeld, C., & Jannach, D. (2010). Beyond accuracy: Evaluating recommender
systems by coverage and serendipity. Proceedingsofthe2010ACMConferenceonRecommender
Journal of Computers
8
Volume 12, Number 1, January 2017
Systems(pp.257–260).
[13] Iaquinta, L., de Gemmis, M., Lops, P., Semeraro, G., Filannino, M., & Molino, P. (2008). Introducing
serendipity in a content‐based recommender system.ProceedingsofInternationalConferenceon
HybridIntelligentSystems(pp.168–173).
[14] Koren, Y., & Sill, J. (2011). Ordrec: An ordinal model for predicting personalized item rating
distributions.Proceedingsofthe2011ACMConferenceonRecommenderSystems(pp.117–124).
[15] Baeza‐Yates, R. A., & Ribeiro‐Neto, B. (1999). ModernInformationRetrieval. Boston, MA, USA:
Addison‐WesleyLongmanPublishingCo.,Inc.
[16] Stilo,G., &Velardi,P.(2009).Time makessense:Eventdiscoveryintwitterusingtemporalsimilarity.
Proceedingsofthe2014IEEE/WIC/ACMInternationalJointConferencesonWebIntelligence(WI)and
IntelligentAgentTechnologies:Vol.2(pp.186–193).
[17] Stilo, G., & Velardi, P. (2014). Temporal semantics:Time‐varyinghashtagsenseclustering.Knowledge
EngineeringandKnowledgeManagement,8876,563–578.
[18] Stilo,G.,&Velardi, P.(2015).Efficient temporalminingofmicroblogtextsanditsapplicationtoevent
discovery.DataMiningandKnowledgeDiscovery.
RobertoSaiaisaPh.D.attheDepartmentofMathematicsandComputerScience of the
UniversityofCagliari.Hegotamasterdegreeincomputerscienceatthesameuniversity.His
currentresearchactivityisfocusedonthedevelopmentoftechniquesandalgorithmsableto
improvetheeffectivenessoftheuserprofilinganditemrecommendation.
LudovicoBorattoisaresearchassistantattheUniversityofCagliari,Italy.Hegraduatedwith
fullmarksandhonorandreceivedhisPhDin2012atthesameuniversity. His research
focusesmainlyonrecommendersystemsanddatamininginsocialnetworks.
SalvatoreCartareceivedaPhDinelectronicsandcomputersciencefromtheUniversity of
Cagliariin2003.Heisassistantprofessorincomputerscienceat the Universityof Cagliari
since 2005. Recently, he has focused on topics related to the social Web, ubiquitous
computingandcomputationalsocieties.
Journal of Computers
9
Volume 12, Number 1, January 2017
