ArticlePDF Available

Web Usability Policies/Standards/Guidelines Do Not Influence Practices at ARL Academic Libraries. A Review of: Chen, Yu‐Hui, Carol Anne Germain and Huahai Yang. “An Exploration into the Practices of Library Web Usability in ARL Academic Libraries.” Journal of the American Society for Information Science and Technology 60.5 (2009): 953‐68.

Authors:
Shandra Lee Knight at National Jewish Health
Shandra Lee Knight
Download full-text PDF
Read full-text
Download citation

Abstract

Objective – To survey the current status of Web usability Policies/Standards/Guidelines (PSGs) found in academic libraries of the Association of Research Libraries (ARL). Researchers sought to investigate whether PSGs are in place, the levels of difficulty surrounding implementation, the impact of PSGs on design, testing, and resource allocation, and the relationship between ARL ranking and usability practice or PSGs. Design – Survey. Setting – North America. Subjects – Academic libraries of the ARL. Methods – An 18‐question survey consisting of multiple choice, Likert scale, and open‐ended questions was sent to all 113 ARL libraries in November 2007. Survey recipients were selected as the person in charge of Web site usability by visiting library Web sites and phone inquiry. The survey was concluded in January 2008 with a response rate of 74% (84 institutions). The researchers used t‐test to detect any difference in ARL library ranking between libraries with and without PSGs. Pair‐wise t‐tests were conducted to identify gaps in difficulty implementing PSGs. In addition, they used Pearson’s Correlation to investigate any significant correlations between variables such as ARL rank and resource allocation. Main Results – Of the 84 respondents, 34 (40%) have general library Web PSGs and 25 (30%) have specific usability PSGs; 41 (49%) have at least one type of in‐library PSG. Of the 43 (51%) libraries that do not have PSGs, 30 (36%) are at universities with institutional Web usability PSGs; 26 (87%) follow those guidelines. There was no statistically significant relationship between ARL ranking and PSG status (see Table 1). The authors asked about difficulty in implementing PSGs. Of the 32 libraries responding to a question about general library Web PSGs, most had slight or moderate difficulty. Twenty‐three libraries with specific usability PSGs identified difficulty levels; some had no difficulty, but a majority had moderate difficulty. For the 26 libraries using institutional Web usability PSGs, most had no or slight difficulty. Pair‐wise t‐tests showed that library Web usability PSGs were significantly more difficult to implement than university Web usability PSGs. Enforcement/agreement issues were reported as the primary difficulty in implementing in‐library PSGs. Technical issues and ambiguity were obstacles at the institutional level. More than half of the 84 libraries have Web advisory committees and about one third have usability committees or Web usability subcommittees. Several libraries answered that they have none of these committees, but indicated that they have some sort of ad hoc committee or user study group to address usability issues. Of the 84 respondents, 71 (85%) have conducted usability testing. Sixty‐two libraries (73.8%) rated usability testing as important, very important, or extremely important: the rate given for the importance of usability testing did not correlate with ARL ranking. Cited most often in open ended questioningwere the importance of iterative testing, library wide buy‐in, and staff and resource availability. Main web pages were tested most frequently. Fifty‐three libraries (74.6%) tested their lower level pages at least once. OPACs were tested the least often. The amount of testing was impacted neither by the existence of library Web PSGs nor usability PSGs. The top two testing methods were in‐person observation and think aloud protocol.Of the 84 libraries, 24 (28%) reported having staff dedicated to Web usability issues; twenty full‐time staff and four part‐time staff. There was a weak association between ARL ranking and hours worked by dedicated staff; no association existed for regular staff who take on Web responsibilities. Fifty‐one (60%) of libraries had regular staff whose duties included Web usability; forty‐six full‐time and five part‐time. Training did not correlate to amount of testing methods used. There was a weak link between ALR ranking and availability of resources and, the authors showed, more testing was done as resources increased. In response to a query about future Web usability plans, the focus was on usability testing and site redesign, with only three libraries planning to refine or establish usability PSGs. Conclusion – The authors hypothesized that “web usability PSGs would influence usability practice within libraries and other institutions” (953). The data show that PSGs do not influence practices. The authors conclude that there is no significant relationship between PSGs and testing practices or PSGs and the availability of resources. Likewise, ARL ranking had no effect on the establishment of usability PSGs. Most libraries are conducting usability testing, and there was a weak link between ARL ranking and availability of testing resources. Highlighted in the open‐ended questions is the lack of usability expertise among stakeholders. Workload, inadequate human resources, and lack of organizational cohesion are also cited as barriers to the adoption of Web usability PSGs. The authors speculate that Web professionals likely use their own working knowledge and internalized guidelines without having formal documentation. The authors further speculate that the difficulty related to creating mental models that adequately represent library tasks may hinder the use of formal usability PSGs. Additionally, libraries may not regard the lack of usability PSGs as a liability, especially in light of the lack of government mandates or standards. The authors recommend educational efforts for key players on the value of Web usability, support for hiring dedicated staff, and formal documentation to guide design practice. The authors plan to compare the collected PSGs in an upcoming project. Future research could focus on non‐ARL libraries, the relationship between PSGs and user experience, and Content Management System (CMS) usability characteristics.
Available via license: CC BY-NC-SA
Content may be subject to copyright.
EvidenceBasedLibraryandInformationPractice2009,4.4
EvidenceBasedLibraryandInformationPractice
EvidenceSummary
WebUsabilityPolicies/Standards/GuidelinesDoNotInfluencePracticesatARL
AcademicLibraries
AReviewof:
Chen,Yu‐Hui,CarolAnneGermainandHuahaiYang.“AnExplorationintothePracticesof
LibraryWebUsabilityinARLAcademicLibraries.”JournaloftheAmericanSocietyfor
InformationScienceandTechnology60.5(2009):953‐68.
Reviewedby:
ShandraProtzko
InformationSpecialist,
TuckerMedicalLibrary,NationalJewishHealth
Denver,Colorado,UnitedStatesofAmerica
Email:protzkos@njhealth.org
Received:27July2009Accepted:29July2009
©2009Protzko.ThisisanOpenAccessarticledistributedunderthetermsoftheCreativeCommonsAttribution
License(http://creativecommons.org/licenses/by/2.0),whichpermitsunrestricteduse,distribution,and
reproductioninanymedium,providedtheoriginalworkisproperlycited.
Objective–Tosurveythecurrentstatusof
WebusabilityPolicies/Standards/Guidelines
(PSGs)foundinacademiclibrariesofthe
AssociationofResearchLibraries(ARL).
Researcherssoughttoinvestigatewhether
PSGsareinplace,thelevelsofdifficulty
surroundingimplementation,theimpactof
PSGsondesign,testing,andresource
allocation,andtherelationshipbetweenARL
rankingandusabilitypracticeorPSGs.
Design–Survey.
Setting–NorthAmerica.
Subjects–AcademiclibrariesoftheARL.
Methods–An18‐questionsurveyconsisting
ofmultiplechoice,Likertscale,andopen‐
endedquestionswassenttoall113ARL
librariesinNovember2007.Surveyrecipients
wereselectedasthepersoninchargeofWeb
siteusabilitybyvisitinglibraryWebsitesand
phoneinquiry.Thesurveywasconcludedin
January2008witharesponserateof74%(84
institutions).Theresearchersusedt‐testto
detectanydifferenceinARLlibraryranking
betweenlibrarieswithandwithoutPSGs.
Pair‐wiset‐testswereconductedtoidentify
gapsindifficultyimplementingPSGs.In
addition,theyusedPearson’sCorrelationto
investigateanysignificantcorrelations
betweenvariablessuchasARLrankand
resourceallocation.
MainResults–Ofthe84respondents,34
(40%)havegenerallibraryWebPSGsand25
(30%)havespecificusabilityPSGs;41(49%)
haveatleastonetypeofin‐libraryPSG.Ofthe
43(51%)librariesthatdonothavePSGs,30
67
EvidenceBasedLibraryandInformationPractice2009,4.4
(36%)areatuniversitieswithinstitutional
WebusabilityPSGs;26(87%)followthose
guidelines.Therewasnostatistically
significantrelationshipbetweenARLranking
andPSGstatus(seeTable1).
Theauthorsaskedaboutdifficultyin
implementingPSGs.Ofthe32libraries
respondingtoaquestionaboutgenerallibrary
WebPSGs,mosthadslightormoderate
difficulty.Twenty‐threelibrarieswithspecific
usabilityPSGsidentifieddifficultylevels;
somehadnodifficulty,butamajorityhad
moderatedifficulty.Forthe26librariesusing
institutionalWebusabilityPSGs,mosthadno
orslightdifficulty.Pair‐wiset‐testsshowed
thatlibraryWebusabilityPSGswere
significantlymoredifficulttoimplementthan
universityWebusabilityPSGs.
Enforcement/agreementissueswerereported
astheprimarydifficultyinimplementingin‐
libraryPSGs.Technicalissuesandambiguity
wereobstaclesattheinstitutionallevel.More
thanhalfofthe84librarieshaveWebadvisory
committeesandaboutonethirdhaveusability
committeesorWebusabilitysubcommittees.
Severallibrariesansweredthattheyhavenone
ofthesecommittees,butindicatedthatthey
havesomesortofadhoccommitteeoruser
studygrouptoaddressusabilityissues.
Ofthe84respondents,71(85%)have
conductedusabilitytesting.Sixty‐twolibraries
(73.8%)ratedusabilitytestingasimportant,
veryimportant,orextremelyimportant:the
rategivenfortheimportanceofusability
testingdidnotcorrelatewithARLranking.
Citedmostofteninopenendedquestioning
weretheimportanceofiterativetesting,
librarywidebuy‐in,andstaffandresource
availability.Mainwebpagesweretestedmost
Table1
Libraries/universitieswith/withoutPSGs
LibraryWebPSGs
LibraryWebusability
PSGs
UniversityWeb
usabilityPSGs
No.oflibraries(%)No.oflibraries(%)No.oflibraries(%)
With34(40%)25(30%)30(36%)
Without50(60%)58(69%)31(37%)
Notsure 22(26%)
Noanswer 1(1%)1(1%)
Total84(100%)84(100%)84(100%)

frequently.Fifty‐threelibraries(74.6%)tested
theirlowerlevelpagesatleastonce.OPACs
weretestedtheleastoften.Theamountof
testingwasimpactedneitherbytheexistence
oflibraryWebPSGsnorusabilityPSGs.The
toptwotestingmethodswerein‐person
observationandthinkaloudprotocol.
Ofthe84libraries,24(28%)reportedhaving
staffdedicatedtoWebusabilityissues;twenty
full‐timestaffandfourpart‐timestaff.There
wasaweakassociationbetweenARLranking
andhoursworkedbydedicatedstaff;no
associationexistedforregularstaffwhotake
onWebresponsibilities.Fifty‐one(60%)of
librarieshadregularstaffwhoseduties
includedWebusability;forty‐sixfull‐timeand
fivepart‐time.Trainingdidnotcorrelateto
amountoftestingmethodsused.Therewasa
weaklinkbetweenALRrankingand
availabilityofresourcesand,theauthors
showed,moretestingwasdoneasresources
increased.Inresponsetoaqueryaboutfuture
Webusabilityplans,thefocuswasonusability
testingandsiteredesign,withonlythree
librariesplanningtorefineorestablish
usabilityPSGs.
Conclusion–Theauthorshypothesizedthat
“webusabilityPSGswouldinfluenceusability
practicewithinlibrariesandother
institutions”(953).ThedatashowthatPSGs
68
EvidenceBasedLibraryandInformationPractice2009,4.4
69
donotinfluencepractices.Theauthors
concludethatthereisnosignificant
relationshipbetweenPSGsandtesting
practicesorPSGsandtheavailabilityof
resources.Likewise,ARLrankinghadno
effectontheestablishmentofusabilityPSGs.
Mostlibrariesareconductingusabilitytesting,
andtherewasaweaklinkbetweenARL
rankingandavailabilityoftestingresources.
Highlightedintheopen‐endedquestionsis
thelackofusabilityexpertiseamong
stakeholders.Workload,inadequatehuman
resources,andlackoforganizationalcohesion
arealsocitedasbarrierstotheadoptionof
WebusabilityPSGs.Theauthorsspeculate
thatWebprofessionalslikelyusetheirown
workingknowledgeandinternalized
guidelineswithouthavingformal
documentation.Theauthorsfurtherspeculate
thatthedifficultyrelatedtocreatingmental
modelsthatadequatelyrepresentlibrarytasks
mayhindertheuseofformalusabilityPSGs.
Additionally,librariesmaynotregardthelack
ofusabilityPSGsasaliability,especiallyin
lightofthelackofgovernmentmandatesor
standards.Theauthorsrecommend
educationaleffortsforkeyplayersonthe
valueofWebusability,supportforhiring
dedicatedstaff,andformaldocumentationto
guidedesignpractice.Theauthorsplanto
comparethecollectedPSGsinanupcoming
project.Futureresearchcouldfocusonnon‐
ARLlibraries,therelationshipbetweenPSGs
anduserexperience,andContent
ManagementSystem(CMS)usability
characteristics.
Commentary
WebusabilityPSGshelpdevelopersand
administratorscreatesiteswiththeenduser’s
needsinmind.Afocusonusabilitycanhelp
limituserfrustrationandenhancesite
functionalityandeffectiveness.Whilethere
areampledesignguidelinesintheliterature,
theauthorshighlightthelackofstudiesofthe
associationbetweenPSGsandactualpractice.
Efficiencyandfunctionalityareimportant
accessissuesandtheauthorsaskareasonable
questionabouttheimpactofusabilityPSGs,
expandingonthecurrentliterature.
Unfortunately,aspresentedinthesurvey
questionnaire,thedistinctionbetweenaWeb
sitepolicythataddressesusabilityissuesanda
specificusabilitypolicyisambiguous.Without
aclearworkingdefinitionofusability,
participantresponsesmaynotaccurately
reflectactualpractice.
Theauthorsclearlydescribethestudy
population,settinganddatacollection
methods,andtheyaccountforlimitationsof
thestudy,recognizingthatthestudyisnot
generalizable.Sincethesurveywasself‐
report,thevaliditymightbelimited;and
whilethesurveywaslong,addition
informationwouldhavebeenofbenefit.For
example,theauthorssuggestquestioningwhy
somelibrarieshavenoPSGs.The
methodologywasdescribedtoallowfor
replicationandtheresultswereclearly
outlinedandpresentedintables.Thisstudy
offersanindicationofcurrentpracticeinARL
libraries,themostprestigiouslibrariesin
NorthAmerica.
Thereissoundevidenceintheliteratureto
supporttheuseofestablisheddesign
principlesandusabilitytestingpractices.The
authorshighlightResearch‐BasedWebDesign&
UsabilityGuidelines,publishedbytheU.S.
DepartmentofHealthandHumanServices,as
oneauthoritativeresource.However,itisnot
obviousthatjustbecausealibraryhasno
formaldocumentationsupportingusability
effortsthatthoseeffortsarenotbeingmade.
ThatARLstudyparticipantsthinkusability
testingisimportant,conductsomeformof
testing,andthinkimplementingPSGsisat
leastsomewhatdifficult,suggestsaneedto
convincepeoplewhyformalPSGsare
important,ifindeed,theyare.Itseems
intuitivethatwrittenusabilityPSGswould
contributetothedevelopmentofqualityWeb
sites,andfuturestudiesmightindicateas
much.Lackofknowledgeandresourcesseem
tobethebiggestbarriers.Itisworthasking,
“Whatpoliciesorpracticesactuallyimprove
userexperience?”Whileexploratory,this
studyservesasaremindertoalllibraryWeb
EvidenceBasedLibraryandInformationPractice2009,4.4
administratorstoatleastcontinuetoask
questionsaboutusabilitypracticeand
resourceallocation.
WorksCited
UnitedStatesDepartmentofHealthand
HumanServices.Research‐BasedWeb
Design&UsabilityGuidelines.
Washington,DC:U.S.Government
PrintingOffice,2006.
70
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.