ArticlePDF Available

Evaluation of End-user Satisfaction among Employees Participating in a Web-based Needs Assessment Inventory


Abstract and Figures

This paper reports on the end-user satisfaction evaluation of ENAI. It highlights the positive impact and effect of online web- based management system for counseling inventory. Participants have reported satisfaction with ENAI and the service impact within community of users is complementary. Improvement of ENAI can be made to enhance the end-user satisfaction by referring to the user's requirements.
Content may be subject to copyright.
P r o c e d i a - S o c i a l a n d B e h a v i o r a l S c i e n c e s 9 7 ( 2 0 1 3 ) 7 6 6 7 7 0
1877-0428 © 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
Selection and/or peer-review under responsibility of the Universiti Malaysia Sarawak.
doi: 10.1016/j.sbspro.2013.10.299
The 9t
onference on
nitive Scienc
f end-
ser satisfaction among employees participating
in a
-based needs assessment inventor
an Norizan
an Hashi
Mohamad Izzat Syafi
Faculty of Cognitive Sciences & Human Development, University Malaysia Sarawak, 94300, Kota Samarahan, Sarawa
is paper reports on the end-
ser satisfaction evaluation of ENAI. It high
ights the positive impact and effect of
based management system for counseling inventor
. Participants have reported satisfaction with ENAI and the service impact
within community of users is complementary. Improvement of ENAI can be made to e
e e
ser satisfaction b
© 2
13 Th
rs. Published b
r Ltd.
Selection and/or pee
eview under responsibility of
he Universiti Malaysia Sarawak
Keywords: Online Counseli ng;
application; usable; user satisfactio
1. Intr
The growth of new technologies has made it possible for the online delivery of psychological support [1]. The
ncluding counselor,
psychologist, psychiatrist, etc. Furthermore, it is predicted that the increasing growth and prevalence as well as the
sers demand will put technology as an essential aspect of healthcare into the future [2]. Many types Internet -
ered support is increasingly being used including online counseling management system in organization [3]
Employer has a responsibility to develop appropriate interventions to respond to the health needs of their
employees. Through the organization of cou
eling services, it is must take into account the life style problems with
assumptions if these problems can be solved to improve performance [4]. Furthermore, The World Health
Organization (WHO) emphasizes that most major life style problem leading to th
nability to work is depressio
[5]. Depression includes mixing a variety of feelings and behaviors can lead to loss of self-confidence
. When a
person is suffering from depression, he/she may lose interest in work and stay away from the family. Stud
problems experienced by employees in organizations can influence their performance and job satisfaction, and
hanges or treatment ca
be made. Accordingly, the use of ICT in healthcare personnel is seen as a highe
demanding for innovative methods and new approaches in counseling services.
Corresponding author. Tel.: +6-082-581508;fax: +6-082-581567.
mail add
Avai lable onli ne a t
© 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
Selection and/or peer-review under responsibility of the Universiti Malaysia Sarawak.
Wan Norizan Wan Hashim et al. / Procedia - Social and Behavioral Sciences 97 ( 2013 ) 766 – 770
Therefore, the focus of this paper is the delivery of online web-based management system for counseling
counseling intervention can be researched.
2. System Background
A system called Employee Needs Assessment Inventory (ENAI) has been developed using open source web-
based technology that can be accessed through a variety of platforms and it has a usable interface where it will not
requires a long training to learn how to use the system. The system provide useful information to the employees,
counselors and employer where it is able to provide a counseling service which is comprehensive and effective to
these three tiers. ENAI is a management system which consists of three tiers counseling programs that allow
workers, counselors and employers to access services related to personal life challenges and improves performance
and production at work. Initially, ENAI was established in 2002 and was used manually using the typed and printed
questionnaires, distributed to the respondents. In 2007, the system was upgraded to local hosting but could only be
accessed in the computer. Subsequently, in 2011 the system was transformed to a web-based application.
ENAI provides a solution counseling awareness that prevention is better than cure to government agencies and
organizations in Malaysia. This application has the ability to analyze the needs of the employee according to the
employee profile that includes life style problems. Employee life style problems can be divided into two categories:
personal problems and work problems [7]. Personal problems are health, finances, family and work issues including
spirituality and work, career, interpersonal relationships and work environment.
ENAI has its own unique characteristics which give uniqueness to the system. Figure 1 illustrates the differences
between ENAI and other online counselling system. ENAI provides facilities for user especially employees to share
their life problems where they are able to key in all data problems that they experienced. The data will be sent to the
counsellor and administrator. These data will be treated as private and confidential by the counsellor and this system
can only be achieved by the counsellor and the coordinator only which means that other employees or employers do
not have access to relevant information with colleagues. Problem of missing information or data loss can be avoided
as ENAI is integrated with a large database to store all information.
Figure 1. ENAI vs other online Counseling System
768 Wan Norizan Wan Hashim et al. / Procedia - Social and Behavioral Sciences 97 ( 2013 ) 766 – 770
life problems because all the information has been captured and stored in the system. The information sharing
process between counsellors and employers and also with other coordinator can be done quickly. Any information
provided by the employee will be filtered by the system whether the employee is eligible for intervention services or
not. The system also will indicate to the counsellor and administrator the employee decision whether he/she wants to
seek further intervention and treatment. The system also can automatically analyze all data by changing the form of
the percentage of employees who are suffering in life problem and need further intervention and treatment. There
are many methods of intervention that can be done such as an individual counselling workshop, group counselling
workshops, awareness programs and policies to transform the company.
The following figures are the two screenshots from ENAI system which Figure 2 (a) is a interface for user login
and Figure 2(b) is a interface for main menu of ENAI.
Figure 2. (a) Login screen Figure 2. (b) Main menu screen
Figure 3 shows an example of generated report in a bar graph captured from data that has been keyed in by
employees using ENAI. From this report, counselor and employer can identify the majority personal life problems
occurred among employees and provide further treatment or prevention.
Wan Norizan Wan Hashim et al. / Procedia - Social and Behavioral Sciences 97 ( 2013 ) 766 – 770
3. Methodology
The aim of this study was to evaluate end-user satisfaction with a well built prototype of ENAI. Evaluation was
done in order to obtain information about user acceptance and satisfaction of the system functional, ability in
management effectively and efficiency. The evaluation was conducted in February, 2012 April, 2012 and all
employees from Royal Malaysian Army Organization were invited to participate in this.
ENAI consisted of three major components: 1) a work and personal related problem assessment questionnaire, 2)
user satisfaction questionnaire, and 3) intervention counseling services feedbacks consisting of a personal health risk
profile and lifestyle behavioral through a web. Participants were asked to assess their own health and lifestyle and
then the data will be processed by computer to generate the intervention feedback. The feedback was provided to the
user immediately after completion of the health and lifestyles assessment questionnaire. The part of evaluation
questionnaire related to satisfaction with the system which is the focus of this study consisted of 36 items and it was
categorized into six major categories. Participants were asked to appraise: 1) system usability, 2) ease of use, 3) easy
to learn, 4) user satisfaction, 5) system capabilities, and 6) overall system performance. Participants were asked to
assess their satisfaction on a 5-point Likert scale (Very Disagree, Disagree, Neutral, Agree, Very Agree). For each
category, a free-text field was available for additional comments. There was also a free-text field at the end of the
questionnaire for user to remark their positive and negative aspects of ENAI. More details on the satisfaction
questionnaire can be found in [8],[9],[10].
4. Satisfaction with ENAI
Descriptive statistics were performed on all data to examine user satisfaction with ENAI. A total of 220
employees have participated in the study and there was a lot of feedback on the system design from participants.
Figure 4. Overall User satisfaction results
Figure 4 shows the percentage of participants that judged positively about ENAI and the percentage is more than
60% for each category. For the first category, the usability of the system is high as 64% participants positively
agreed on the effectiveness and efficiency including time saving. On the other hand, both category ease of use and
category easy to learn shows that the system was not hard to use as it was effective in fulfil
requirements. Because of the purpose of use is clear, the system design to such an extent that it would be easy to
understand. 68% of participants indicated that they were positively satisfied with ENAI and would definitely
recommend ENAI to others. In general, participants were positively satisfied about ENAI application as part of their
support system and useful in order to improve performance. It is also can be a channel to detect and identify their
life problems so that they can be helped. Additional analyses showed that fewer participants who made one or more
remarks in the free- text field for each category were satisfied overall with ENAI.
The free-text field at the end of the questionnaire was also been analyzed and showed that 97% participants put
remarks for positive and negative aspects field which includes neutral comments. From the positive aspects,
participants remarked their satisfaction on the system usability and capabilities. However, although participants
770 Wan Norizan Wan Hashim et al. / Procedia - Social and Behavioral Sciences 97 ( 2013 ) 766 – 770
generally satisfied with ENAI, minor input from free-text field contained predominantly critical remarks. From the
ENAI. In addition to obtain user trust, it is necessary to think separately about the design and the ease of case.
Participants were doubted about the confidentiality of the information given by them. Furthermore, there was a lack
of confidence among participants in the system security. With fewer participants who made remarks for positive and
This study showed that in general end-user satisfaction with web-based ENAI was high but there still are
opportunities to increase satisfaction. The results from the satisfaction evaluation had proven that ENAI could be
able to help employer and counselor to encounter and to manage personal life issues among employees. It is also can
be used to access information which is can save time and energy, the information sharing can be done very quickly,
the confidentiality of information can be ensured and data loss can be avoided.
5. Conclusion
In general, participants were satisfied with ENAI. This project has identified the themes of personal life
counseling management in the state public and private sectors in Malaysia. In this project, ENAI has been
developed to strengthen the existing system and has gone through the formative and summative evaluations. The
results of ENAI evaluation has proven that the system could be able to provide convenience to the three tiers
(employer, employees and counselor) in the counseling management of personal life in public and private
organization in Malaysia. The results of the system evaluation for original functions outside the system design are
high under the system that has a clear intended purpose. It is also important to make someone feel less inhibited by
increasing security and improving the system usability. In future, it is planning to improve the user interface and to
increase confidentiality and security.
This project is supported by University Malaysia Sarawak (UNIMAS).
[1] Ritterband, L. M., Gonder-Frederick, L.A., Cox, D.J., Clifton, A.D., West, R.W., Borowitz, S.M. (2003).Internet
Interventions: In Review, In Use, and Into the Future. Professional Psychology: Research and Practice, 34(5), 527 534
[2] Yellowlees P, Brooks PN. Health online: The future isn't what it used to be. The Medical Journal of Australia(1999) 171:525 529
[3] Efstathiou, G., Kalantzi-Azizi, A. (2005, 8 - 11 June 2005). Students' psychological web-counselling:
A European perspective. Paper presented at the Fedora Psyche Conference, Groningen, Netherlands.
[4] Kanter, R. (1977). Work and family in the United States: A critical review and agenda for research and policy. New York: Sage
[5] g Model: How to be an Invited Guest
at Every Table. Journal of Workplace Behavioural Health, Vol. 20, No. ¾, pp. 219-242.
[6] Googins, B. & Davidson, B.N (1993). The Organization as Client: Broadening the Concept of Employee Assistance Programs. Journal
of Social Work, Vol. 38 (4).
[7] Sharulazman Mohd Ghazali (2002). Employee Assistance Program (EAP) Needs: An Exploratory Study at Seven Branches of Bank
Utama (M) Berhad Projek Tahun Akhir Yang Tidak Diterbitkan. Kota Samarahan: University Malaysia Sarawak
[8] Lund, A.M. (2001) Measuring Usability with the USE Questionnaire. STC Usability SIG Newsletter, 8:2.
[9] Chin, J.P., Diehl,V.A., Norman, K. L. (1988) Development of an Instrument Measuring User Satisfaction of the Human-Computer
Interface. ACM CHI'88 Proceedings, 213-218.
[10] Lewis, J.R. (1995) IBM Computer Usability Satisfaction Questionnaires: Psychometrics Evaluation and Instructions for
use. International Journal of Human-Computer Interaction, 7:1, 57-78.
Full-text available
Evaluation is a way to measure the usability of a computer-based system. Usability is the ability to carry out tasks effectively, efficiently and with satisfaction. Usability is the extent to which users can use a computer system to achieve specified goals effectively and efficiently while promoting feelings of satisfaction in a given context of use. It can be broadly defined as " ease of use " , and " usefulness ". This study aims to evaluate the usability, of the Desktop Virtual Reality Prototype (DVRP) courseware in enhancing knowledge on drug abuse among student teachers in a Teacher Education Institution. A total of 35 student-teachers were identified through simple random method to evaluate the usability of DVRP. In order to assess the usability of the DVRP courseware, the USE Questionnaire (Lund, 2001) was employed. USE stands for Usefulness, Satisfaction, and Ease of use. The results of this study on the usability show that the uses are very positive using the DVRP courseware and very effective and efficient to be used to enhance knowledge on drug abuse.
Full-text available
This study is a part of a research effort to develop the Questionnaire for User Interface Satisfaction (QUIS). Participants, 150 PC user group members, rated familiar software products. Two pairs of software categories were compared: 1) software that was liked and disliked, and 2) a standard command line system (CLS) and a menu driven application (MDA). The reliability of the questionnaire was high, Cronbach's alpha=.94. The overall reaction ratings yielded significantly higher ratings for liked software and MDA over disliked software and a CLS, respectively. Frequent and sophisticated PC users rated MDA more satisfying, powerful and flexible than CLS. Future applications of the QUIS on computers are discussed.
Full-text available
The provision of health care over the Internet is a rapidly evolving and potentially beneficial means of delivering treatment otherwise unsought or unobtainable. Internet interventions are typically behavioral treatments operationalized and transformed for Web delivery with the goal of symptom improvement. The literature on the feasibility and utility of Internet interventions is limited, and there are even fewer outcome study findings. This article reviews empirically tested Internet interventions and provides an overview of the issues in developing and/or using them in clinical practice. Future directions and implications are also addressed. Although Internet interventions will not likely replace face-to-face care, there is little doubt that they will grow in importance as a powerful component of successful psychobehavioral treatment. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Full-text available
This paper describes recent research in subjective usability measurement at IBM. The focus of the research was the application of psychometric methods to the development and evaluation of questionnaires that measure user satisfaction with system usability. The primary goals of this paper are to (1) discuss the psychometric characteristics of four IBM questionnaires that measure user satisfaction with computer system usability, and (2) provide the questionnaires, with administration and scoring instructions. Usability practitioners can use these questionnaires with confidence to help them measure users' satisfaction with the usability of computer systems.
Although most employee assistance programs (EAPs) focus on the problems of individuals, many are broadening their function to address the rapidly changing human and social issues of the environments in which they operate. Refocusing practice to include the organization as the client establishes an alternative paradigm for conceptualizing practice. This article discusses traditional EAP practice, the evolution of EAPs, the changes confronting corporations, and the alternative paradigm in terms of the opportunities it offers and the risks of adopting it.
Wells Fargo and Company's Employee Assistance Consulting (EAC) is an established and successful internal corporate-based EAP that delivers organizationally congruent services to a diversified financial services company with 80+ businesses and over 146,000 team members. While some elements of traditional EAPs are evident in the services provided by this entity, EAC also provides highly specialized and customized consultations to its corporate partners, business group customers and employees. Using a highly integrated service model that aligns closely with the specific strategy and operations of each Wells Fargo business, EAC is an invited guest at many of the company's most influential tables. EAC collaboratively partners with senior and line management, Human Resources, the Employment Law Department, Disability Management (called WorkAbility at Wells Fargo) and Risk Management, Corporate Benefits, Corporate Security, Learning and Development, and others to provide leadership and organizational influence on such wide ranging issues as threat assessment and management, business incident management, workplace trauma, health and productivity, and organizational and employee effectiveness. This article describes proven methods and strategies for maximizing organizational influence and positioning EAP consultants as effective business partners.
Over the next 10 years, the healthcare system will change to focus more on preventive medicine and healthcare in the home, with fewer doctors and a new class of home healthcare providers. Healthcare professionals need to debate how best to manage these changes.