Usability statistics for a) average number of requests per session, b) the average time to wait for an attention and the duration of the attention In Figure 5, we can see average usage statistics per session for both courses. The figure on top shows that the number of requests is higher in sessions 3 and 6, which correlates with 

Context in source publication

Context 1

... 17 was different to the one initially requested 4 The online system is not necessary. It is 13 better and easier to make the request directly to the instructor/collaborator Table 2. Most frequent negative comments about the application. When analyzing these results and the objective statistics, we detected that complaints 1, 2 and 3 were related to each other, since they are related with problems that mostly happen during the critical sessions of the subjects, and because typically the same person mentioned both complaints in the survey at the same time. Regarding the first complaint, in Figure 5b we see that the average waiting time is low except during critical sessions; therefore it is clear that the available human resources are not enough, so it is necessary to find new mechanisms to reinforce the critical sessions reallocating better the available personal along the other sessions and finding strategies that make students to distribute better their work along the different sessions. Regarding the second complaint, we need to mention that although the average number of instructors and collaborators per session is three, in the last sessions the process of solving questions could take more time since instructors need to go through a complete system where many modules and components are involved, or to debug large software programs that execute a multithreaded application. On the other hand, the third complaint refers to a situation where the requested person does not come on time since she/he could be solving other students’ problems, then a different person to the one requested came in an attempt to guarantee the attention; however, students complain because this increases the attention time since they need to explain the problem to the new assistant from the beginning. One solution we have included for the next year is the possibility of requesting the assistant with a specific or a generic instructor or collaborator; therefore, the queue will show only the specific requests plus the generic ones, instead of showing the requests to all other instructors or collaborators. The fourth complaint is that some students show their preference to go back to the traditional approach of students standing up from their workstation and start looking for the instructor/collaborator to directly make the request. Unfortunately, this mechanism had many drawbacks, for instance: a) if the instructor is occupied with other students, he/she will need to pay attention to the person making the new request interrupting the current attention (in comparison, the online system allows an uninterrupted attention). b) students would need to go to all the other instructors/collaborators to make the same request, or to wait for a specific one when others are available, besides, if the students made the same request to different persons all of them could try to come to assist (the new system avoids these problems and optimize the distribution of the requests), and c) In case the instructor forgets the request, the students will not receive the attention on time (the new system minimizes this problem). 62 students answered this question and their comments are summarized in Table 3. From this question, we want to highlight again that 12 students considered that the system was good and not additional improvements were needed. In addition, many of them added that they considered it as a clean and fair mechanism for requesting help at the laboratory. The second idea is not related with the application itself, although for the third idea we could include some kind of warning messages in the instructors and collaborators forms to display their attention time and to indicate them if their last attention was too long. Regarding the fourth item, some students suggested the creation of a special kind of request to solve short questions and that can be set on the queue on the first positions for a fast attention. This is a functionality we could add in addition to our current prioritization mechanism including new rules for controlling the number of requests of this kind that a couple can do in order to try to guarantee an optimal use of this feature. 1 The system is good and no additional 16 improvements are needed 2 Increase the number of instructors and 12 collaborators 3 Include a mechanism to control the time of the 7 attention, and that the person who is specifically requested came and not other 4 Create a new kind of request for short questions 5 5 To provide information about the expected time 4 to start the attention not only the number of students in the queue Table 3. Statistics about how to improve the system Finally, some students requested the possibility of providing an estimated time about how much students would have to wait for the requested person to come. Although this information is difficult to estimate, an approximated value could be provided taking into account the usage statistics per session provided in Table 1 and Figure 5, as well as the average duration of the assistances during the current session. V. CONCLUSIONS In this paper, we have described a complete web-based system that allows students from two massive laboratory courses to request help from instructors or collaborators using an ordered and transparent mechanism. This system has the advantage that it can be accessed using any kind of hand-held device (PDA) or PC through an Internet connection. The system includes different kind of forms that allow students, instructors and student collaborators to access to the online request service and organize the process of providing the assistance. Moreover, the system includes additional tools for performing continuous evaluation allowing instructors to follow better the students’ work. In addition, objective usage statistics are shown to the collaborators allowing them to improve their assistance and to instructors to evaluate them. On the other hand, the results of objective and subjective evaluations allowed us to measure the impact of the system on the students and collaborators, and to collect ideas for future improvements. The results show a high degree of acceptance (25% gave it a high score above 9 in contrast to only 18% who gave it a score below 5), and where many of the complaints were due to external factors not related with the system itself. Usability measures confirm the high use of the platform, and prove the important work that student collaborators do to solve basic questions and to energize the class. Finally, we have shown results about the things that students considered that should be improved and those that were not good, in order to provide important user’s experience information in case of extending the current platform to other subjects. VI. FUTURE WORK Considering the results obtained from the objective and subjective evaluations, as well as comments from collaborators and instructors, we propose the following ideas as relevant improvements for the system. First, in order to reduce the waiting time, we propose the inclusion of colorful messages to the instructors and collaborators at the end of the assistance in case that the attention was too long. Second, in [12] it is described an online forum monitored by the instructors, where the students can propose, discuss or answer to information related with the subject. The metrics extracted from the participation in the forum allow the instructors to predict the final students’ scores. We plan to follow a similar approach here taking the usability metrics as a predictor of the students’ and collaborator students’ scores. Our goal would be to provide additional tutoring over the potential students which could not succeed in their project but also to help the best students to achieve better scores. Finally, [13] describe and analyze a complete list of web- based interactive programming environments and tools that can be used to simplify the learning of programming languages and concepts. These applications are analyzed and classified according to their capabilities and focus. In our case, the idea would be to use similar URLs and display them through the queue form of the students, in order to provide dynamic help in case students mention that they have a specific problem with the programming language or about how to use a given electronic component. Since this information is displayed in the form during the time they are waiting for the attention to start, the students can take advantage of that time to check this proposed information. VII. ACKNOWLEDGEMENT We want to specially thank to Luis García, Francisco Romero, José Miguel Jiménez, and Victor Iglesias for their administrative support at the laboratory and all the members of GRIDS innovation group in the Electronic Engineering Department at UPM; we also thank Prof. Javier Macias and Ignacio Shyri Villar for their important software contributions. Finally, we want to thank Álvaro de Guzmán Fernández González and Roberto Barra Chicote for their support as course coordinators for LCEL and LSED subjects respectively, as well as to all instructors and student collaborators for using the system. This work has been supported by INAPRA project (DPI2010-21247-C02-02) and for the UPM innovative project No. IE12_13-09004 leaded by Pedro José Malagón ...