1.

INTRODUCTION

Accreditation is a process of quality assurance and control both in higher education and other settings, where by, as a result of assessment or inspection or both, programs and institutions are recognized as meeting minimum acceptable standards. Accreditation is both a process and a status. As a process, accreditation involves self-evaluation and external peer review. As a status, accreditation provides public notification that an institution or course meets standards of quality previously set. Therefore the accreditation review seeks not only to meet standards, but also to continuously explore new ways in which to enhance the quality of training and education provided.

In the framework of the European studies under the Bologna process (see e.g. [1]), the Spanish University System presents a huge number of Masters courses. This fact has yielded the creation of an official procedure of “accreditation” of this kind of degrees. In this work, we present and discuss data collected from the official accreditation process recently carried out for the Masters on “Photonics and Laser Technologies”, coordinated by the University of Vigo (UVigo) and involving three Universities: Vigo (UVIGO), Santiago de Compostela (USC) and A Coruña (UdC) in the autonomous region of Galicia (Spain). Teachers and students are split in four different campus and theoretical lessons rely on a videoconference system. Accreditation is made by the Agency for the Quality of the University Galician System (ACSUG).

The accreditation procedure consists on two steps. The first one is the gathering of documentation proving how the different aspects involved in the master’s organization, teaching and evaluation are taken care of. The dossier is analyzed by an external commission. In the second step, the commission visits the institution and interviews representatives of the different groups of interest involved (teachers, management team, students, graduates, potential employers, other personnel). Finally, a global assessment is performed. In the case of the master in photonics by the Galician Universities, the evaluation was positive, and guidance to improve the weaker points was provided.

Among others beyond the scope of the present report [2], one of the pillars of the evaluation leading to accreditation is that of indicators of performance, being one of the most important the collection of data from alumni. To this aim, the academic commission of the masters prepared a series of questionnaires that were sent by email to all the alumni. Thanks to the data collected during the procedure, the academic and administrative authorities can have first-hand information about many different important parameters of the master course and its quality. These data play a fundamental role in order to make future decisions about the studies offered in the Galician University System. Another pillar is that of public information and transparency. Since the commission assessed as “good practices” several aspects of the master course organization concerning these pillars, the goal of this work is to explain some actions that were carried out and that might be useful to consider in other processes of accreditation.

2.

INDICATOR OF PERFORMANCE: SURVEYING GRADUATE STUDENTS

An important indicator of the quality of any degree or master course is the satisfaction expressed by former students. However, it is not always easy to obtain credible responses from a representative sample of people who have already left university. It is well known that unemployment is one of the most important problems in Spain, from the socio-economic point of view. Therefore, having data of employment after graduation is one of the most important statistics in higher studies.

We designed a survey of extreme simplicity, which could be answered in an average time of two minutes. Making it easy and quick to answer, the expectation was to promote participation. The survey was carried out using the online platform surveymonkey®. It was sent by email to graduates in academic years between 2007/2008 and 2013/2014. Answers were anonymous and the participation was 52 out of 115 alumni from the MSc in Photonics and Laser Technologies from the three universities of Galicia (Spain). This amounts to a participation of 45%, much higher than the average participation in similar surveys performed by the standard university procedures. Moreover, it should be pointed out that a number of graduates did not receive the survey because they had changed e-mail address since graduating. Thus, the effective participation rate (number of answers/number of receivers of the survey) was even higher. The questionnaire was open between December 12^th and December 19^th, 2014.

In the following, we plot the result of the survey. Data have been analyzed from a gender perspective, separating male and female responses.

Fig. 1:

Employed alumni within one year after finishing the master course.

One of the most important proofs of the satisfaction with the studies is the response to the direct question: “will you recommend this masters course to other persons?” The results to this question are plotted below:

Fig. 2:

Response to the question: “Will you recommend these studies?”

Graduates were also asked to give a numerical value to their general satisfaction with the master and with some particular issues like the adequacy of the contents of the syllabus of the different subjects.

Fig. 3:

General satisfaction evaluated in a range from 0 (no satisfaction) to 9 (total satisfaction). The horizontal axis shows the number of answers for each numerical value.

Fig. 4:

Satisfaction with the contents of the master program, evaluated in a range from 0 (no satisfaction) to 9 (total satisfaction). The horizontal axis shows the number of answers for each numerical value.

The acquisition of competencies is a fundamental point for assessing any official studies. We found it important to provide an evaluation on this point.

Fig. 5:

Answer to the question: “Evaluate the utility of the competencies acquired in the master course for your present responsibilities (0 useless – 9 extremely useful)”.

The former students were also asked about their satisfaction regarding the teachers and the videoconference system.

Fig. 6:

Teaching by the faculty, evaluated in a range from 0 (no satisfaction) to 9 (total satisfaction).

Fig. 7:

Videoconference system, evaluated in a range from 0 (no satisfaction) to 9 (total satisfaction).

The time evolution of the employment and satisfaction indicators are plotted below:

Fig. 8:

Temporal evolution of the employment and satisfaction indicators

Globally, the assessment of the master course is clearly positive. That 82% of the graduates would recommend the studies is especially noteworthy. On the other hand, it is interesting to know the employment data. In a context of economic crisis and extreme unemployment rates in Spain, they can be considered as adequate. Nevertheless, this statistic can be further improved and these data point to the necessity of developing the contents of the master in view of the labor market. Employment data are better for men than for women. It is worth remarking that the assessment of the teaching staff reaches the remarkable value of 87%. Concerning the videoconference system used for the theoretical lessons of the master course [3], [4], a dip in the evaluation marks can be appreciated in the 2011/2012 course, when the commercial Adobe Connect® system was used for the first time. It can probably be explained by the initial inexperience of teachers in profiting from all the possibilities of such system. This interpretation is consistent with the subsequent recovery, after efforts to prepare all teacher for optimizing online lessons [5]. Regarding competencies, the average evaluation is of 70%. Thus, it seems there is room for improvement and this self-assessment of graduates is an important basis for future updates of the master contents and teaching methods.

No strong trends can be observed in the temporal evolution of the different indicators. This shows that the quality of the teaching remains at a satisfactory and stable level. The goal of future developments will be to further enhance them.

3.

PUBLIC INFORMATION: THE IMPORTANCE OF A USEFUL WEBPAGE

Public information is essential for alumni studying the course and for potential future students. An up-to-date webpage centralizing all information in a self-contained and consistent way is of utmost importance for accreditation. Thus, an effort was made by the master commission in order to provide a webpage in which all administrative information can be obtained but keeping its visual appeal and the usefulness for the students [6]. Information must be easily found upon navigation. Thus, both structure and contents are important. We describe here the solution that was implemented for the photonics master.

4.

CONCLUSIONS

Accreditation is a process devised at Spanish institutions to ensure the quality of the teaching provided at universities. The master in Photonics and Laser Technologies involving the three Galician universities has recently passed the evaluation and overcame the filtering criteria. The experience can be useful for future accreditation processes since some of the solutions found might be profitably exported to other cases.

In this note, we have outlined some of the actions taken to meet the criteria of having quantitative performance indicators and transparent and complete public information. We conclude that: