16 August 2017 Study on process evaluation model of students' learning in practical course
Author Affiliations +
Proceedings Volume 10452, 14th Conference on Education and Training in Optics and Photonics: ETOP 2017; 104526N (2017) https://doi.org/10.1117/12.2269992
Event: 14th Conference on Education and Training in Optics and Photonics, ETOP 2017, 2017, Hangzhou, China
Abstract
In practical course teaching based on project object method, the traditional evaluation methods include class attendance, assignments and exams fails to give incentives to undergraduate students to learn innovatively and autonomously. In this paper, the element such as creative innovation, teamwork, document and reporting were put into process evaluation methods, and a process evaluation model was set up. Educational practice shows that the evaluation model makes process evaluation of students' learning more comprehensive, accurate, and fairly.
Jie, Pei, SHEN, and Youxiang: Study on process evaluation model of students' learning in practical course

1.

INTRODUCTION

Practical course of science and engineering such as Curriculum Design, Specialty Experiments aims to develop the students’ ability to research, analyse and solve problem, train students’ practical skill and innovation ability. How to assess the students’ performance objectively and exactly in practical course is one of the ubiquitous problems. The traditional way of evaluation mainly adopts final course grade which is determined by assignments and class performance and mark on the final with different percentages. Due to lack of a uniform criterion of justness, traditional evaluation methods is based on teacher’s subjective judgments, and usually results in area indexing of students’ grade is insufficient, and can’t evaluate the students’ performance objectively and exactly. In allusion to this phenomenon, statistics method is used in the comprehensive assessment of practical course in many colleges such as standard deviation, cluster analysis, principal component analysis, and so on[1-3]. These methods improve the traditional evaluation method, and the effect is good, but the statistics and calculation is complex and needs huge work, which will increase the workload of the teachers, moreover, these methods still can’t avoid subjectivity of teachers. The fuzzy comprehensive evaluation based on the theory of fuzzy mathematics is an effective method which considers a multiplicity of different influence factors and can assess the students’ performance objectively and scientifically [4-6].

Here, based on Engineering Education Accreditation Standards and the theory of fuzzy mathematics [7-9], on the basis of confirming the evaluation’s factor assembly and concrete evaluation factor, through two layers evaluation, the quantitative comprehensive evaluation model of graduation requirement achievement degree is established.

2.

ESTABLISHMENT OF AHP-Fuzzy EVALUATION MODEL

2.1

Determination of the factor set

Based on Engineering Education Accreditation Standards, the major that wishes to accredit, should be set reasonable training objective meet Graduate Attribute. Engineering Education Accreditation Standards Graduate Attribute Profile has 12 elements, 7 elements of them relevant to practical course: Problem analysis, Design/development of solutions, Investigation, Modern tool usage, Occupational norms, Individual and teamwork and Communication. So these 7 elements of Graduate Attribute Profile is set as the first achievement degree evaluation index. Set X is the first achievement degree evaluation’s factor set, let X=(X1, X2, ···, Xm), m=1, 2, 3, ···, 7. Each the first achievement degree evaluation’s factor consisted of several the second achievement degree evaluation’s factor. Let Xm= {Xm1, Xm2, Xm3, ···, Xmi}(i=1, 2, ···, n). Achievement degree evaluation index as shown in Table 1.

Table 1.

Achievement degree evaluation index of practical course

The first achievement degree evaluation indexThe second achievement degree evaluation indexAchievement degree
ExcellentGoodMediumPassedFailed
Problem analysis (10%)Understanding principles of mathematics, natural sciences and engineering sciences in course (10%)10000
Analyse problems using first principles of mathematics, natural sciences and engineering sciences (30%)01000
Identify, formulate, research literature and analyse complex engineering problems reaching substantiated conclusions using first principles of mathematics, natural sciences and engineering sciences (60%)01000
Design/developme nt of solutions (30%)Design solutions for complex engineering problems using the knowledge contents of the course (35%)10000
Design solutions for complex engineering problems using comprehensive knowledge (50%)01000
Design innovative solutions for complex engineering problems (15%)00100
Investigation (20%)Master the basic methods of experimental design (20%)01000
Design of experiments, analysis and interpretation of data, and synthesis of information to provide valid conclusions (70%)01000
Pointing out the defections and puts forward improvement way (10%)00100
Modern tool usage (20%)Familiar with literature search and analysis (20%)10000
Skillful in using softwares are required for this course (30%)01000
Complete hardware design and test debug (50%)00100
Occupational norms (5%)Active learning attitude (10%)10000
Observe class disciplines (40%)10000
Complete the task of course responsibly (50%)10000
Individual and teamwork (5%)Understanding the role of individuals in the team (10%)01000
Participate in and facilitate work teams (60%)01000
Have good team work spirit, and maintain good cooperate with other team members (30%)01000
Communication (10%)Being able to comprehend and write effective reports and design documentation (60%)00100
Explain and answer questions clearly and accurately (10%)00100
Make effective presentations and give and receive clear instructions (30%)01000

2.2

Determination of weight vector W

The traditional methods to determine the weight of each attribute include the expert estimates method, the weight statistic method, the frequency statistic method and so on [10]. Here, fuzzy analytic hierarchy process is applied to determine the weight of the first achievement degree evaluation’s factor and the second achievement degree evaluation’s factor. As the following step:

Step 1: The nine-point scale fuzzy analytical hierarchy process is used to construct the fuzzy judgment matrix: X = (xij)n x n (i, j=1,2,···,n). The meaning of each scale measurement is explained in Table 2.

Table 2.

Scale of preference between two elements

ScaleDefinitionExplanation
1Equally preferredTwo activities contribute equally to the objective
3ModeratelyExperience and judgement slightly favour one activity over another
5StronglyExperience and judgement strongly or essentially favour one activity over another
7Very stronglyAn activity is strongly favoured over another and its dominance demonstrated in practice
9ExtremelyThe evidence favouring one activity over another is of the highest degree possible of affirmation
2, 4, 6, 8Intermediate valuesUsed to represent compromise between the preferences listed above
1, 1/2, 1/3, ···, 1/9ReciprocalsReciprocals for inverse comparison

Step 2: The pair-wise comparison matrix, X = (xij)n x n (i, j=1, 2, ···, n), the eigenvector is given by equation (1):

00046_PSISDG10452_104526N_page_4_1.jpg

for all i = 1, 2,…, n

Step 3:The eigenvector needs to be normalized, and the weights are computed by equation (2):

00046_PSISDG10452_104526N_page_4_2.jpg

for all j= 1, 2,…, n

Step 4: Measurement of consistency: the consistency ratio is calculated as per the following steps:

  • (1) Calculate the eigenvector or the relative weights and λmax for each matrix of order 00046_PSISDG10452_104526N_page_4_3.jpg.

  • (2) Compute the consistency index for each matrix of order n by the formulae: 00046_PSISDG10452_104526N_page_4_4.jpg.

  • (3) The consistency ratio is then calculated using the formulae: 00046_PSISDG10452_104526N_page_4_5.jpg.

where RI is a known random consistency index obtained from a large number of simulation runs and varies depending upon the order of matrix. Table 3 shows the value of the random consistency index (RI) for matrices of order 1 to 10.

Table 3.

Average random index (RI) based on matrix size

N12345678910
RI000.520.891.121.361.411.461.491.52

If the value of CR is less than 0.1, it implies that the evaluation within the matrix is acceptable or indicates a good level of consistency in the comparative judgements represented in that matrix. In contrast, if CR is more than 0.1, inconsistency of judgements within that matrix has occurred and the evaluation process should therefore be reviewed, reconsidered, and improved.

2.3

Determination of the evaluation set

For recording students' learning in practical course conveniently, the model adopts 5 class marking system that including excellent, good, medium, pass, fail. The corresponding score: excellent: 95, good: 85, medium: 75, passed: 65, failed: 50. The evaluation set is defined as Y= {Y1, Y2, Y3, Y4, Y5} = {95, 85, 75, 65, 50}. The appraisal set is defined as an appraisal vector:

00046_PSISDG10452_104526N_page_4_6.jpg

where rmn is the fuzzy membership degree of appraisal of the factor Xm to

2.4

Fuzzy AHP Evaluation

To consider the collaborative impacts of all achievement degree evaluation index, at the same time, keep the information of individual achievement degree evaluation index, a fuzzy arithmetic operator 00046_PSISDG10452_104526N_page_5_2.jpg is chosen. The appraisal of the second achievement degree evaluation index factors can be calculated in the following appraisal vector Bi = Wi · Ri.

The final fuzzy evaluation vector B of the first achievement degree evaluation index factors is made by multiplying the weight vector W and the appraisal vector R, as shown by B = W · R.

Further, quantification of result can be obtained by 00046_PSISDG10452_104526N_page_5_1.jpg.

3.

APPLICATION OF THE MODEL

The model has been applied to integrated course project for circuit and electronic. The fuzzy judgment matrix X and Xm were obtained by pair-wise comparison, the fuzzy judgement vectors are as follows:

00046_PSISDG10452_104526N_page_5_3.jpg

The relative weights of each element from the first achievement degree evaluation’s factor and the second achievement degree evaluation’s factor and the consistency ratio of each matrix were analyzed, as follows:

W = (0.127, 0.296, 0.228, 0.199, 0.027, 0.047, 0.077)T, CR=0.047.

W1 = (0.103, 0.292, 0.605)T, CR1=0.05. W2 = (0.344, 0.535, 0.121), CR2=0.023.

W3 = (0.225, 0.696, 0.079)T, CR3=0.092. W4 = (0.162, 0.309, 0.529)T, CR4=0.011.

W5 = (0.137, 0.493, 0.37)T, CR5=0.04. W6 = (0.103, 0.605, 0.292), CR6=0.037.

W7 = (0.575, 0.114, 0.311)T, CR7=0.096.

Since all CR < 0.1, indicates a good level of consistency in the comparative judgements represented in that matrix and the pair wise comparison matrix can be accepted.

For convenience of statistics and calculation at the teaching process, the weights were rounded, and expressed as percentages.

W = (10%, 30%, 20%, 20%, 5%, 5%, 10%)T,

W1 = (10%, 30%, 60%)T, W2 = (35%, 50%, 15%)T, W3 = (20%, 70%, 10%)T,

W4 = (20%, 30%, 50%)T, W5 = (10%, 40%, 50%)T, W6 = (10%, 60%, 30%)T,

W7 = (60%, 10%, 30%)T

Take a student as an example: the achievement degree of this student in the whole teaching process as shown in Table 1. The appraisal of the second achievement degree evaluation index factors were calculated by relation equations Bi = Wi · Ri, the results as shown in Table 4.

Table 4.

The appraisal of the second achievement degree evaluation of a student

The weights of the first achievement degree evaluation index factorsThe second achievement degree
ExcellentGoodMediumPassedFailed
10%10%90%000
30%35%50%15%00
20%090%10%00
20%20%30%50%00
5%100%0000
5%0100%000
10%030%70%00

The final fuzzy evaluation vector B calculated by relation equations B = W · R, the results as shown in Table 5.

Table 5.

The final fuzzy evaluation of the student

 The final fuzzy evaluation
ExcellentGoodMediumPassedFailed
B20.5%56%23.5%00

Further, quantification of result can be obtained by 00046_PSISDG10452_104526N_page_6_1.jpg.

4.

CONCLUSION

This paper choose the evaluation index from Engineering Education Accreditation Standards and establishes a process evaluation model of students' learning in practical course by the fuzzy comprehensive evaluation method and fuzzy analytical hierarchy process. Take integrated course project for circuit and electronic and a student as an example, the procedure of model establishment is introduced. It can be concluded that the model could accurate, clear reflection of the performance of student in practical course, facilitate teacher assessing the student more objective and fair. This model could be extended to the graduation thesis, production internships and other practical courses.

REFERENCES

[1] 

TANG Wen, WU Ying, “Study on the Evaluation of the Score of Experimental Course Based on Statistic Analysis.” Journal of Shanghai Institute of Technology (Natural Science). 13(3), 241–244(2013).Google Scholar

[2] 

LIU Tie-bing, YUAN Qiu-ping, ZHANG Yan-ping, “Grading Probing of Food Analysis Experiment Course Based on Standard Deviation.” Journal of Anhui Agri. Sci.. 39(18), 11003–11005(2011).Google Scholar

[3] 

PENG An-hua, “Assessment of Experimental Score Based on Grey Clustering Model.” Journal of Huaihai Institute of Technology (Natural Science Edition). 19(1), 9–12(2010).Google Scholar

[4] 

FAN Yuqing, ZHANG Yuemei, WANG Xiuhai, “Application of FCE-AHP in Performance Evaluation of Comprehensive Experiments of Environmental Science.” Experiment Science and Technology. 13(5), 80–82(2015).Google Scholar

[5] 

ZHANG Zhi-ying, “An Application of Fuzzy Evaluation in Assessing Graduation Thesis.” Journal of Zhejiang Sci-Tech University. 28(3), 467–470(2011).Google Scholar

[6] 

GU Zeng-ju, “Grade Assessment Model of Fuzzy Comprehensive Evaluation and Its Application-ERP Sand Map Simulation.” Experiment Science and Technology. 8(6), 99–102(2010).Google Scholar

[7] 

Lin Jian, “The Accreditation, Reformation and Development of Engineering Education.” Research in Higher Education of Engineering. 2, 10–19(2015).Google Scholar

[8] 

Hu Yong-hong, He En-hui, [Comprehensive Evaluation Method], Science Press, Beijing, 167–168(2000).Google Scholar

[9] 

SONG Xiao-qiu, [Principles and Methods of Fuzzy Mathematics (Second Edition)], China University of Mining and Technology Press, XuZhou, 20–22(2004).Google Scholar

[10] 

Xie Ji-jian, Liu Cheng-ping, [Fuzzy Mathematics Method and its Application (Second Edition)], Huazhong University of science and Technology Press, WuHan, 215–219(2000).Google Scholar

© (2017) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Jie Huang, Jie Huang, Pei Liang, Pei Liang, Wei-min Shen, Wei-min Shen, Youxiang Ye, Youxiang Ye, } "Study on process evaluation model of students' learning in practical course", Proc. SPIE 10452, 14th Conference on Education and Training in Optics and Photonics: ETOP 2017, 104526N (16 August 2017); doi: 10.1117/12.2269992; https://doi.org/10.1117/12.2269992
PROCEEDINGS
7 PAGES


SHARE
Back to Top