It has been noted previously that engineering as a discipline is reactionary1. Changes in how engineers are trained or how engineers conduct their work has waited for the emergence of new technology or shifts in the social zeitgeist. A committee with the National Academy of Engineering used scenario-based design2 to anticipate how engineers can best alter training and performance now to align with future industry needs. In the area of technology, it is anticipated that data storage and communications such as fiber optics will become increasingly important concepts to develop and understand. Another anticipated area of preparedness is changing the conceptualization of systems from one of separate pieces to one of a dynamic network of interconnected parts. Quantum sized electronics, predicted to be developed in the next few decades, will require an intimate knowledge of photonics and electronic theory. Processes using laser guidance or refinement are expected to become more precise, and smart materials that can react to their environment will respond to optical inputs. The anticipated increase in the use of digital systems for simulation, data analysis, and even everyday nonscientific tasks will require all engineers to understand digital systems and communications in their work.
Recommendations for engineering education include more interdisciplinary training and knowledge. The engineer of 2020 is seen as an individual with general knowledge of many areas rather than a master of a single area. Ingenuity, or being able to adapt to constraints as they appear, and communication skills for working in teams are traits deemed critical to instill in students in order for them to succeed as engineers in 2020 as well1. Indeed, it is suggested that engineering may benefit from embracing students as a component of each course they take in college rather than requiring a dedicated four-year degree 1. Looking somewhat further into the future, it is estimated that fully 50% of the United States population will belong to a non-white minority group by 2050, requiring engineering as a whole to develop methods for attracting and recruiting a diverse workforce in order to be large enough to meet the demands of the country. Case studies of specific engineers have been suggested as an alternative to teaching rote mechanics of engineering as well as a way to help students understand how to think about engineering. This personable spin on traditional content delivery could also provide a method of attracting populations traditionally underrepresented in the engineering domain.
Research devoted to understanding learning and retention has identified both general facts and the organization of facts as relevant components of understanding3. Students may not need every individual detail of an engineering or math course explained to them, but instruction must be thorough enough that students will encounter their misconceptions about the physical world and recognize them as incorrect. Otherwise, students may engage in cognitive dissonance, holding appropriate factual information about the world and additional inaccurate assumptions that prevent the proper application of these facts in long term memory. Research in the area of cognitive psychology has demonstrated that simple experience can be enough to correct misconceptions4. Cognitive psychology work in the area of mental models has also found that forcing students to form hypotheses before carrying out laboratory tasks can improve their ability to notice and correct the inaccurate portions of their mental models for complex biological process such as respiration5. In addition to correcting students’ misconceptions and providing facts, instruction needs to provide an appropriate organization for the facts given to students. Research with chess experts has demonstrated that appropriate organization of chess moves allows individuals to answer questions about momentary flashes of board arrangements more quickly and accurately than novices. Similar research has demonstrated that those with experience in electrical engineering are better able to integrate the components of a circuit diagram, allowing them to see a conceptual picture of a diagram rather than devote all of their attention into identifying individual pieces.
Metacognitive skills are important components of effective learning as well. If students actively monitor their progress during learning, they are better equipped to note areas of difficulty, notice areas where their preconceptions are in disagreement with presented material, or ask for assistance when needed. By teaching students to learn how to monitor their progress in school settings, teachers can assist students in tailoring their study habits to learn more material more accurately in less time.
Previous research examining the skills necessary in the physical science workforce is in agreement with the work cited above6. Survey work noted that only sixteen percent of professional Ph.D. physical scientists regularly used knowledge of their specific dissertation area at work. Rather than rely heavily on their dissertation methodology, respondents were required to use more general scientific methods, whether learned in graduate training or on the job. Out of a list of ten different skills, over 50% of participants rated management, interdisciplinary team skills, critical thinking, writing, and presentation skills as highly important to their work. This suggests rather than relying on previously learned experimental designs to speak for them, current doctorates are required to learn skills they are not taught during their graduate training to be efficient at work.
Additional research examining the communication skills necessary for current engineers also utilized survey data collection methods7. Overall, a majority of the engineers responding to the survey answered that they are required to write collaboratively with their peers. Seventy two percent of surveys collected indicated that the amount of time spent writing for their position had increased over time. A total of 41% of responses reported unclear written communication was a cause of inefficiency in the workplace. Difficulties from inappropriate written communication ranged from wasted time to generating feelings of mistrust between team members. It was found that clear, concise communication and the ability to change writing style based on an anticipated audience were the two most commonly reported suggestions for how to improve the communication skills training of future engineers.
Photonics Leaders II at North Carolina State University’s Science House is a three year, year-round science and information technology program for 69 high school students, their parents and 57 teachers. The goals of the program are to prepare underrepresented minority (URM) high school students for careers in science, technology, engineering and mathematics (STEM) and to aid parents and teachers in preparing these students to compete in an academic college setting and the global scientific workforce. The five program components are recruitment and retention activities, physics content, teacher professional development, parental engagement, and dissemination and evaluation. Program components are achieved through the principles of hands-on investigations, engagement in a supportive yet challenging environment, participation in leadership and professional development training, and interaction with outside professionals.
Several hypotheses were formulated for this research. It was hypothesized that significant gains would be found between timepoints for both the electronic theory and fiber optics tests for the treatment group. It was hypothesized that no significant differences would be found between timepoints for the control group in either electronic theory or fiber optics test scores. It was also hypothesized that significant differences would be found between the scores of the treatment and control groups on both the electronic theory and fiber optics post-tests, with the treatment group’s scores being significantly higher than the control group’s scores.
The data analyzed for this reporting is part of a larger three year longitudinal study. Analyses performed for this report focus on quantitative differences between a piloting sample and equivalent control group for a new fiber optics and electronic theory curriculum.
A total of 21 participants were recruited for this research. A total of 11 students volunteered to be pilot participants for the fiber optics and electronics theory curriculum. Nine participants were recruited from the body of students who had previously finished the Photonics Leaders II program to serve as a control group. Overall, all but one participant in this research had completed at least two years of training in the Photonics Leaders II program. This novice student was admitted into the treatment group. All participants were volunteers.
The fiber optics and electronic theory curriculum developed for this study consisted of four sessions of five hours each held on Saturdays during a fall semester (total of twenty contact hours). The first session began with the electronic theory and fiber optics pre-tests, a review lecture on electronic theory, a laboratory activity, a guest speaker, and an introduction to a diary activity designed to allow students to reflect on their newly acquired knowledge. In addition to promoting students’ reflection on knowledge, the journal activity was used to promote metacognitive skills and improve written communication skills. The laboratory activity required students to construct different kinds of circuits using breadboards, light-emitting diodes, and jumper wires. The guest lecturer was an industry professional from a local business that discussed career opportunities surrounding fiber optics. The second session consisted of a lecture on fiber optics, including safety protocol for the use of soldering equipment. Students also received a virtual talk from an invited guest speaker who discussed careers related to fiber optics and technical aspects of working with fiber optics technology. During the third session, students were asked to formulate a minimum of three research questions about fiber optics. Students were required to work in teams to form a testable hypothesis, design a method for testing the hypothesis, record their observations, and form an overall conclusion concerning each event under study. During their experimentation, students were able to identify errors in the instruction manuals for their circuit board equipment that resulted in faulty circuits. Rather than correct the instructions for the participants, students were assigned to use systematic methods to determine how to alter the guides so that they could complete the desired circuits. During the fourth session, students prepared and delivered presentations that included a question and answer session. Students discussed all of the components from the fiber optics and electronic theory curriculum, with a majority of their time devoted to findings from the circuit research performed in the third week. During the question and answer session, students answered questions from staff, other participants, and attending participant family members. Participants also completed post-tests at this time.
Compensation differed between the treatment and control groups. Participants in the treatment group were paid a total of one hundred dollars while participants in the control group were paid three hundred dollars.
Participants were given a total of fifteen minutes to complete all tests at each timepoint. Students in the treatment group completed tests with pen and paper while students in the control group completed them online. Students in the treatment group completed the electronics theory test and then the fiber optics test at each timepoint. Participants in the control group were allowed to complete their pre- and post-tests in any order at each timepoint.
The fiber optics and electronics theory tests were designed by an individual with several years of experience as a fiber optics specialist and a graduate student in statistics at North Carolina State University. Each test consisted of ten questions for a total of twenty questions. Each question was worth one point for a total of ten points in each test, with some questions having multiple parts. Question topics for the electronics theory test included calculating the total current of a circuit, naming the type of transistor shown in an image, completing Ohm’s law, and answering questions about the polarity of a diode. Question topics for the fiber optics test included the composition of fiber optic cable, the attenuation inside fiber optic cable, the kinds of signals that can be sent through fiber optic cable, and the functions of the different components in fiber optic cable. The primary course instructor graded the control group tests using a grading rubric.
It was found that data for both treatment and control groups was non-normally distributed for the electronics theory knowledge tests. For this reason, most comparisons for this sample were conducted using non-parametric tests of significance. For the treatment group, it was found that a significant gain in content knowledge occurred between the time of pre- (Mean=0.54) and post-testing timepoints for the fiber optics portion of the curriculum via the use of a paired samples t-test (Mean=0.71, t=-2.72, p<.05). It was found that the electronic theory test results were not a normal distribution and for this reason non-parametric testing was used, specifically a Wilcoxon signed-ranks test. It was found that a significant increase in content knowledge occurred over time between the pre- (Mdn=0.35) and post-testing time points (Mdn=0.8,) z=- 2.49, p<.05, r=-0.59) for the electronic theory test for the treatment group.The control group findings revealed no significant differences between timepoints for the electronic theory (pre-test Mdn=.30; posttest Mdn=.38, test z=-1.29, p>.05, r=-.08). Additionally, no significant differences were found between time points for the fiber optics test (pre-test Mean=.63; posttest Mean=.66, t=-.57, p>.05).
In order to identify differences between the treatment and control groups, a total of 3 Mann-Whitney tests were used. Statistically significant differences were identified between the treatment (Mdn=.80) and control (Mdn=.38) groups for the electronics theory post-test (U=11.5, p<.017, r=-.14). However, no statistically significant differences were identified between groups (treatment Mdn=.68; control Mdn=.67) for the fiber optics post-test( U=27, p>.017, r=-.03). A Mann-Whitney test for differences between the treatment (Mdn=.35) and control (Mdn=.30) groups at the electronics theory pretest found no significant differences ( U=36, p>.017, r=-.02). Due to the three tests performed for this research question, it was required that a Bonferroni correction (alpha/number of tests) be performed. For this reason, tests of between groups differences used an alpha level of .017 for significance.
The treatment group experienced a statistically significant increase in fiber optics content knowledge between the pre- and post-test assessments and suggests that the lectures, activities, and tasks completed during the intervention were effective in their delivery of curriculum content. Also, the treatment group experienced a significant gain in content knowledge scores overtime on the electronic theory test and suggests the curriculum was equally effective for this domain as well. But, the control group did not experience statistically significant gains in between the testing timepoints and suggests that neither the high school courses they enrolled in nor the extracurricular activities they participated in during the retention period contributed to their understanding of electronic theory or fiber optics.
These statistically significant differences were found between the treatment and control group electronic theory post-tests suggests that the electronic theory curriculum was more effective than the coursework or extracurricular activities of the control group for learning material in this domain. No significant differences were found between the treatment and control groups on the electronic theory pretest and this suggests that both groups had equivalent knowledge concerning electronic theory before the beginning of the intervention. As both groups were comprised of self-selected volunteers who could be assumed to have equivalent motivation, this would also suggest that the differences between the groups at the post-test timepoint were due to the curriculum content. No significant differences were found between the treatment and control groups for the fiber optics post-test and suggests that the curriculum for this portion of the curriculum was not more effective for learning material in this domain than the everyday activities of the control group.
When the findings concerning between and within groups’ effects are examined together, the findings from these analyses suggest clear successes for the use of programs similar to the Photonics Leaders II intervention. Treatment group and control group participants had equivalent electronic theory knowledge scores at the pre-test timepoint, but only the treatment group showed significant improvement at the post-test timepoint, and the treatment group post-test scores were significantly higher than those of the control group. Although the treatment and control group fiber optics post-test knowledge scores were not significantly different from each other, only the treatment group showed significant gains in between pre- and post-test timepoints.
When comparing the pre- and post-test means for the fiber optics test where the control group’s means were 60% and 65% percent respectively; the treatment group’s means were 53% and 70% respectively. Although the pre-test scores and post-test scores were not significantly different from each other, the average percentage improvement for the treatment group was 17%, over three times higher than the control group’s average percentage improvement. For this particular dataset, there is the possibility that the starting means were too different for the increased improvement to result in a significantly different gain in the treatment group’s score.
Given these positive findings, the Photonics Leaders II program research team plans to examine scaling this electronic theory and fiber optics module for a middle school setting and conducting testing with younger students. In addition to extending the pool of piloted students, another goal of program staff is the creation of a toolkit to help middle and high school teachers implement existing fiber optics and electronic theory activities or create new activities in this domain. Aligning this toolkit with existing national science standards will ensure activities implemented and created are as beneficial to students as possible. Additionally, after three years of intensive research with students, parents, and teachers for the Photonics Leaders II program, the project team plan to review the design rationale for the entire program, from initial conceptualizations to iterative adjustments that occurred between program semesters in order to determine which program practices were effective or ineffective. Examples of program elements to be reviewed are student selection, approaches to delivering curriculum materials, and student assessment.
This study was not without limitations. The distributions of several variables for this dataset were not normally distributed, preventing the use of more advanced statistics that could have been more informative. A larger sample size could allow the use of techniques such as repeated measures ANOVA, which would allow testing for within groups’ effects, between groups’ effects, and interactions of effects within a single test. As is common in education research, fewer than 100% of students completed each test. While a total of 21 students were recruited for this research, recruiting nine more for a total of 30 participants would make this dataset a candidate for statistical analysis with hierarchical linear modeling, which can include participants with incomplete data8.
While an expert in fiber optics and statistics constructed the fiber optics and electronics theory tests, neither test has been used with enough participants to allow reliability and validity to be established for them. Future research with these tests should be certain to obtain these values as soon as appropriate sample sizes are reached. While the testing that occurred for this report was carried out between students of equivalent fiber optics knowledge, electronics theory knowledge, and previous participation in the Photonics Leaders II program, the sample did not include a control group from outside of the program participants. To prevent the influence of cohort effects on interpreting the benefits of the Photonics Leaders II program an external control group should be included in future research. While significant differences were obtained between the treatment and control groups, ultimately statistical tests for this report compared the retention of one group with the acquisition of another.
Future studies should test both groups at the one month after program completion timepoint to compare the retention of both groups. Future studies should also seek to measure additional timepoints beyond one month as additional measures of retention. Measures of long term transfer should be examined as well, such as the influence of program participation on high school or college laboratory section grades, or troubleshooting circuits on novel devices9, 10 Additionally, all students participating in this research volunteered, meaning that results for this research may not be generalize for students not self-selecting into such programs.
The National Science Foundation Innovative Technology Experiences for Students and Teachers under the division of Research on Learning in Formal and Informal Settings fully funded the Photonics Leaders II Program Award #0833615.
National Academy of Engineering, [The Engineer of 2020: Visions of Engineering in the New Century], The National Academies Press, Washington, DC, (2004).Google Scholar
Go, K., Carroll, J. M. (2004). The blind men and the elephant: Views of scenario-based system design. Interactions, 44-53.Google Scholar
Division of Behavioral and Social Sciences and Education. [How Students Learn: Science in the Classroom], The National Academies Press, Washington, DC, (2005).Google Scholar
Greene, J. A., & Azevedo, R. (2007). Adolescents’ use of self-regulatory processes and their relation to qualitative mental model shifts while using hypermedia. Journal of Educational Computing Research, 36(2), 125-148.Google Scholar
Modell, H. I., Michael, J. A., Adamson, T., Goldberg, J., Horwitz, B. A., Bruce, D. S., et al. (2000). Helping undergraduates repair faulty mental models in the student laboratory. Advances in Physiological Education, 23, 82-90.Google Scholar
Smith, S. J., Pedersen-Gallegos, L., & Riegle-Crumb, C. (2002). The training, careers, and work of Ph.D. physical scientists: Not simply academic. American Journal of Physics, 70(11), 1081-1092. doi: 10.1119/1.1510884.Google Scholar
Keane, A., & Gibson, I. (1999). Communication Trends in Engineering Firms : Implications for Undergraduate Engineering Courses. International Journal of Engineering Education, 15(2), 115-121.Google Scholar
Raudenbush, S. W., & Bryk, A. S., [Hierarchical Linear Models: Applications and Data Analysis Methods (2nd Edition), Sage Publications, Thousand Oaks, California, (2002).Google Scholar
Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4), 207-217.Google Scholar
Schneider, W. (1985). Training high performance skills: Fallacies and guidelines. Human Factors, 27(3), 285-300.Google Scholar