Outline (midterm) of research project
Peer Feedback – Asynchronous CMC verses Face-to-Face Methods:
The Influence on Student Achievement in the Creation of Electronic Portfolios in a Web-enhanced Environment
Marian Maxfield
Kent State University
Purpose
The goals of this study are first to assess the effectiveness of rubric-guided face-to-face peer feedback on student achievement (grades / points). Second, to assess the effectiveness of rubric-guided computer mediated communications (CMC) peer feedback on student achievement, and third, to assess the effectiveness of the quantity of peer feedback on student achievement when creating an electronic portfolio. The purpose of this study is to analyze and further the understanding of the effects of different forms of peer feedback (asynchronous CMC and face-to-face) and the quantity (length) of peer feedback when creating an electronic portfolio. Is rubric guided evaluative peer feedback more effective on student achievement in face-to-face or computer mediated communication exchanges when creating an online portfolio? Do students provide more quantity of feedback in a CMC or face-to-face setting? Does the quantity of the feedback affect student achievement?
Literature Review
Peer Feedback
Yanes, M.J. (2004). Distance education in traditional classes: A hybrid model. The Quarterly
Review of Distance Education, 5(4), 265-276. Retrieved August 31, 2005 from
EbscoHost.
Yates conducted a study concerning feedback with the use of computer-mediated-communications (CMC) through the use of a discussion forum and e-mail. This study is similar to the one being conducted. Yates taught a face-to-face pre-service teacher preparation course but with the use of discussion forums for peer feedback. The e-mail was only used for dialogue with the instructor. Yates thought that feedback was important for interaction as well as expanding meaning. This was perceived through Vygotsky’s “sociocultural constructivism as a process of social negotiation of meaning” (p.256). Yates referred to the Zone of Proximal Development (ZPD) to describe the cognitive distance between the actual knowledge of the student and the potential of the student, which could be brought about through collaborations with peers. The ZPD allows for social groups (or what I would refer to as social constructivism) to create new learning and understanding. Yates created a rubric using Bloom’s taxonomy to evaluate the level of cognitive activity of student reflections and peer feedback on the discussion forum. The results indicate that students participate more in a CMC environment than in a face-to-face environment. A greater percentage of students participated and provided feedback with the use of CMC. Students functioned at the upper levels of ZPD through interdependence and interpersonal interactions with peers (p. 273). Yates discussed that the discussion forum preserves information, which provides students the opportunity to reflect and provide a more in-depth analysis.
Hebert, B.G., & Vorauer, J.D. (2003). Seeing through the screen: Is evaluative feedback
communicated more effectively in face-to-face or computer-mediated exchanges?
Compuers in Human Behavior, 19, 25-38.
The overall hypothesis of the authors was that judges (those providing feedback) of the work would provide positive feedback and the targets (those receiving feedback on work) would have a clearer understanding of the feedback in the face-to-face (F2F) feedback versus the computer mediated communications (CMC) feedback. First, the authors dealt with the differences between positive and negative feedback given in both FTF and CMC. There was more positive feedback provided in the face to face than in the CMC group. Second, authors noted that the feedback provided in the face-to-face group was more comprehensible than the feedback provided in the CMC. The feedback was especially clearer when providing feedback about skill-based issues. It was easier for the targets to discern the feedback from their peers when face-to-face. The authors also noted that there was more detail in the CMC and possibly due to the time lapse that provided students the opportunity to re-read and review the comments posted by their peers.
Rubrics for evaluation
Standford, P., & Siders, J.A. (2001, January). Authentic assessment for intervention.
Intervention in School and Clinic, 36(3), 163-167. Retrieved October 22, 2005, from
Academic Search Premier.
The article focuses on rubrics and how they can meet the needs of a special needs students. However, I believe that these same ideas can be applied to all students. A rubric is designed to set criteria for outlining the expectations for the assessment of the project or paper. It is one way to quantify a project. There are many advantages and reasons for creating a rubric prior to the assignment. Creating a rubric for students provides “common focus for the instruction” as well as a common ground for both students and teacher (165). Rubrics can aid students in locating strengths and weaknesses. “The learning process becomes more concrete with the narration and feedback inherent in the rubric” (p.165). There are other uses, which allows the student to review the learning task internally as the project is created. When a rubric is used in conjunction with peer feedback, it “…provides a social context and often a lesson threatening and more inclusive response to the special learner” (p. 165).
Electronic portfolios
Ahn, J. (2004, April). Electronic portfolios: Blending technology, accountability &
assessment. THE Journal, 31(9),12-14. Retrieved March 3, 2005 from Academic
Search Premier.
Ahn provides the basics about e-portfolios. The first is the benefit of creating an e-portfolio such as creating community, self-reflection, and teacher feedback. The issue with this article, as with many, is that peer feedback is a missing component of the learning and assessment process. The planning stage is an important aspect of creating the portfolio. During this stage it is important to have clear, defined standards and organized guidelines for students. If the expectations are provided before the creation of the e-portfolio, students can work constructively to work and meet the requirements and the teacher can provide appropriate feedback to the student. Self-reflection is part of the e-portfolio process and even though this is important it appears that many articles do not address the issue of peer feedback as an assessment component when designing electronic portfolios.
Mason, R., Pegler, P., & Weller, M. (2004). E-portfolios: An assessment tool for online
courses. British Journal of Educational Technology, 35(6), 717-727. Retrieved
March 4, 2005 from EbscoHost
E-folios are often used as an assessment tool for courses that are taught online, especially in education. This study hypothesized “…that e-portfolios can be a fitting assessment model for courses designed in learning objectives” (p. 726). The researchers specified the end of course assessment (ECA) as an e-portfolio with the use of learning objectives. The learning objectives are micro in scope to allow a student to individualize the artifacts within the e-portfolio. The study was designed with online Master’s students (n = 45) from 16 different countries. The course is designed to have two activities within each of the four modules. This means that students will end the course with eight e-portfolio artifacts. The data was collected with phone questionnaires and discussion forums. The authors concluded that the learning objectives were helpful but more structure and guidance is needed when creating a portfolio (p.724).
Extra Notes: 1. The educational psychology aspects that are involved with this project are social constructivism (peer feedback) and possibly ZPD as discussed in the literature review.
Method
Participants
This quasi-experimental study incorporates the use of convenience sampling. Students will be assigned through convenience based on course registration. The sample includes undergraduate educational candidates at Kent State University. There are eighty students (N=80) within the four course sections. (20 students in each class section.) The participants will be randomly assigned into pairs. The number of males and females and the ethnicity of the population are unknown. The age of the students is eighteen years of age or older. All students will perform the feedback process as part of the class. Data will only be collected on students who consent to the study. The target population is the undergraduate candidate who utilizes a form of peer feedback as a means of achievement when creating an electronic portfolio in a web-enhanced course.
Operational Definitions
The key variables in the study are the methods of peer feedback, which include, face-to-face, e-mail, discussion forum, and blogs, and student achievement. Face-to-face is a peer feedback method of allowing students the opportunity to work face-to-face to discuss and use a rubric to provide written feedback on paper. This is measured by having the pairs of students meet for 20 minutes approximately a week before the project is due and use a rubric to write comments. The other methods of peer feedback employs the use of asynchronous computer mediated communications (CMC.) “A new form of feedback is emerging with the expansion of the Internet” (Tuzi, 2004, p.217). The asynchronous computer mediated communications allows for “delays in communications and interactions take place at different times” (Yanes, 2004, p.266) allowing students to provide feedback at anytime or place. E-mail is a method CMC peer feedback, which will allow the students the opportunity to provide feedback by using the online course rubric and e-mailing. This is measured by having students provide at least one e-mail feedback related to the online project rubric at least 24 hours prior to the due date of the project to the student and instructor. Discussion forum is a method CMC peer feedback, which will allow the students the opportunity to provide feedback by posting the web link to their project and using the online rubric to add comments and suggestions through the use of discussion threads. This is measured by having students provide at least one discussion thread containing feedback related to the online project rubric at least 24 hours prior to the due date of the project. Blog is a method CMC peer feedback, which will allow the students the opportunity to provide feedback by using a web log (blog) to post the link to the project and the contributing peer feedback by commenting. This is measured by having students provide at least one comment to the blog that contains feedback related to the online project rubric at least 24 hours prior to the due date of the project. Student achievement is based on the grading with the use of each online course project rubric. The first 5 projects are worth 15 points and the final project is worth 25 point for a course total of 100 points.
Materials / Instruments
The instruments used to gather data are the instructor created course web site rubrics for each of the six projects to creating the electronic portfolio. The purpose of the rubrics are to provide participants with guides for both formative assessments of peer feedback and the summative assessment of the final grading. The methods of peer feedback, which include face-to-face and the different forms of computer mediated communication: e-mail, discussion forum, and blogs, were used as a means for students to provide feedback. The discussion forum will be provided by PHP, the blog by blogger.com, and the students grades located on a server based grading system at www.folioweb.org which allows the instructor to grade and add feedback for sections of the projects that correlate to the rubric.
Procedure and Design
Students are provided a rubric for each project that is placed on the course web site Students will use the rubrics as a basis for peer feedback. The researcher will use a convenience sampling method. The students will be paired into groups randomly. Each pair will provide feedback according to the assigned method (Face-to-face, e-mail, discussion forum, or blog.) The first class will use a face-to-face method. The students will be reminded to use the course rubric on the course web site to evaluate the projects. The students will be provided approximately twenty minutes during tri-weekly lab sessions. The second class will use e-mail as a method of peer feedback. The students are required to use the course rubrics on the web site. When the project is complete, the student will e-mail their partner informing them that the project is ready for feedback. The peer reviewer will reply to the e-mail with feedback and forward it to the instructor. The third class will use a discussion forum (e.g. Moodle) as a method of peer feedback. Each pair of students will have a private password-protected discussion area. Each student is to post his or her electronic portfolio project link when the project is completed, making it convenient for the peer reviewer to locate the project. The fourth class will use a blog (e.g. www.blogger.com) as a method to provide peer feedback. The students will be reminded of the course rubrics located on the course web site. Each student will post the link to the project when it is completed making it convenient for the peer reviewer to locate the project. The peer reviewer will provide peer feedback by adding comments. The instructor will confirm peer feedback before the deadline. All feedback must be complete before the due date of the project. Each project will be graded by the instructor using the identical rubric. The instructor will use the Folioweb (www.folioweb.org), an online server based grading system. The Folioweb allows the instructor to complete an evaluation by giving feedback. The feedback provided on the rubrics will be collected and analyzed into categories to determine the length of feedback.
References
Ahn, J. (2004, April). Electronic portfolios: Blending technology, accountability &
assessment. THE Journal, 31(9),12-14. Retrieved March 3, 2005 from Academic
Search Premier.
Benson, A.D. (2003, Winter). Assessing participant learning in online environments. New
Directions For Adult and Continuing Education, 100, 69-78. Retrieved March 3, 2005
from Academic Search Premier.
Cafferella, R.S., & Barnett, B.G. (2000). Teaching doctoral students to become scholarly
writers: the importance of giving and receiving critiques. Studies in Higher Education, 31(1), 39-52.
Chang, C.C. (2001). Construction and evaluation of a web-based learning portfolio
system: An electronic assessment tool. Innovations in Education and Teaching
International, 38(2), 144-155. Retrieved March 7, 2005 from EbscoHost.
Ford, M.L. (2002, Fall). Preparing students for assessment in the on-line class. New Directions
for Teaching and Learning, (91), 77-82. Retrieved May 5, 2005, f rom: EbscoHost.
Gold, S. (2001). A Constructivist Approach to Online Training for Online Teachers.
Journal of Asynchronous Learning Networks, 5(1), 35-57. Retrieved March 6,
2005, from OhioLINK.
Hebert, B.G., & Vorauer, J.D. (2003). Seeing through the screen: Is evaluative feedback
communicated more effectively in face-to-face or computer-mediated exchanges?
Compuers in Human Behavior, 19, 25-38.
Mason, R., Pegler, P., & Weller, M. (2004). E-portfolios: An assessment tool for online
courses. British Journal of Educational Technology, 35(6), 717-727. Retrieved
March 4, 2005 from EbscoHost.
Ormrod, J.E. (1999). Human learning (3rd ed.). New Jersey: Merrill Prentice Hall.
Standford, P., & Siders, J.A. (2001, January). Authentic assessment for intervention.
Intervention in School and Clinic, 36(3), 163-167. Retrieved October 22, 2005, from
Academic Search Premier.
Tsai, H., Lowell, K., Liu, P., MacDonald, L., & Lorh, L. (200?). Graduate student perspectives
on the development of electronic portfolios: part two. TechTrends (48)3 56-60.
Tuzi, F., (2004). The impact of e-feedback on the revisions of L2 writers in an academic writing
course. Computers and Compostion 21 217-235. Retrieved August 31, 2005, from
EbscoHost.
Williams, E. (2004). Incorporating the Use of E-mail into a Language Program.
Computer Assisted Language Learning, 17(1), 109-122. Retrieved March 6,
2005, from OhioLINK.
Yanes, M.J. (2004). Distance education in traditional classes: A hybrid model. The Quarterly
Review of Distance Education, 5(4), 265-276. Retrieved August 31, 2005 from
EbscoHost.
