Engagement: Mastery Oriented Feedback

Lesson Pages

Sizing Up the Barrier

When it comes to success in the classroom, research (see Marzano, Pickering, Pollak, 2001, Ch. 4) has demonstrated that students are likely to attribute their success to one of four things:

Luck
Other people
Natural ability
Effort

For example, if a student fails a math test, he may attribute his failure to

Bad luck declaring that he was unlucky (e.g. "the questions happened to cover mainly the material I was least expecting! If only....")
Others (e.g. "the professor just doesn't like me. In fact, I don't think he likes people in general")
Natural ability (e.g. "I'm just not a math person. Nothing I can do about that") or
Effort (e.g. "I should have paid attention in class and studied, and I didn't").

Of course these same four could be used in reverse if the student succeeded on the test.

Of these, students have control over exactly one: effort. Promoting the degree to which students believe they are capable of significantly influencing the outcomes of their coursework, career trajectory, etc. relates to the element of social cognitive theory known as "self-efficacy" (Bandura, 1986). Broadly, self-efficacy refers to the degree to which an individual feels empowered and equipped to effect the outcome of their situation or endeavors. When people believe that they can meaningfully influence the outcome of something that matters, they are far more likely to apply themselves. In other words, when students feel that they can grow and achieve based on the merits of their own efforts, they are more likely to work hard and thus their belief becomes self-fulfilling.

The problem is, not all feedback contributes to developing student sense of self-efficacy, and some feedback may be counterproductive (see the works of E.L. Deci). Without realizing it, faculty may be expending effort to provide feedback that is either ineffective or counter-effective.

Scenario

Three students sit apart. One female student at left wonders "Am I improving?" the male student at middle looks distressed while thinking "this just isn't for me," and the female student at right slouches down, thinking "maybe I should drop this class."

Sara is taking a statistics course for the first time. She was both nervous and excited for the opportunity to learn a new skill at the beginning of the semester and worked hard to try to understand the challenging new concepts. The first few homework assignments have come back marked up extensively with numerous errors reported (via "x" marks) and her scores have been desperately low. Now, just a month and a half into the course, she is considering dropping it. She feels as though she just isn't "cut out" for statistics.

Javier is taking a class in cultural anthropology, a class he knew would stretch him, but he wanted to try it because he is considering majoring in anthropology. The course hinges on a long-term project that enables students to demonstrate the culmination of skills learned during the semester. Javier applies himself with rigor, but "checkpoint" submissions are simply marked as "complete" with no further feedback. When Javier finally submits the final project, he scores poorly and determines that anthropology must not be for him.

Yoon He, an international student from Korea, was excited by the opportunity to grow in her ability to analyze and discuss English-language literature in a 100-level American literature course. In the first essay assignment, students were charged with presenting a literary analysis of The Crucible using one of a few critical theories. Yoon He was excited to turn in her work and gain feedback regarding her ability to analyze literature in her second language. When the feedback arrived, however, almost all of the feedback was correcting grammar errors that had more to do with syntactic differences between Korean and English than commentary on her analysis and opportunities to strengthen it. Though she kept on in the course, she didn't feel like she knew if she was improving in the very element (literary analysis) that was her purpose in joining the class in the first place.

Feedback in each of these cases was ineffective or counter-effective. Rather than encouraging Sara, Javier, and Yoon He to achieve higher degrees of mastery in their respective subjects, feedback was distracting, demoralizing, or otherwise resulting on an emphasis on personal qualities (e.g. aptitude) rather than encouraging increased self-efficacy and sustained effort.

How can instructors provide more effective feedback that leads to sustained engagement for diverse students?

Essential Concepts

As a further element to frame this lesson, it is important for participants to share an understanding of what is meant by "effective feedback."

According to CAST's UDL Checkpoint 8.4 (2013), effective "feedback ought to be relevant, constructive, accessible, consequential, and timely." These elements are central to providing feedback that is "mastery-oriented"; that is, feedback "that guides learners toward mastery rather than a fixed notion of performance or compliance... and emphasizes the role of effort and practice rather than 'intelligence' or inherent 'ability' as an important factor in guiding learners toward successful long-term habits and learning practices."

In this lesson, instructors will have the opportunity to explore specific methods and tools both on Canvas and elsewhere on the Web to facilitate effective, efficient, mastery-oriented feedback.

To begin, it may be helpful to articulate what is meant by the different qualities of effective feedback. According to the body of research curated by CAST (2013), effective feedback is:

Table 1. *Criteria for effective feedback and descriptions thereof.*
Criteria	Description
Relevant	Relevant feedback is feedback that underscores aspects of the work that can be improved to bring the student closer to the objective skills/knowledge that the assessment was designed to assess. For non-example, in the scenario of Yoon He, the feedback that was focused on grammar was not relevant in regards to the stated goal of demonstrating skill in literary analysis.
Constructive	Constructive feedback focuses on what a student can do to improve/make progress going forward. Identifying errors and giving comments for how to correct them is far more constructive than simply marking things wrong. For nonexample, in the scenario or Sara, the red X's without any explanation or support for improvement didn't give Sara a clear way forward.
Accessible	Accessible feedback is feedback that can be interpreted and utilized by the student. For example, providing print written feedback to a student with a visual impairment is inaccessible and therefore ineffective. Accessibility of feedback will depend on the individual case.
Consequential	Consequential feedback is that which has value to the edification and improvement of the student. For example, formative feedback on a project when the student still has time to improve the final product is more consequential than feedback delivered when the student no longer has any opportunity to improve.
Timely	Timely feedback is feedback that is received as quickly as possible such that a student's mind is still able to adjust to new information based on work they had submitted. For example, a student who has a logical misconception in a math class who is corrected early prevents that misconception from "gelling" in their mind. A student who does not receive timely feedback may complete numerous tasks with the misunderstanding and thus lose points and find it more difficult to learn the correct reasoning.

A theme that resounds through the above qualities of effective feedback is that it is based on the premise that students can and should improve based on the provided feedback. While this may seem like an obvious point, the reality is that often feedback (like that noted in the common scenarios above) does not lead to this outcome. If feedback does not lead to student improvement, then what's the point?

Big Idea

The value of feedback is directly proportional to the degree to which it leads to potential for improvement. If feedback does not lead to student improvement, it is a waste of everyone's valuable time. When feedback is designed so as to move students closer to mastery, it is called "mastery-oriented feedback".

Now to the practical. How can instructors use the features of Canvas (and other Web-based tools) to support them in providing mastery-oriented feedback? A great deal of this depends on how instructors use the following tools in such ways that reflect the concepts discussed above. Armed with those concepts, this next section shows how they can be put into practice.

Methods
1) Canvas "SpeedGrader"

SpeedGrader is Canvas' simplified, native tool for grading and providing feedback on assignments, quizzes, and discussions. Instructors can see the Canvas guide that provides an overview of the tool and another than provides instruction as to how it is used. It can be used with or without rubrics (which are addressed next!). The SpeedGrader always enables instructors to give wholistic feedback quite quickly. Detailed feedback (e.g. grammar correction) is possible in SpeedGrader automatically if students have uploaded a file submission in one of the following formats: .pdf, .doc/.docx, or .ppt/.pptx. See the Annotated Comments Guide for more information about detailed feedback in SpeedGrader.

Pro Tip: Detailed Feedback via Canvas and Google Drive

If an assignment calls for detailed feedback, instructors should assign students to submit work via file upload in .pdf, .doc/.docx, or .ppt/.pptx format. SpeedGrader will then provide options for commenting on the document itself. Alternatively, instructors can encourage students to submit links to their work shared via Google Drive.

SpeedGrader always allows instructors to provide feedback via a text-entry box, file attachment, or audio/video comment. Though it is not "traditional," many instructors find that leaving audio or video feedback enables them to provide more natural, personal and complete feedback more quickly; it is more like talking about the work rather than writing notes about it.

2) Canvas Rubrics

Rubrics can be created using the "outcomes" tab in a given course page and then add a rubric to assignments (this method is especially helpful if a given rubric will be used for more than one assignment). Alternatively, instructors may make a rubric at the same time they create an assignment using the same methods described in "add a rubric" above, except that the new rubric is made on the spot instead of being located and attached. Once a rubric is attached to an assignment, it will be available for use in the SpeedGrader.

Pro Tip: Why and How to Design Quality Rubrics

There is ample research regarding rubrics, which generally demonstrate their effectiveness in variety of outcomes for students in higher education. What is clear is that rubrics support students by providing clarity in how work will be graded and support instructors by providing a faster and more focused way to assess student performance. As a bonus, the tiered and differentiated nature of rubrics may allow for students to quickly assess which areas warrant the most work to develop. For example, a student may see that he is doing well in the "organization" criterion but needs to do more to develop his "quality of content." For more information about designing quality rubrics, see this excellent PowerPoint (Tomei, 2015).

3) Canvas Quizzes

According to the Canvas Guide on quizzes, "Canvas has four different types of quizzes:

A graded quiz is the most common quiz, and Canvas automatically creates a column in the Gradebook for any graded quizzes you build.
A practice quiz is ungraded and can be used as a learning tool to help students see how well they understand the course material. Practice quizzes do not appear in the syllabus or Gradebook.
A graded survey allows you to give students points for completing the survey; however, it is not graded based on right or wrong answers.
An ungraded survey allows you to obtain opinions or other information from students; however, students do not receive a grade for their responses. Ungraded surveys do not appear in the syllabus or Gradebook."

Each of these has a wide range of options for customizing how students may interact with the quizzes. For example, instructors may control how many times a student may attempt the quiz, time limit, and if/when students see the correct answers. These tools can be used thoughtfully to match the function of the quiz (Assessment For, As, or Of Learning?) and to support mastery-oriented feedback. For example, when cheating is not a threat (e.g. students cannot see others' screens or when the quiz is intended as a pre-assessment), it may be best to provide students with immediate feedback (c.f. the "timely" quality of effective feedback). This not only benefits the students who are corrected with immediacy, but also saves time for the instructor who can see student results (including analytics), but did not actually have to grade anything.

4) Web-based Options

We already mentioned using Google Drive as a great place to provide formative feedback on written work. But the Web is teeming with tools for assessment. Here are just a few that instructors may consider.

Table 2. *Tools for formative feedback and descriptions thereof.*
Tool	Description
Socrative	Socrative allows instructors to quickly assess students with prepared activities or on-the-fly questions to get immediate insight into student understanding. It then uses auto-populated results to determine the best instructional approach to most effectively drive learning. Instructors may also find this demo helpful.
EdPuzzle	EdPuzzle transforms video-based learning with interactive tools that engage learners, deepen understanding, and track progress. Teachers, trainers and instructional designers use EdPuzzle to quickly add images, text, and questions to existing online videos.
Quizpedia	Quizpedia is a tool for making multimedia quizzes which provide instant and detailed analytics. However, what makes it really unique is that it is designed in such a way as to encourage instructors to have students make the quizzes themselves.
Kahoot!	Kahoot! Works best for in-person/blended learning settings. Students quiz in realtime with engaging competitive features (individuals or teams). It is an excellent tool for review of content/concepts, pre-testing the class, or other informal functions. It provides immediate analytic data.
Mentimeter	With Mentimeter you can use live polls, quizzes, word clouds, Q&As and more to get real-time input - regardless if you’re remote, hybrid or face-to-face.

Summary

Providing mastery-oriented feedback means giving feedback in such a way that students are likely to use it to make actual, practical gains. While this may seem obvious, many of the common practices of providing feedback do not lead to this end. Research has shown that, to be effective and mastery-oriented, feedback ought to be relevant, constructive, accessible, consequential, and timely.

Canvas has a number of tools that can be used to support instructors in providing quality feedback, and there are also numerous tools on the Web for designing engaging useful assessments to support teaching and learning. However, no tool is itself effective. The work of designing mastery-oriented feedback depends quite entirely on the instructors' decisions regarding those principles that govern effectiveness (relevant, constructive, accessible, consequential, and timely.) When feedback is used well, all students - regardless of where they began - can not only demonstrate their learning, but learn, improve and grow because of the assessment.