Blog 4: Results, Observations, and Reflections
by Jenny Krystopowicz, Van Ness Elementary teacher
In my first three blogs I asked you to think about a time during childhood or more recently, when you made, invented, or altered something and the challenges that may have occurred during your making experience. It is likely you used some capacity of critical thinking to meet your end making goal. I also asked you to think about how you would rate yourself while making. What measurement did you choose to objectively measure your success? What boundaries prevented you from assessing yourself fairly? Perhaps you found that your self-assessment of something abstract, such as making, can be extremely difficult.
Learning While Making is a method of teaching I use in the classroom to support the development of critical thinking and give students the opportunity to think about their successes and challenges encountered. Learning While Making integrates the recommended strategies to support and measure critical thinking in the learning environment since it encompasses targeted, purposeful hands-on making experiences that are designed to engage students in multi-sensory and interdisciplinary learning. It empowers students to take on the role as a maker where they build, create, design or produce while developing transferable skills that will improve learning across all disciplines.
Lens 1: Perseverance
Before students started Stage 1 of the Making Stages, they had to complete Angela Duckworth’s Grit Scale.
Duckworth’s research focuses on two traits that predict achievement: grit and self-control. Grit is the tendency to sustain interest in and effort toward very long-term goals (Duckworth et al., 2007). Self-control is the voluntary regulation of impulses in the presence of momentarily gratifying temptations (Duckworth & Seligman, 2005; Duckworth & Steinberg, 2015).
Students were asked to rank themselves on the following questions using a scale 1-5:
Question 1: New ideas and projects sometimes distract me from previous ones
Question 2: Setbacks don't discourage me. I don't give up easily.
Question 3: I often set a goal but later choose to pursue a different one.
Question 4: I am a hard worker
Question 5: I have difficulty maintaining my focus on projects that take more than a few months complete
Question 6: I finish whatever I begin
Question 7: My interests change from year to year.
Question 8: I am diligent. I never give up.
Question 9: I have been obsessed with a certain idea or project for a short time but later lost interest.
Question 10: I have overcome setbacks to conquer an important challenge.
While Duckworth notes that these scales were originally designed to assess individual differences rather than subtle within-individual changes in behavior over time, I am having students complete pre and post reflections to identify instances of perseverance (a trait of critical thinking) while making. For example, I will examine if a student thinks he or her showed an improvement that demonstrates staying on task and not giving up until a solution is formulated, a process is determined, or a decision is reached.
Pre-Assessment vs Post-Assessment Results and Observations:
It was my hypothesis that student’s perseverance scores would improve from pre to post; however, this was not the case. As demonstrated in the data, there was a decrease in every domain. In addition, 7 out of the 9 students’ overall Grit score was lower than their pre assessment score. While the initial pre scores were high, 3.9 out of 5, the post average was .4 less.
I can conclude that this happened for several reasons. One, I believe students initially scored themselves high because they were thinking about things and experiences in their everyday life. Whereas when taking the post assessment, I asked students to specifically think about making in the classroom. The making experience was fresher in their minds, and perhaps since there were some challenges along with a clear structure in place, students felt like they were not as successful compared to other personal experiences. Second, the questions in the grit assessment can be used on a broad scale and apply to many areas in life. As a result, these questions did not effectively capture the overall making experience for my students. Last, more time is needed to expect students to show adequate growth in a critical thinking skill. The assessments were taken 4 months apart, and while some growth can happen, I believe it would be beneficial to assess students over a longer period.
Lens 2: Reflective Capacity
To measure Reflective Capacity, I used the Critical Thinking: Drawing Conclusions rubric from Wasisabi Learning. I scored each component of the rubric by analyzing completed student document sheets from Stages 4 and 6. I referred to different sections of the document sheets from the stages and matched it with a component on the rubric.
For example, in Stage 6 students had to answer:
How did receiving peer feedback support your project?
What was the most enjoyable part of making?
I used these responses to give students a score in the Making Summaries component. Students scored a 4 if they offered a clear and concise summary/ evaluation of the process. Their reflection had to be considered, deliberate, and appropriate, and clear next steps were provided.
EXAMPLE OF STUDENT RESPONSE:
“I know what other people see and not just what I see”
Students also had to answer in Stage 6:
What did you learn about your authentic self during your making experience?
How will you apply this learning to other areas in school and your life?
I used these responses to score Knowledge Expansion. Students scored a 4 if they provided creative ideas for expanding and allying knowledge, evaluating each for feasibility. They had to identify areas of further research and suggests next steps.
EXAMPLE OF STUDENT RESPONSE:
“The making stages make sure I am stable and I don’t just change my mind in the middle.”
In Stage 4, students had to answer:
What did your partner say that was the most helpful to you?
What feedback will you include in your revisions?
I used their responses to score Useful Failure. Students received a 4 if they showed awareness of mistakes and areas for growth and improvement. They also needed to critically evaluate all feedback and create and execute next steps to improve and progress.
EXAMPLE OF STUDENT RESPONSE:
“To adjust the frontside of my project and double check my mistakes”
Last, in Stage 4 students must also give feedback to their peer’s authentic self-project by:
Telling something you liked:
Asking the innovator a question:
Giving the innovator a positive suggestion:
I used their feedback sheets to score Reflecting on the Process. Students scored a 4 if they had critical reflections that are task-focused, and which enable development of themselves or their peers, or refinement of processes and structures.
EXAMPLE OF STUDENT RESPONSE:
“Can you clarify why there is green paint on the top right corner of the project?”
“You did a great job of making the project calming and durable”
Results and Observations:
Students scored highest in Stage 6, Making Summaries, as this was their final opportunity to reflect on the entire making experience and speak about their most enjoyable parts of making. Students scored the lowest in Stage 4, Reflecting on the Process, which was one of the first opportunities students were given to provide feedback to their peers. Initially, some students were hesitant or shy when giving peer feedback, so the dialog was limited. However, with more time and practice this routine will become a norm.
Reflections:
While it is evident the GRIT survey is a very valid snapshot of one’s own perception of self-perseverance, it became apparent while conducting my research that using this scale is not the best method given the reasons indicated.
As a result, I have created a new scale that will measure one’s perseverance specifically while making. This survey, which has been designed to be given at the beginning and at the end of the school year will be similar to the grit scale in that it’s a five point scale which will allow students to express how much they agree or disagree with a particular statement. The questions I created were designed to correspond with each making stage (all 6) which naturally fosters the development of critical thinking. You can see where each question has been matched with a specific stage. Having this pre and post survey data enables me to determine if after a year of making, where students are following the structure of the making stages, improved their own perception of their critical thinking skills.