Marie Bocchi and Laetitia Delba from the University of Mons in Belgium.
We are going to present you our work about the observation of teaching practices.
We prepared this talk with a colleague Christophe Bagut and we want to thank you, the organizers
of this event.
So let's talk about the problematic and our context, the education system in French-speaking
Belgium.
The situation is unfortunately quite alarming, both for the students and for the teachers.
Indeed, the education system is considered as rather ineffective with low results compared
to other OECD countries and other Belgian communities.
It is also rather discriminatory towards students from low economic backgrounds.
Results from the program for the international student assessment show a large difference
in terms of performance between students depending on their economic background.
And for the teachers, French-speaking Belgium is characterized by a high turnover with
around 35% of the teachers leaving their job in the first five years.
So these are issues we can see in our system, but these issues also exist in other education
systems.
That's why we wanted to talk with you about that today.
So in such context, teacher training represents a major societal challenge and our team in
the University of Mons in Belgium is responsible for the training for teachers in upper secondary
education.
So there are pre-service teachers who will teach to 15 to 18 year old students.
In French-speaking Belgium, this training is very short, 300 hours after or during a
master's degree.
And teacher education in our department is based both on research and on training.
So that means that teacher trainers have a double act.
They are both trainers and researchers.
That is our case for Laetitia and myself.
We are trainers in the teacher training program I will present you just after that.
But we are also conducting researches on that program.
So the research gets information and data from the teacher training program and this
program is improved thanks to the results of the research.
First I will present you briefly the teaching, the training program.
Teacher education happens in two different places.
Future teachers are trained at university in a lab in which they have the opportunity
to play the role of a teacher before being sent to real classrooms and real schools.
During their training at university, they meet the supervisors to analyze video recordings
of their lessons.
And research conducted by Antoine de Robert-Masur about this training program has shown that
future teachers express difficulties when they have to describe their practices.
So even when they are confronted with the video of their practices, they have difficulties
to reflect on their practices on an objective basis.
They are also unable to justify their practices with valid reasons.
So faced with these challenges, this presentation presents the development of an innovative
observation tool with a dual objective.
A training objective, our aim is to provide each future teacher observed during micro-teaching
activities and internships with the most objective information possible on their practices so
they can improve and reflect on their practices.
And we have also a research objective.
Our aim is also to produce scientific knowledge on the actual practices of future teachers.
So here is the plan of our presentation.
First, we will describe the innovative tool I developed during my PhD thesis in order
to achieve that double objective.
Then we will present you a new observation tool focusing on classroom management and
developed by Laetitia in continuity with the previous tool.
And finally, we will present you an adaptation of the observation tool developed by Christophe
and used in in-service education because we wanted to develop something that can be useful
for the training and the research in both areas, pre-service teacher education and in-service
training.
And we will present you a brief conclusion before answering your question.
So the tool we wanted to share with you today is called Mirror of Teaching Practices.
It was developed during a thesis which was recognized by the Philippe Mestat Prize awarded
by the Academy of Research and Higher Education in Belgium.
And you can consult this tool online with the following link.
Here it is how it looks in a real situation.
So during the micro teaching activities and the internships, the researcher trainer observes
the future teacher's practices during the lesson that the future teacher is giving.
With the observation tool inserted in the observation software, I will show you that
in more details in a few seconds.
The observation grid was operationalized on the basis of the scientific literature and
then inserted into various observation software programs which were subjected to a comparative
analysis to determine which software was best suited to the research.
So if you have questions about that part of our work, we will be happy to answer that
because our team has tested several softwares in order to find a solution that was easy
to use in real classrooms without internet.
It was one criteria we wanted to follow.
We have also published two book chapters on how to create an observation grid and on the
criteria to use when choosing the most suitable observation software for a research.
These two chapters are published in a methodology book, so if you want more information about
that, we can also answer your question.
This type of observation with an observation grid inserted in an observation software is
also used in the field of sports when, for example, the number of places is recorded
during soccer or other sports.
So we use that kind of software in classrooms in order to be able to discuss on an objective
basis with all the future teachers we are training.
That looks like that in a real situation.
After the lesson, the supervisor and the future teacher meet and discuss the video recordings.
The supervisor is able to show the results of his or her observations and the results
are synchronized with the video, allowing precise video clips to be reviewed if necessary.
Just like that, for example, on the feedback line in red, one can see all the feedbacks
of the future teachers that have been observed by the trainer and it is possible to review
all the feedbacks or some feedbacks at some point, but it allows us to navigate in the
video and to discuss some precise video clips with the students.
This coding work is realized in a live situation, so in the micro teaching room or in real classrooms
in secondary schools without internet connection.
It was one of our requests because when we are going in real schools and real classrooms,
we don't have internet, so we wanted to do all that operations without internet connection
with just a camera and a computer for each future teacher participating in our training
program.
And I will let Laetitia show you in more detail the tool and a few results.
Hello everybody, so I'm going to describe a bit the tool that was first used in Marie's thesis.
So, the tool is divided in categories and subcategories.
We have five categories to observe teaching practices linked to classroom management.
We have four categories used to observe teaching practices which are more linked to the management
of the learning process and we also wanted to observe the students.
So, we observed the students' verbal interventions, for example, and all these categories are also
divided in subcategories.
Our aim was to give the most precise feedback possible.
As I already said, the tool is available online and today we are not going to talk about all the
categories which is impossible in 30 minutes, but we are going to focus on two categories.
One to see how the teacher can check the student's understanding and how the teacher can designate,
name students.
So, let's have a look first at checking the understanding.
Scientific literature has shown that checking the student's understanding is an important
teaching practice.
So, we wanted to evaluate to see how teachers were checking this understanding, how they were
finally making something invisible visible.
And to check the understanding, teachers can do it in two ways, in a more stereotypical way or
in a specific way.
And I will a bit describe these two ways of checking the understanding.
When a teacher checks the understanding in a stereotypical way, he uses phrases like
all right, okay, do you understand?
And generally, the answer of the students is yes.
But the teacher doesn't really know if the students have understood the content or the
instructions.
Generally, students simply reply yes, we understand, even if they don't.
So, this is not really something effective if we have a look at scientific literature.
On the contrary, if we check the understanding of the students in a specific way, we use
phrases like can you explain what we have seen today?
Can you rephrase what I've just said?
Or can you reformulate, rephrase what we have seen?
And so, the teachers can see really the elements that the students have understood and the
elements that they don't.
They haven't, sorry.
And so, it is important because the students have the opportunity to show what they understand,
they really understand.
And for the teachers, he understands what is understood.
And so, if there is a problem of understanding, he can already act directly.
Also, it is interesting on the classroom management level because students are, have
to be active in class.
They have to be able to answer the teacher's questions at all time.
And generally, when they are focused on the task, they don't misbehave.
So, this is important to check the understanding in a specific way, both for the learning process
and for classroom management.
We also studied how teachers name students because scientific literature has shown that
there are different ways to designate students, but some of them are more effective.
For example, using a random, an explicit random designation system is more effective than
choosing students among the volunteers.
Why?
Because generally, when you choose students among the volunteers, these students already
know the answer and the answer is generally correct.
But the students who don't know the answer, they don't reply in the class and the teacher
doesn't know if they have understood.
So, here in our program, the tool is divided in six subcategories from designated among
the volunteers to continuing discussions.
And now I'm going to show you some results.
Here, it was during a micro-teaching activity.
We had 15 students, no, 16 students, and we analyzed how they checked the understanding
of their students.
So, we can see that during the micro-teaching activities, most future teachers use the
stereotypical checking for understanding.
And they didn't use the specific checking for understanding or they use it quite infrequently.
We can see that the mean is only one for this category.
But during the internship, after speaking with the supervisor, we can see that the mean
for specific checking of understanding is increasing.
It's not identical to the stereotypical checking for understanding, which is still quite high.
But we have seen that with only two supervisions, future teachers are already improving their
way of checking the understanding.
And something interesting also that we analyzed during the internships is how they name students.
And we can see that in classrooms, most of the time, teachers, future teachers don't
even designate students.
They generally let students speak freely.
And when they designate students, they generally choose the volunteers because it's a very
reassuring to ask the volunteers because the volunteers know the answer.
And generally, the answer is correct.
So, it's quite reassuring for future teachers because they know that some students have
learned something.
So, these are the results.
Something that is also part of our training is a report.
So, our future teachers have to write a report based on the micro-teaching activities and
they get the results of the observation.
And it was interesting for us to analyze these reports because some students, when they saw
their video, they were not sure what they were looking at.
And some students, when they saw their video, they didn't realize that they didn't name
students to answer the questions.
And they only apprehend this idea with the observation data.
So, for example, you can see here we have a quote from one of these reports.
And we have one future teacher and she wasn't aware that she was not naming students, that
she was letting them speak freely.
And when she was choosing, she was only choosing volunteers and she only did that three times,
which is not a lot because it was for a 40-minute lesson.
So, generally, this first version of the tool showed us that future teachers tend to use
more learning management strategies than classroom management strategies.
They don't use a lot of social interventions and when they designate students, most of
them don't do it.
So, they generally let students speak freely and when they designate students, they choose
among the volunteers.
So, with these results, we showed that it was, it highlighted that it was important
to improve our classroom management training.
And LaTuture was pointing in the same direction as classroom management is still considered
as one of the main difficulties for novice teachers.
And so, we decided to create a new version of the tool, which was focusing only on classroom
management this time.
So, we are working a bit differently here.
We are no longer using a camera and a computer.
We are using two tablets.
So, we use one tablet to register, to record the video and the other one to code.
So, we work in the same way.
We created an observation grid based on scientific literature and we use that grid to observe
students in real classrooms with real students.
We did two observations, one at the beginning of the internship and one at the end of the
internship.
And it was, we also recorded the feedback, which was given to each teacher at the end
of his performance.
And the feedback was recorded and it was the feedback between the future teacher, the supervisor
from the university and the co-parenting teacher.
And last year, we used this device to work with pre-pre-service teachers and I'm going
to present now some of the results.
Quite happily, we can see that during both observations, our pre-service teachers implement
quite a lot of preventive strategies, which is positive.
And also, after the first day briefing, we noticed some changes in the practices.
For example, when we had to name students, they avoided using ineffective strategies
and they used effective strategies like the implicit random designation.
And also, they started using more active supervision strategies.
So they were not staying behind their desk, they were moving around the classroom during
group work or individual work and during instructions or group work, they were also more, they were
looking at the class more frequently.
On the other hand, we, there is still some improvement to be made in our training.
For example, we have seen that the corrective interventions are not implemented by future
teachers.
They rely mainly on preventive actions.
So this is something that we have to work on as we have seen that from one observation
to the other, some teachers use less effective and less diversified strategies.
Some were only using verbal recommends, so they were only asking students to stop misbehaving,
but they were not reminding the students of the expected behavior.
So this is something that we are working this year.
There is something that we are improving in our training.
And if you're interested, also, the tool is also available on our website.
If you're interested, also, the tool is also available online freely.
So I will present you briefly a third part of our work, because we wanted to show you
that we are also using these tools in in-service training.
So an adaptation of the mirror of teaching practices was used in that context.
You can consult the complete grid in English on that link.
This time, the aim of this tool is to carry out observations throughout the school year
and serve as a basis for coaching sessions in schools for teachers wishing to improve
their practices.
For example, the results of the case study conducted with this tool show that the teacher
observed on several occasions modifies some of her practices in line with the training
objectives.
Nevertheless, even in in-service training, we see that the teacher has difficulty implementing
some certain practices.
For example, she had difficulties to implement checking, specific checking for understanding,
just like Laetitia showed you before.
Even in in-service training, that kind of teaching practices was hard to implement for
that teacher.
And a good result for us was to see that 77% of the coach advice was followed.
So it was interesting for us to see that we can have an impact on the in-service teacher
practices with a tool and some advice that we could give to her.
So here is a little conclusion.
During this talk, we proposed an observation tool aimed at helping pre-service teachers
to reflect on their practices.
And at producing scientific knowledge, this tool was used successfully in different contexts
for the training and the research.
Results also point out that pre-service teachers show interest in the observation tool to analyze
their practices.
That was one of our goals.
We didn't want that it was only a tool for the researcher or for the trainer.
We wanted that to be useful for the teachers and also for the pre-service teacher in order
to reflect on their practices.
So that goal seems to be achieved too.
We don't have a lot of time today to show you that.
But we have also another research showing that the future teachers are using this tool
in different contexts to reflect on their practices.
For example, when they are writing a report or when they are discussing with the co-operating
teacher.
So it's very interesting.
Another thing is that results strengthen the idea that giving a survival kit to novice
teachers on basic pedagogical practices or even vital pedagogical competencies to ensure
their professional integration and avoid desertion is a necessity.
The results of our research show that we train teachers to some practices, but we can
do even more to train them to be able to manage classroom and the learning process.
And it's very important to avoid that in the absence of training on important teaching
practices.
Some teachers develop these on the job or even do not develop them at all and that students
are put in a difficult position.
So for your information, the last version of the tool we presented you today is also
available in this new book, but it's in French, just in case.
But other publications in English are available and will be available in the next few months
on that website, which was created by our team.
We want to thank you very much for listening and we will be very happy to discuss that
topic with you.
So we are happy to hear your questions.
Great, thank you so much for the presentation.
If anyone wants to ask a question or comment, please, you can use the raised hand function
and I can call on you.
So I can get started.
One of the questions we've been having here, I think, especially in regards to Sean Kelly's
work, is the difference between having observational tools that are very directive, let's say,
this is what you should do and this is what quality looks like, versus tools that are
more descriptive, that just kind of describe a range of practices or characterize the frequency
with which things are happening.
I'm curious if that's a tradeoff or kind of a balance that you've come across in your
work and how you've been able to do that.
And how you feel, if that is how you feel about, you know, where you land on that.
Okay, I'll try to give you a first answer and my colleague will complete me if it's
necessary.
We were facing exactly the same question when we were creating these tools.
And to be very honest with you, I'm not sure to have completely solved that issue.
Let me give you one example.
For example, in our tools, there are some categories that are very precise and that
even if there is some limits to that, but we can say, oh, you have done that practice
10 times, it's not very effective.
For example, the stereotypical remarks that Laetitia showed you before, we know, thanks
to the research that it's not very useful.
And on the other side, we can show you that we did something more effective twice.
For example, that part of the tool is quite, from our point of view, satisfactory, but
there are some limits to that, but I can discuss that with you after that.
But we have also in the tools some categories that are only descriptive because we can have
a tool with 100 categories.
And for example, in my thesis, I only check if the students give feedback to the students,
but it's not very interesting.
So that's why I developed later another tool more precise to say to the students the quality
of their feedbacks.
But it's a real dilemma to have only descriptive statistics, but then what do you do with that
and what can do the teacher with that or to have more qualitative categories.
But the problem with that second part is that you need to be quite objective because if
it's too qualitative, we have also tools with a Likert scale and you have the teaching practices
and you have to say if it's good, very good or poor.
There are a lot of tools like that in our institutions, but it's very subjective.
So that's why we propose that tool.
It's maybe not perfect.
Sometimes we have a lot of criticism like it's too rigid to count behaviors like that,
but it's more objective that tools that are a lot more subjective.
It's also important that we combine the tool with supervision, which means that the results
are interpreted after by the supervisor and we can discuss them with the students and
we can rewatch some parts of the video.
So it's something it's not the observation tool himself.
It can be quite descriptive maybe, but at the same time, we have the supervision and
the ability to rewatch the video.
It's something more qualitative and maybe something more interesting to improve teaching
strategies.
Great, Sean.
Hi, thanks, y'all.
Thanks for the presentation.
You know, related to this question that Mark raised, one thing I heard in the talk was
that you started out a bit more holistic and then narrowed your focus a little as you went.
When you do that, you are partly moving away from sort of overall judgments of effectiveness
because you're saying we're going to focus on these practices.
I really want to talk to you about these practices.
I'm not judging your overall effectiveness.
We're focusing on these things that can be improved.
So there's a way that just doing that alone kind of moves you more towards improvement
rather than judgment.
But the other thing I heard was that you also had a lot of emphasis on as like overall outcomes
or concerns, teacher attrition, teachers leaving the profession early, teachers quitting.
And so along those lines, you know, one suggestion for future work would be that you could more
closely tie your activities to a model related to that where you were looking at the ways in which
intervening variables that come before attrition, you know, things like efficacy, a sense of
reassurance capital, a sense that teachers are well-trained and professionalized, confidence.
You know, those kinds of intervening variables were affected by this process you're going
through because, you know, it sounded as if you wanted teachers to feel like that observation
process they went through built them up and reassured them that they, you know, made progress
towards, you know, really engaging in expert pedagogical practices.
So, you know, I could see, you know, your research, you know, you're very clearly focused
on novice teachers. It sounds like you have some very clear outcomes in mind and, you know, the
logic of the observation could be even more narrowly tailored around those things. Does
some of that make sense or was I reading too much into things?
No, it's interesting. Maybe we wanted to show you a big picture of our problematic with the
attrition, but we are still working on the improvement on some of certain teaching practices.
So, the diminution of attrition is one of our goals, but it's a big goal, not very close to us,
but we are working in a very short training and our main goal is to improve
some teaching practices and we have some models that we can share with you today because the
time was short, but we have some models that what we wanted to observe and what we wanted
our future teachers to be able to do in some context, to be able to do in some context,
to observe and what we wanted our future teachers to be able to do in some context
and judging with the students if it's appropriate or not. For example, for the classroom management
part, we have like several practices, very important, that we wanted to train the teachers
and after to see if they are able to implement them in classrooms and finally, we hope
it will have an impact on attrition, but it's not really our main goal. We share that with you
to give you a big picture, but our main focus in that research is to train teachers to certain
practices and to see if they are able to implement them after coaching sessions. I'm not sure if it's
more clear like that. Okay, thank you. I mean, but that does change things then because
you know, then it is sort of a more inherently judgmental process because you're saying,
you know, we've decided for this teacher, this classroom, this is what they should be doing.
And we're trying to get them to do that. And that is different than saying we're going to put the
teachers through this observation process and we're going to see if the teachers then themselves
feel more competent at the end of the observation process, more confident, more capable going
forward. That would be a different way of looking at it. Thanks. But I think maybe it's not,
we have here, you can see I'm doing my best with the English language. So maybe I'm a little
maladroit. Clumsy. Maybe I'm a little clumsy when I share with you what we are doing because I don't
think it's dichotomous. I think we can have some idea of what practices should be interesting to
teachers and the teachers too. And it's more like a dialogue between them and us. It's not
just us that impose one way to do something. It's like we propose them all sorts of practices
and they are looking what can fit their context and their students. So maybe we are closer,
the two options are more closer than what I was trying to explain.
Okay, thank you. Haralambos? Hi everyone. Thank you for a very interesting presentation. I really
like the way that you're building on analyzing teaching and you're trying to scaffold
pre-service teachers. I have a question that relates to how you decompose teaching practice.
So in what you showed about classroom management, you decompose classroom management in several
different components. And I think that's really important in terms of supporting
teachers, especially pre-service teachers, do this hard work even when it comes to classroom
management. But I'm wondering if there are also any drawbacks of this very detailed decomposition,
if you encounter any drawbacks and if working at a less detailed level of this decomposition
could also be helpful. This is not a critique. I'm just thinking, what I'm asking you is just
to reflect on what you have been doing and share with us any affordances and limitations of this
decomposition. So when building the observation tool first, we did a review of scientific
literature to see which were the effective practices focusing on classroom management.
And after we made choices, because it was impossible to evaluate everything,
there are things that are more observable than others that we can observe more easily.
For example, we know that it's important that a teacher has high expectancies for the students,
but it's something that it's not really easy to observe with a grid. So this is something
more general that we can appreciate at the end of the lesson, but it's not something that is
inserted in the tool, because in the tool we wanted something more that are more easy to
observe. For example, how to meet students, or are the teacher using positive social strategies like
saying hello, saying thank you, encouraging students. These things are something that we
find in the tool. And based on all this, we can give more general feedback. But we had to make
choices because it was impossible to have everything, and we had to take into account
also the technical difficulties and the limitation of the softwares.
I don't know if it's a solo question. Your question.
Thank you. I was also asking whether, you know, this very detailed decomposition, if it also has
certain limitations in addition to the affordances that it apparently has in terms of
guiding pre-service teachers.
But in fact, your question is a big question in observation. You can read that question in some
books. It's the question between a grid with a few categories, but it doesn't give you a lot of
information. Basically, for example, you can see an observation tool with two categories,
such as the teacher talks or the teacher doesn't talk. But with that, you can say a lot of things.
And you have very big tools with 100 categories and that are not really manageable. So, we try to
find a compromise between the two types of grids in order to have sufficient information to discuss
with a pre-service teacher. So, in our point of view, from a trainer perspective, that time,
it's sufficient for us and we can discuss with these tools with our students. And they are not
too short and not too long in order to allow us to have constructive conversations with them
in schools just after they were doing their lesson. So, we have also time constraints.
For example, we have to give feedback to the future teacher in around 10 or 15 minutes. So,
the tool has to be manageable. One other manner to answer your question is that we are more and
more working with what we call modules of observation. So, we are not always observing
everything. So, we have a big tool with, for example, five modules. That's what I show you
with the new book. We have seven modules and it depends on the objective of the trainer or
of the trainee. And they can even discuss before the observation, what do you want me to observe
next time? Do you need me to give you advice on your classroom management or on the way you check
understanding? And you can have, it's another solution, a big tool separated in several modules
and you focus on one module or another depending on the needs of your training. But I'm not sure I
answer your question. Yeah, that's fine. Thank you. My pleasure. Great, thank you. Next question.
Yes, my name is Stefan Graf from Denmark. Thank you for your presentation and this
interesting discussion about the principle problematic dilemma of using such observation
tools. My question is rather simple. I am just struggling with understanding how are you using
it in teacher education? Who is actually doing the coding? Is it researchers or is it the
counselor or is it the students themselves, the student teachers? And I didn't really get
who is doing the rating because I think a big part of learning to notice must be trying to
look closer and using such coding systems. So, as I understand it, it must be a rating from the
researcher or the counselor. So, that was my question. Who is rating actually?
Okay, I will give you a first answer and then my colleague will complete. In fact, that's
what I was trying to explain in the beginning of my presentation. It's particular in our context
is that Laetitia and myself, we are both researchers and both counselors in that program.
So, basically, it depends on the school year but we have each year between 20 and 40 students
and we are doing the observations in our department. But we know that we have a specific
profile because we are also researchers. So, we are really trained to do that kind of coding.
So, we also wanted to train other trainers. We didn't want that the tools we presented you today
were only researchers' tools. That's why we have different projects. We couldn't show you
everything today but we have also another project, for example, with Canadian trainers
that are using our tool but they are no researchers but they are using our tools
to train other people. And the last part of my answer is that we are also using the tool
in order that the teachers themselves are coding their own videos. So, we have another
training but it's an in-service training in which each teacher has to record himself or herself
a movie in the classroom and to code with the same tool. So, we are developing that kind of
observation for different people and not only researchers. Yeah, thank you for that. Because
I was just wondering because we are using, you know, the students doing their own observation
and coding and then they do compare the codings between each other and then learn to observe
is our intention. That's just what I was interested in that. Thank you.
Okay, great. Thank you. It looks like we have time for maybe one more short question.
Okay. One thing I'm always interested in this sort of work is do you have cases where teachers
don't agree with the rubrics kind of view of what is the right thing to do?
And kind of, I'm just curious if that's come up and how that you resolve that.
Yes, it's a very good question. I think and my colleague can add this on her own experience.
I think we have very few remarks from the future teachers or the in-service teachers because
they are wanting to improve and they are in research of concrete advice. So, to be honest
with you, it's almost 10 years I'm working on that topic and I have never disagreed with a teacher
or a future teacher on that kind of tool. I have received a lot more criticism from other researchers
or other trainers that disagree with some part of the literature we were using. For example,
the literature on effective teaching, it's not really popular in French-speaking Europe. So,
we have more debate with colleagues than with teachers, to be honest with you.
And when we go to classrooms, generally, it's rather accepted that we work with the video and
then we code. And most of the time, we have the cooperating teachers for more part on classroom
management. We make the same comments because it's something you notice. We notice basic things like
students misbehaving or the future teacher welcoming students or the future teacher naming
all the students in the class or just naming the volunteers. So, generally, we agree on the
main elements of the observation tool. Yeah, great. Thank you so much for presenting today.
The next seminar series will be in March. I don't think we've finalized the date or I think we
actually just finalized the date, but there'll be an email sent out about what date it is
as well as the dates for April and May in the coming days. Thank you so much for joining us
today and we hope to see you again in March. Thank you very much. Thank you. Bye. Thank you.
Thank you.