Why this tool?

How do you support busy leaders and teachers to take advantage of the strong body of evidence now available about effective teaching and effective school practices which support learning?

How do we know what is happening nationally so that education agencies can see what the strengths and needs are of our schools, and whether policy is having its intended effect?

The Teaching, School, and Principal Leadership Practices (TSP) Survey Tool is designed to answer both these questions.

The tool that you can access now is the result of an extensive development process, including:

• a review of current and relevant research
• identification of domains (topic areas) to cover
• creating or finding specific questions that exemplified each domain and would be of practical use to schools
• discussion of our work with a strong Advisory Group
• in-house trial with recent teachers to refine the questions for the online trial
• online trial of a set of questions with 38 schools
• psychometric analysis of the results to see how the questions in each domain related to each other, showing that they were giving evidence of a common concept
• refinement of the questions and reducing the total number, using comments made by teachers and principals in the trial, the results of the psychometric analysis, and Advisory Group discussion
• feedback from trial school principals and the Advisory Group on the use of the tool.

A strong Advisory Group

It was essential to have a strong Advisory Group informing and supporting this work. It was also vital to bring together sector group leaders, the government agencies, researchers and professional development providers, to ensure different knowledge and expertise was brought into play, and that we forged common ground.

We thank the Advisory Group members for their stimulating and productive discussions, practical support with the trial, and their enthusiasm about the tool and its usefulness for schools and providing a national picture.

Our Advisory Group met three times as we developed the TSP Survey Tool for its first use in 2017:

• to discuss background papers on the domains we suggested should be included in the surveys, with the research behind them
• to give feedback and try out the draft items we developed to take to the trial, and
• to discuss the trial results and help us finalise the items to include in the survey now available.

Our Advisory Group meets annually to discuss the national report from the use of the TSP Survey Tool in the previous year.

Advisory Group Members

• Robyn Baker (consultant, heading the TLIF team, formerly Director, NZCER)
• Pauline Barnes (Education Council)
• Mere Berryman (University of Waikato; replaced by Therese Ford for one meeting)
• Mary Chamberlain (Evaluation Associates)
• Whetu Cormack (NZPF President, principal Bathgate School)
• Philip Stevens (Ministry of Education)
• Mary Hall (NZSTA)
• Stuart McNaughton (University of Auckland, Chief Science Advisor Ministry of Education)
• Frances Nelson (convenor for the Centre of Leadership Excellence at the Teaching Council)
• Mark Potter (NZEI Executive, principal Berhampore school)
• Jenny Poskitt (Massey University)
• Ro Parsons (ERO)
• Claire Sinnema (University of Auckland)
• Gary Sweeney (NZAIMS, principal Pukekohe Intermediate)
• Denise Torrey (principal Somerfield school)
• Melanie Webber (PPTA Vice President)
• Mike Williams (SPANZ President from 2017, principal Pakuranga College)
• Howard Youngs (AUT)
• Cathy Diggins (Ministry of Education)

38 schools trialled the TSP Survey Tool

38 schools took part in the trial, which ran during March 2017. They gave a sound cross-section of school types and socio-economic decile:

440 teachers (165 primary, 66 intermediate and 209 secondary), 17 non-teaching school leaders (a total of 457 teachers) took part in the survey, and 36 principals.

Feedback improved the TSP Survey Tool

We asked for feedback at the end of each set of questions (by domain), particularly around any questions that did not make sense or where the wording could be improved. We also asked how long the survey had taken and for overall comments at the end. This feedback identified some questions that were not sufficiently clear. This reinforced the need to keep the survey as short as we could — while covering the ground and providing useful specific information. It gave us some assurance that the questions were seen as useful. We got some very positive feedback from principals on the insight they gained from the reports for their school, and their usefulness.

Psychometric analysis gives assurance

Psychometric analysis came next. We did this for three reasons:

• To ensure that the questions we asked in each domain were conceptually coherent, and to check that each domain had a strong association with other domains. The analysis showed that we are able to construct robust scales using combined domains.
• To see whether each domain had a range of questions in terms of how they were scored (in technical terms the ‘difficulty’ level). We wanted respondents to have information about practices that were relatively common, and also practices that were relatively uncommon.
• To check whether the response scales for the questions gave us enough differentiation.

Shortening and strengthening the surveys

The NZCER team used the feedback from trial participants and the psychometric analysis to shorten the surveys. We still include a range of questions for each domain. The psychometric analysis also highlighted the need to change the response scale for the Teaching Practices survey, to provide more differentiation.

We took our shortened version to the Advisory Group for their review, to ensure that the survey would be clear and meet people’s needs. Some further refinements were made.

We then checked that the final set of questions for each domain had conceptual coherence. The psychometric analysis gave us assurance that the survey we provide here gives a valid picture for individual schools, for Kāhui Ako, and at the national level.

We’ll continue to develop the TSP Survey Tool

We welcome any comments or thoughts you have about the Teaching, School, and Principal Leadership Practices Survey Tool. Please let us know how you have used it also! Contact: tspsurveys@nzcer.org.nz

The NZCER development team

• Cathy Wylie: responsible for the School Practices and Principal Leadership sections (Cathy originally co-developed the Educational Leadership Practices survey with Viviane Robinson)
• Sue McDowall and Jenny Whatman: responsible for the Teaching Practices section
• Rachel Felgate: Senior Statistician, responsible for the statistical analysis
• Hilary Ferral: responsible for psychometric analysis and design of reports back to principals (she previously undertook the psychometric analysis and reporting design work for the Educational Leadership Practices survey)
• Heleen Visser: team leader, also responsible for leading the work with the on-line team
• Our school services team: Leah Saunders, Ben Gardiner

Any questions?

Please contact us at tspsurveys@nzcer.org.nz