The collaborative STEPS research project started in 2014 and is being carried out by a team comprising Indigenous and non-Indigenous partners on both sides of the Tasman. The team is led by Dr Amohia Boulton and Associate Professor Margaret Cargo (University of Canberra). A fuller description of the STEPS research has previously been posted (refer below).
As previously reported, mixed-method concept mapping methodology was used in 2016 to brainstorm practices and strategies with participants that support culturally safe evaluation. The strategies generated were successfully consolidated, in the first half of 2017, resulting in a final set of 106. In the next phase of the research, we engaged participants in sorting these 106 strategies into conceptually meaningful groups or clusters. The rating phase of the study followed and involved each of the strategies being rated on their relative importance and achievability. Approximately 400 participants, on both sides of the Tasman, were involved in the STEPS data collection which was conducted in a series of waves using small group, one on one and online methods. Data collection concluded late in the year with preliminary analysis beginning prior to this.
Using multi-dimensional scaling and hierarchical cluster analyses, concept mapping software generated concept cluster maps for each country. Aotearoa New Zealand team members further analysed the NZ map with Australian team members similarly analysing the Australia data. During January 2018, we recruited a small group of Indigenous and non-Indigenous study participants in Aotearoa New Zealand to contribute to refining the NZ map. Refinement included reviewing cluster groupings and the cluster labels generated by the concept mapping software as well as determining thematic map “regions”. Following discussion with the Project Advisory Group, in February 2018, the research team has consolidated the final cluster maps.
The 12 cluster Australia map reflects three thematic regions: (1) An Evaluation Approach that Honours Community; (2) Core Heart of the Evaluation; and, (3) Cultural Integrity of the Evaluation. The 11 cluster New Zealand map reflects four regions: (1) Authentic Evaluation Practice; (2) Building Māori Evaluation Expertise; (3) Integrity in Māori Evaluation; and, (4) Putting Community First. Differences between the maps are being explored in the ongoing analysis along with commonalities. Later in 2018, results of the research will be shared with evaluation conference participants. Translating results into practice will be a key focus of dissemination taking in to account how resources can best be harnessed to ‘grow’ evaluation that works for Indigenous communities.
