Introduction
Attainment analysis is the part of Smartgrade where you can see question, distractor, topic, part and overall analysis for your assessments.
In this section you can:
- Easily spot patterns using heat maps
- Check out the summary rows to spot gaps in knowledge
- View question text from within analysis
- Use distractor level analysis to understand student misconceptions in multiple choice questions in online tests
- Use topic level analysis to review areas which may need re-teaching
- Review the standardised scores and percentile ranks in the Overall Results tab
- Benchmark your school against others
- Aggregate, group and refine using the options and filters button
- Export your reports to CSV
For information about permission levels in analysis, please see this help article.
Heat maps
All of our reports have heat maps by default and dark colours signify positive outcomes. They enable patterns to be spotted easily - maybe a student has struggled with the test, or a question has been particularly challenging.
Of course, you can turn the heat map off if you like; there is a slider towards the bottom of the page where you can control this.
Summary rows
Towards the bottom of all our reports are summary rows.
For instance, in the Question Level report, potential marks for each question are displayed along with the average that the group attained with the percentage of unanswered questions. You can easily identify questions where students have struggled. Take a look at the next tip for viewing the question text from within the report.
On the Question Level report, questions can be ordered by clicking on the icon next to the word Average. In the example below, the 'less than' arrow has been selected, meaning that questions are now ordered by percentage correct in ascending order.
Summary rows expand to cover relevant categories if you use, for instance, 'Group by...' in OPTIONS & FILTERS.
View question text
If you have identified a question where students have not performed so well you can view the question text by hovering over the question name at the top of the analysis page - no need to rummage around looking for test papers any more! Details of any topics associated with the question are also displayed. Note: The publisher needs to have entered question text / topics in Smartgrade for this to be available.
Distractor level analysis
For online assessments with multiple choice questions (MCQs), the Distractor Level report displays a table of students' answers to 1 mark MCQs.
The correct answer is recorded as option A; then the heatmap highlights more common misconceptions through darker colours. The summary rows at the bottom of the table show the percentage of students selecting each option. Hovering over the question name reveals the question text, along with the correct and incorrect answers. Similarly, hovering over the table displays the individual student's response:
The table can be aggregated by class, school and all schools (other schools can be displayed on the table if you are taking part in trust wide assessment):
Topic level analysis
Where questions have been assigned topics by the publisher, the Topic Level report groups the marks students have attained by their topic. Questions can be assigned to more than one topic, so the marks may not add up to the test total. Marks can be shown as numbers or can be changed to percentages by using the slider towards the bottom of the page. Using the summary rows can help identify curriculum areas which you may choose to re-teach to your class.
You can compare how your school or MAT has performed in each topic in comparison to all other schools taking part in the assessment. Click here for more information.
If there is a topic that scored lower than others (bearing in mind topics have varying potential marks), you may want to investigate which questions were associated with it, with a view to possible re-teaching. Click on the OPTIONS & FILTERS button towards the top of the screen. Click on the Filter tab and select the appropriate topic from the list.
If you go back to the Question Level report it will only display questions which have been assigned to that topic. Don't forget you can hover over the question name to see the question text.
Overall results
The Overall Results report displays several columns of data. The first two display the total raw scores and the raw percentages achieved by the students. The columns under the heading of 'School/MAT' are calculations based on the sample of students from within your school or MAT. These columns may be hidden (click the cog at the bottom of the page to manage displayed columns). If the columns are empty then the School / MAT standardisation has not yet been run. This help article explains how complete this process. Once calculated, the report will display a Smartgrade, or Smartgrades, (if enabled) which are school or MAT defined grades. You then get the Percentile Rank of the student within the assessment (where the top percentile is 100 and the bottom is 1). The student is also given a standardised score, where the average is 100. Most students will get a score between 70 and 130. This help article goes into more detail about these terms and how the calculations are done. If the assessment is a global assessment (i.e. published by one of our partners) then the 'Global' columns will be displayed. Again, these columns may be hidden, click on the cog towards the bottom of the screen to manage displayed columns. Dates of when we are running our global standardisations (and therefore, when you can expect to see data in these columns) can be found here.
Each of the columns can be sorted by clicking on the column header. In the screenshot below, I have ordered my students by Raw Score.
This analysis report is from one of our global assessments and I can see that my highest attaining student (scoring a percentile rank of 100 in my MAT) got a global percentile rank of 89. This means my highest attaining student is a little way behind the highest attaining student in the national sample. Having said that, as an average (see the summary rows), my students, as a group, achieved a global percentile rank of 72 - so is well into the top half of all schools who took part.
Comparing your school to others in the sample
For schools using one of our partner assessments, or if you are part of a MAT that has created a shared assessment, you can see how your school has done against all other schools taking part in the assessment at question, topic, part or overall result level. Simply click on the At drop-down towards the top of the page and select Other schools (or specific schools if they are from your MAT).
You will see your school ranked with all the other schools you selected. School names will only appear if they are part of your MAT, other schools are anonymised. The data will be aggregated, so depending on which report you are looking at there may be average scores by question, topic, part or even percentile rank or standardised score.
Towards the bottom of the screen, you can see how many schools took part in the assessment and increase the page size to enable more schools to be seen if required.
The data can be further aggregated to show your organisation (school or MAT) performed against all others. To do this, click on the OPTIONS & FILTERS button towards the top-right of the screen then click on the Aggregate by drop-down and select All schools.
This aggregation can be used on any analysis report, even at question level, so you can see, in detail, where your cohort has performed less well (or better than!) other schools taking part.
Options and filters
The OPTIONS & FILTERS button enables you to aggregate, group and filter your results on any of the reports. Top tip: if there is a little red dot on the left-hand side of the OPTIONS & FILTER button it means that a condition has been applied.
To clear any filters, aggregations or groupings, click on OPTIONS & FILTERS and select RESET towards the top of the slide-over.
In the OPTIONS & FILTERS button, the first tab is Options. This is split into two sections:
1) Aggregate - here you can Aggregate by class / school / all schools. This is particularly useful when you are using the grouping option, as you can see averages by group rather than seeing students' names.
2) Group - there are a couple of options here. Group by grade will show different options depending on what you have set up and calculated in your assessment.
When I group, for instance, on Percentile Rank (MAT) I can see my students in deciles:
If I were then to aggregate by school, I see the same data, but now see how many and what percentage of students are in each decile:
This kind of grouping works particularly well if you have set up School / MAT Smartgrade. Below I have set up performance indicators for my practice SATs papers. I have grouped by Smartgrade (School / MAT) and aggregated by school. I can see what percentage of students are working at "Expected" and at "Higher Standard":
Beneath Group by Grade is Group by Demographic. This is where you can drill down into which groups may not be attaining as well as others. This works either aggregated, where you'll see averages by the group, or disaggregated, where you will see students' names. The following screenshot is grouped by gender and has been aggregated by school:
In this example, I can see that there is an even number of boys and girls who took the assessment, but that the girls have outperformed the boys on this particular test.
If I now add the other schools in the MAT (or 'Other Schools' if you are using a global assessment) using the At drop-down towards the top of the screen, I can see how each school has performed by gender. Towards the bottom of the table, the summary rows display the averages of the table above.
Towards the top of the Options & Filters is a Filter tab. Earlier we mentioned that one of our favourite filters is the Topic filter where you can filter to see questions that relate to a certain topic.
Beneath this is Filter by characteristics. Here you can select which students you wish to include or exclude from your analysis. Multiple filters can be applied here, either with the report aggregated or with students' names showing.
Don't forget if you get a little lost, or you can't remember which filters you have or have not applied, the RESET button at the top removes any filters, groupings or aggregations so you can start from scratch.
Export to CSV
Pretty much every screen in Smartgrade is exportable to CSV. You can then use this download to import data into other software (don't forget about the Bulk Export feature for large amounts of data for import into other platforms) or to manipulate even further in Excel.