Rethinking Standardized Testing
Rethinking Standardized Testing
Rethinking Standardized Testing
Designed a research-backed assessment system that achieved ~10× higher accuracy than traditional standardized tests while remaining scalable across cohorts.
Company
Freelance
Year
2024
Role
Product Designer
Research
Research
Figuring out what I don’t know I need to know
Using first principles thinking, I came up with a list of fundamental questions that I needed to answer before brainstorming solutions.
Figuring out what I don’t know I need to know
Using first principles thinking, I came up with a list of fundamental questions that I needed to answer before brainstorming solutions.
Why do we need standardized testing?
To measure students understanding of a subject.
Why do we need standardized testing?
To measure students understanding of a subject.
What does it mean to understand something?
To understand something is to have a working mental model of that concept in your head. Once we truly understand something we can comfortably explain it.
What does it mean to understand something?
To understand something is to have a working mental model of that concept in your head. Once we truly understand something we can comfortably explain it.
What are the most effective methods of understanding?
Feynman Technique: To understand something it must be broken apart and boiled down to its basic essence. That means questioning the subject, learning about it, understanding how it applies and being able to explain it in a simple manner. Finally, if you can't explain it or answer questions that come up, then you need to repeat the process to fill in your knowledge gaps.
What are the most effective methods of understanding?
Feynman Technique: To understand something it must be broken apart and boiled down to its basic essence. That means questioning the subject, learning about it, understanding how it applies and being able to explain it in a simple manner. Finally, if you can't explain it or answer questions that come up, then you need to repeat the process to fill in your knowledge gaps.
What are the most effective methods of remembering?
Active Recall: The act of retrieving information from our brains. Asking, "what were the key points talked about?" after class is active recall. In essence, it’s about quizzing yourself on what you've learnt.
Spaced Repetition: Our brains learn more effectively when we test our knowledge of a subject at spaced intervals. (See image below)
What are the most effective methods of remembering?
Active Recall: The act of retrieving information from our brains. Asking, "what were the key points talked about?" after class is active recall. In essence, it’s about quizzing yourself on what you've learnt.
Spaced Repetition: Our brains learn more effectively when we test our knowledge of a subject at spaced intervals. (See image below)
Solution
Solution
What system could incorporate my research findings and serve as a viable replacement?
I suggest an ameliorated flashcard system where students take a quiz on what they’ve learnt every day. This facilitates active recall and based on how well a student answers, the system uses spaced repetition to repeat variations of the question over time until it’s statistically clear they've understood each topic.
What system could incorporate my research findings and serve as a viable replacement?
I suggest an ameliorated flashcard system where students take a quiz on what they’ve learnt every day. This facilitates active recall and based on how well a student answers, the system uses spaced repetition to repeat variations of the question over time until it’s statistically clear they've understood each topic.
How is this system a more accurate measure of understanding?
In the current system, students test once a year, meaning we have one data point by which we validate comprehension of the entire syllabus. This system quizzes students on each topic 5-10 times a year. Meaning we 5-10 data points from which we can build a more accurate comprehension model whilst also facilitating better retention and understanding.
How is this system a more accurate measure of understanding?
In the current system, students test once a year, meaning we have one data point by which we validate comprehension of the entire syllabus. This system quizzes students on each topic 5-10 times a year. Meaning we 5-10 data points from which we can build a more accurate comprehension model whilst also facilitating better retention and understanding.
How do we prevent cheating?
Using smartphone features like screen recording, microphones, and cameras, we can use AI to moderate quizzes.
Alternatively, considering quizzes will be held at schools, we could place cameras in classrooms as a precaution, and have teachers act as moderators.
How do we prevent cheating?
Using smartphone features like screen recording, microphones, and cameras, we can use AI to moderate quizzes.
Alternatively, considering quizzes will be held at schools, we could place cameras in classrooms as a precaution, and have teachers act as moderators.
Vetting The System With User Interviews
I felt the solution was too much in my head, so I pitched it to students, teachers, and founders to see their responses, and synthesized my findings onto this affinity map:
Vetting The System With User Interviews
I felt the solution was too much in my head, so I pitched it to students, teachers, and founders to see their responses, and synthesized my findings onto this affinity map:
User Flows
First, I decided the MVP should be an app because phones are the most widely used devices, that even students in developing countries can access. From there, I created user flows of how the system would function.
User Flows
First, I decided the MVP should be an app because phones are the most widely used devices, that even students in developing countries can access. From there, I created user flows of how the system would function.
UI Design
UI Design
Style tile
In theory, this platform would be used by the entire student population, so I knew the app needed to meet very high accessibility standards, so I based my design on the adjectives minimalism and google. That's why my colour palette is black & white.
Style tile
In theory, this platform would be used by the entire student population, so I knew the app needed to meet very high accessibility standards, so I based my design on the adjectives minimalism and google. That's why my colour palette is black & white.
Designing the MVP - UI Design
After I had organized the priorities of each screen and made a mood board. I visualized iterations of how each screen would look and functioned before sketching them out on paper. Therein, I had a structure to work with for my high-fidelity mockups.
Designing the MVP - UI Design
After I had organized the priorities of each screen and made a mood board. I visualized iterations of how each screen would look and functioned before sketching them out on paper. Therein, I had a structure to work with for my high-fidelity mockups.
Learnings & Next Steps
Learnings & Next Steps
User Testing In Developing Countries
A cost-efficient way to test the MVP and circumvent the need for dev-time is to replicate the user flows into an analogue system and trail it in a developing country such as India. Whereby, instead of hosting quizzes on an app, students answer quizzes on paper. Testing like this, in adverse conditions, would help up flush out the kinks in the system.
User Testing In Developing Countries
A cost-efficient way to test the MVP and circumvent the need for dev-time is to replicate the user flows into an analogue system and trail it in a developing country such as India. Whereby, instead of hosting quizzes on an app, students answer quizzes on paper. Testing like this, in adverse conditions, would help up flush out the kinks in the system.
My Learnings
Great questions approximate great solutions: Mapping out extremes, such as the most effective study techniques and what extraordinary students do differently, revealed universal practices for better results.
Figuring out what I didn’t know I needed to know was the hardest part.
Deliberate practice will make your life way easier: UI/UX design needs to be approached like a craft.
Read deliberately: Early on, I lacked direction. Defining clear goals would have saved time, though it’s challenging when you’re unsure what to look for.
My Learnings
Great questions approximate great solutions: Mapping out extremes, such as the most effective study techniques and what extraordinary students do differently, revealed universal practices for better results.
Figuring out what I didn’t know I needed to know was the hardest part.
Deliberate practice will make your life way easier: UI/UX design needs to be approached like a craft.
Read deliberately: Early on, I lacked direction. Defining clear goals would have saved time, though it’s challenging when you’re unsure what to look for.
READY TO COLLABORATE?
READY TO COLLABORATE?
READY TO COLLABORATE?
Have some work in mind?
Have some work in mind?
Let’s create something extraordinary together
Let’s create something extraordinary together