One of the great flexibilities that LLMs give people is the ability to use Natural Langauge to access structured and semi-structured information. This can extends to even more abstract concepts and comprehesion assessment.

“Quiz Me” is an application designed to leverage LLMs and the Natural Language processing to assess a persons comprehension of complex text.
The goal is to help add more fluidity to the comprehension assessment and be able to “meet the student where they’re at” in terms of language capability. If the goal of comprehension is understanding, then the mechanism for conveying it can be more flexible.

The key enabler for this is the ability to use much (MUCH) longer contexts to support complex texts, complex question-answer chains, and an ability to focus prompts

What is it?

  • an application that uses LLMs to allow a reader to respond to comprehension questions in a “long form” format

How?

  • a book/text is uploaded and an LLM builds an understanding of it (e.g. major themes, characters, locations) on a chapter-by-chapter and book-long basis
  • the understanding is then turned into a number of questions for each chapter (question bank)
  • a reader is then presented question by question on the major themes of each chapter with the LLM assessing for “correctness”
  • if an answer is “incorrect”, the LLM asks elaborating questions until the response is “correct”

Core Components

Text Processor

  • uses an LLM to parse and process the uploaded text
  • parsing includes;
    • identifier creation for each chapter
    • question generation
    • question-identifier relationship patterns
  • all of the above is persisted for later use

Test Coordinator

  • presents questions bank
  • tracks progress through chapters and comprehension
  • persists entire sessions and answers

Code

(Quiz-me)[https://github.com/willschipp/quiz-me]