.webp)
It’s no secret that data is one of the most powerful tools in an L&D leader’s toolkit. By measuring what learners are thinking, feeling, doing, and producing as they progress, we can connect training to on-the-job behavior and real business outcomes—the ultimate proof of value.
In a recent Training Industry Leader Talk, SweetRush Director of Systemic Solutions John Cleave delved into the practical side of learning analytics, showing how L&D teams can use their existing tech stacks to embed decision points, scenarios, and capstone simulations that capture meaningful progress and performance data.
As always, the L&D leaders in the studio audience asked some great questions. We caught up with John to answer your five most pressing FAQs on how to implement a learning analytics framework.
The answer to this question depends on how you would like to do the evaluation: using an LMS, an LLM, or an LLM in a completely automated process:
LMS: Most LMSs allow you to include modules in which learners upload documents—for example, as assignments—so you can have human facilitators manually evaluate their submissions using a rubric, then enter an assessment and grade for tracking purposes.
LLM: If you seek to use a Large Language Model (LLM) such as Claude or Copilot to evaluate work products, you can have learners submit their work product via the LMS, then feed each document and a rubric into the LLM to return an evaluation, which you can then record.
Completely automated: If you seek to automate the process entirely, vendors such as Questionmark provide this capability and take care of submitting learners’ work products to AI and recording the result for you. Many LMS vendors are working on adding this capability to their platforms as we speak, so your LMS may offer this capability at some point in the future if it doesn’t already.
For live instruction, there are a number of ways to capture learning analytics data:
First, you can distribute a workbook, test, or assignment during class that learners must complete and submit—either during the class or afterward—which is then assessed and their scores recorded.
Second, you can display a QR code on the screen that directs learners to a survey or question-asking platform such as Kahoot!. These platforms typically allow you to access learner submissions so you can capture data on how they answer each question.
Third, you can follow a blended learning approach: Have learners both attend an in-person or virtual class as well as log on to an LMS in order to complete an assignment, work through a simulation, or answer questions, either after the class or in advance as pre-work.
For online self-study, you can embed components such as knowledge checks into the eLearning itself, then use an LMS in order to capture and report learner answers, as most of my examples demonstrated. For the learners, this experience is seamless: They go to the LMS to complete eLearning, and the LMS—at least those that support SCORM cmi.interactions—captures their answers automatically.
In either case, the data points themselves don’t change; it’s a matter of working around the logistics of either a live class or self-study.
Multiple-choice questions are simply a means to capture learner responses for the purpose of analytics. These can be used either in a case study analysis or on their own—say, to assess whether learners can recall concepts. We have often used multiple-choice questions together with a case study by providing a case and having learners analyze it, then answer multiple-choice questions.
For example, in sales training, we might present a customer situation and ask learners what they think the customer’s greatest needs are and what product they would recommend. We’ve also used case studies with long-format responses: Instead of choosing from a set of answers, learners are asked to fill in the blank or write paragraphs.
A rubric is a grading tool that spells out what “good” looks like in regard to a submitted work product; that is, what you will be looking for as evidence that someone has achieved a level of mastery. Rubrics typically have categories of qualities to look for, with each given a proficiency scale—for example, poor, fair, good, and excellent—often with examples.
For instance, in sales training, imagine that you have a learner create a sales presentation for a fictitious customer. You might develop a rubric to weigh the learner’s submission on elements such as coherency of their pitch, appropriateness of their proposed solution, articulation of client needs, strength of the value proposition, grammar, and so on.
Though a rubric is often closely associated with a competency model, the latter is typically a general description of a person’s abilities across many contexts—for example, their skill at asking questions, identifying needs, handling objections, negotiating, etc.—and not closely tied to a particular work product, such as a presentation. So in some respects, a rubric is the application of a competency model in a particular context, usually an assignment or another submission.
We have used a wide variety of tools to create a simulation, depending on client needs and budget. We’ve built simple branching simulations using Articulate Storyline, Adobe Captivate, or other eLearning authoring tool. We’ve also built simulations using specialized simulation tools such as Regis SimGate.
For some clients, we’ve built simulations from the ground up using HTML5 and have used AI video tools such as HeyGen or Synthesia to create lifelike character videos to use within them. We’ve built simulations in VR authoring environments as well. A growing number of LMSs are offering the means to develop AI-based simulations—for example, a simulated role-play where AI generates character responses—and we’re actively exploring that option as well.
(Want more answers to your learning analytics challenges? Check out John’s Thought Leader Q&A with eLearning Industry.)
Capturing meaningful data doesn’t require a massive technological overhaul. As John’s insights demonstrate, whether you’re leveraging your existing LMS, exploring new AI integrations, or designing custom simulations, you already have many of the building blocks to start measuring true job readiness.
If you’re ready to move beyond basic completion metrics and prove the business impact of your training, we’d love to talk shop with you. Reach out to explore the options for a custom learning analytics framework tailored to your learners and organization.