There are many skills that you’ll need to learn as an instructional designer, but data analysis should stay as one of the sharpest tools in your toolkit. Your ability to gather, manage, and analyze data will help you uncover the root of a business problem, make targeted design decisions, and effectively monitor and evaluate training outcomes.

Data is also the language of the business world. In other words, it doesn’t matter how well-versed in education and theory you are if you can connect this expertise to business goals in your stakeholder’s language: return on investment, key performance indicators, efficiency gains, and other success metrics.

Wondering where to start? Take a look at these five ways to use data to drive your instructional design practice. For each application, I will also share a real world example from my own experience as an instructional designer to show how data supported decision-making.

1. Getting to Know Your Learners

Your learner is at the center of any learning experience so it’s essential to understand their needs and background. You can do this by collecting pre-assessment data through diagnostic tests, surveys, or interviews. You can also gain an understanding of the learners’ pre-existing skills and knowledge related to the subject matter area through a practice known as “skills mapping.” The demographic information you collect - such as age, educational background, cultural context, and language proficiency - will also help you to understand the diversity of your learners so that you can build a more inclusive experience.

Real World Example: While designing a Spanish language course for a multinational company, we discovered from demographic data that many learners were fluent in Portuguese. Portuguese speakers often learn Spanish more quickly due to linguistic similarities, so we decided to create a separate learning path and course materials to ensure the course was equally challenging for this subgroup.

2. Building Robust Learning Objectives

Adult learners are motivated by a sense of mastery. You don’t want to make your learning experience too hard or too easy, but just right - and you can use performance data and benchmarking to find that sweet spot. Performance data is a history of how learners performed in similar situations or assessments while benchmarking allows you to compare your objectives and outcomes to industry standards. Is there alignment? Is there specificity? Is it realistic? Do we have a clear definition of success? If yes, then you’re on the right path.

Real World Example: Here’s an example of how this practice showed up in my work in a slightly different context. I developed a career development matrix at a previous company. We hadn’t collected any data on the newly created roles in the matrix, so I needed to do some industry research to ensure our expectations for each role aligned with the industry standards and that we established a clear definition of success for each role. The goal was to provide an achievable path for employee growth and mastery. Organizational data is often lacking when a robust data collection system or practices does not yet exist. In these cases, you’ll need to look to the broader L&D community and industry experts for a starting point and then start collecting data to inform future adjustments.

3. Choosing Effective Instructional Strategies

Learning analytics are key to understanding what’s happening with the learner when they are interacting with the learning experience and what you can do to improve it. If it’s a course - where are they spending their time? Where do they get stuck? Time on task, quizzes, and engagement levels are all metrics that can tell us more about what’s working and what’s not. Additionally, it’s important to gather feedback data from the learners themselves to ensure that they see the value in the instructional activities and resources.

Real World example: Here’s the hard truth: Learning design is not always in perfect agreement with business needs. On more than one occasion, data might show that there is a lot of value in a specific learning activity or product or that the learners really like it, and the company decides to eliminate it anyway. Frequently, this happens when something is too costly or not scalable. Do your best to communicate the value of each learning experience or feature effectively, but be flexible and adaptable if a difficult decision needs to be made.

4. Refreshing Relevant and Engaging Content

Learning happens when learners are given the right information in the right place at the right time. Review how and when your learners access your content to gather valuable clues about ways to refresh and resurface it to keep them engaged. Similarly, if learners are consistently getting bad results in a certain area of the assessment, this may be a sign that you need to create more supporting materials or revamp the existing curriculum.

Real World example: The marketing team proposed an email campaign designed to showcase popular content from our curriculum library, paired with helpful study tips. The strategy was to lead with high-performing content that was pre-selected by the learning design team and based on data. This content served to capture learners' attention and increase the likelihood that they would open the email and then spend more time practicing on the learning platform.

5. Measuring Impact & Targeting Learner Success

The evaluation data is arguably the most important data set in your whole project because it will show if you have achieved what you set out to do. If you collect data on learner progress via formative assessments during the course, you will be able to adjust content and offer extra support in real time for a more personalized experience. Additionally, pay attention to the high achievers as they can show you best practices to make the most out of the learning materials available. Finally, data from a final or summative evaluation will show you if the learning objectives (business goals) have been achieved.

Real World example: A recurring insight from evaluation data is that learners often perform well on both summative and formative assessments, but struggle to apply their knowledge in real-world situations when they are under more pressure. This was the case with a safe medical administration course for nurses I once designed. Due to the COVID-19 pandemic, the training had to be delivered in an e-learning format, but the evaluation data later revealed that nurses needed hands-on, in-person training to effectively retain and implement the procedures in their actual work environment. Future iterations included an in-person element as soon as conditions were safe to do so.

These are just a few of the ways that you can use data to drive your instructional design process. I recommend the book “Evidence-Informed Learning Design: Creating Training to Improve Performance” by Mirjam Neelen and Paul A. Kirschner to dive a little deeper. Remember, it’s easy to get excited about adult learning theory and the myriad of tools and technology when you’re new to the field, but don’t lose sight of the data as your most valuable roadmap to effective learning experiences.

See you out there!

This blog post was written by: