Introduction
Your platform allows you to measure the effectiveness and prove the impact of learning through the use of smart questionnaires.
This article explains how you can enable learning evaluation at course level and set the evaluation properties.
Enabling the learning evaluation for courses
To enable evaluation for one or more courses, go into the Navigation menu > Content and delivery (folder icon) > Learning evaluation.
While on the Courses tab, click on the plus button in the top right corner of the page.
In the Enable learning evaluation on courses right panel, select the courses you would like to enable learning evaluation for. Use the filters and the text search to drill down the courses by type and language.
Press Next to proceed.
Set the evaluation properties of the evaluation process. Superadmins can pre-configure the default evaluation properties in the Evaluation designer, establishing the evaluation options proposed by default when activating the learning evaluation for a course in the Manage menu, where these default settings can be applied or adjusted on a course-by-course basis.
-
Evaluation process structure: Configure the evaluation process by enabling or disabling the proposed questionnaire types. Select a questionnaire for every questionnaire type you selected from the dropdown menus. Click on Preview questionnaire to see the respondent's view of the questionnaire. The available questionnaire types are:
- Learner initial questionnaire: This questionnaire cannot be disabled; it is always part of the evaluation process, for all courses. Select which questionnaire to use for the evaluation process of the selected courses.
- Instructor initial questionnaire (for Instructor-Led Training (ILT) courses only)
- Learner follow-up questionnaire
- Manager follow-up questionnaire (for ILT courses only)
Please note! The ILT standard questionnaire is only available for ILT courses, and the e-learning standard questionnaire is only available for e-learning courses. All the custom questionnaires you created are available for both e-learning and ILT courses.
-
Confidential data: Set whether the data collected from questionnaires is confidential or nominative. This setting is available only when enabling Learning Evaluation for a single course in the right panel. When you select more than one course, this setting is hidden and the confidentiality configuration is inherited from the evaluation process properties for the chosen methodology (ILT or e-learning).
- Confidential: Nobody can identify which respondent has provided a particular answer.
- Nominative: Superadmins can identify which respondent has provided a particular answer. Power Users with permissions for learning evaluations (Learning impact) can identify which respondents have provided a particular answer for the users assigned to them, meaning respondents' first and last names will be visible in reports.
Click Confirm to finalize the enablement of the learning evaluation for the selected courses, which are now listed in the Courses tab of the Learning evaluation main page.
Please note! The number of enrollments displayed on the Courses tab reflects the total number of users enrolled in the course and can include learners enrolled before learning evaluation was enabled for the course.
Outcome of the learning evaluation enablement
All the learners
- enrolled in e-learning courses before the evaluation was enabled
- enrolled in ILT sessions and events created before the evaluation was enabled and having an end date after the evaluation was enabled
will receive an email linking to the learning evaluation questionnaire after they complete the e-learning course or any session of the ILT course for which the evaluation was enabled and according to the timeline set for the questionnaire in the Evaluation designer.
Unenrollment scenarios
- Course instructors remain synchronized with the learning evaluation process unless they are removed from their associated ILT sessions before all learner enrollments in the related course are deleted from the Enrollments tab. No audit trail event is recorded when instructors are removed from the learning evaluation process.
- You can exclude learners from the evaluation (for example, those who did not attend the course) by unenrolling them from the course before you enable learning evaluation.
- For ILT courses, you can unenroll learners from a course where learning evaluation is enabled, (Courses/Course title/Enrollment tab/Unenroll). In this case, they are removed from the learning evaluation process and stop receiving the scheduled emails. But if you unenroll learners from a session, (Courses/Course title/Sessions tab/Session title/Enrollments and evaluations tab/Unenroll) they will remain in the evaluation process and continue to receive the scheduled emails.
- When you enable Learning evaluation for a course that includes learners with the Completed enrollment status, those learners will not receive the scheduled Learning evaluation questionnaires. You can unenroll the learners and enroll them again. They will then receive the scheduled Learning evaluation emails when they complete the course or when you manually set their enrollment status to Completed.
Managing the evaluation properties of a course
When enabling learning evaluation, you can configure the properties as outlined in the previous chapter, Enabling the learning evaluation for courses. However, after the learning evaluation is enabled, the only properties that can be adjusted are the enablement or disablement of questionnaire types (that is, adding or removing the follow-up questionnaire, the manager evaluation or the instructor evaluation). This means that:
- Once a questionnaire is selected for a course, it cannot be replaced with a different questionnaire but you still have the option to modify the questionnaire's content directly through the Evaluation designer. If you do so, the changes will apply to all new enrollments across any course where this questionnaire is selected, and not to existing enrollments.
- You cannot change the confidential or nominative status of the evaluation.
To enable or disable the evaluation questionnaires for a course, from the Courses tab of the Learning evaluation main page click on the ellipsis menu at the end of the course row and select Change properties.
Remember that:
- The learner's initial questionnaire cannot be disabled.
- The changes will apply to all new enrollments across any course where this questionnaire is selected, and not to existing enrollments.
- The evaluation properties on the Manage section can be modified at the process level using the Evaluation designer. However, for adjustments to take effect, properties must be changed directly via the Learning evaluation > Manage submenu and they will only apply to users being enrolled after that change.
Managing multiple courses
You can change the properties of multiple courses with a single action by selecting the checkboxes next to the courses’ names from the main table and clicking on Choose action in the bottom right corner of the page. From the menu that opens, select Change properties.
If you select both e-learning and ILT courses, you will see two different steps in the right panel:
- Properties ILT: Manage the properties of the selected ILT courses.
- Properties e-learning: Manage the properties of the selected e-learning courses.
Best practice. If you need to change the questionnaire for only certain courses that originally used Questionnaire A—while retaining Questionnaire A for other courses that also use it—you will need to create a new questionnaire B and a new course B, and enable the learning evaluation selecting this new questionnaire, for this new course B, specifically for the courses requiring the update. This approach allows you to implement changes selectively without affecting other courses that continue to use the original Questionnaire A.
Disabling the learning evaluation
At any time, you can disable learning evaluation from the Courses tab of the Learning evaluation main page by clicking on the ellipsis menu at the end of the course row and by selecting the Disable learning evaluation option.
As an option, you can disable the learning evaluation for multiple courses with a single action by selecting the checkboxes next to the courses’ names from the main table and clicking on Choose action in the bottom right corner of the page. From the menu that opens, select Disable learning evaluation.
When you disable the learning evaluation for a course, that course is removed from the table in the Courses tab.
Please note that:
- When you disable evaluation for a course, evaluations will not be proposed for newly enrolled learners. The data collected up to the moment you disable it is kept in the platform, and the planned evaluations continue. To cancel the learners’ evaluation, unenroll them from the session or course while the learning evaluation is enabled for the course.
- You can reactivate learning evaluation for a course where it was previously disabled, which will automatically reinstate the originally selected questionnaires. This means that you will only be able to enable or disable questionnaire types (follow-up, manager, instructor), but you will not be able to change the selected questionnaire, nor can you alter the evaluation's confidential or nominative status.
Troubleshooting
The following tips can help troubleshoot common problems with learning evaluation questionnaires.
Sending restrictions
The evaluation questionnaire is not sent to:
- The learners who have completed the e-learning course or ILT session (enrollment status = Completed) before the evaluation is enabled.
- The learners enrolled in the sessions that ended before the learning evaluation activation.
- The learners who completed the evaluation included as training material in the course (it is the same questionnaire).
Evaluation questionnaire training material not available
Learners can also access the evaluation questionnaire through in-platform notifications created after the course completion or if the questionnaire is a course training material.
The learning evaluation training material (also known as the feedback questionnaire) may display one of the following error messages to learners:
- Learning Impact Questionnaire will be available once the course has been synchronized. The learning evaluation for this course is not yet available. Learners may return later or await the email notification
-
Sorry this questionnaire is not available, please contact your administrator for support. The learner's enrollment cannot be found or created when opening this questionnaire. The administrator must review the learner's enrollment. When the learner is not enrolled in the course, opening the feedback questionnaire training material triggers an automatic enrollment solely based on the username. This allows learners who were not meant to evaluate the course to complete the initial questionnaire. However, for these automatic enrollments, no emails, reminders, or follow-up evaluations will be sent. In addition, names and emails will not appear in the raw data extractions, as the platform does not store this information. To prevent these automatic enrollments:
- Make sure that the feedback questionnaire is activated or added to a course only after the learning evaluation has been enabled.
- Ensure that users are enrolled in ILT sessions or e-learning courses only after the feedback questionnaire has been added.
- For enrollments created before enabling the learning evaluation, verify that users’ enrollment status in the ILT session or e-learning course is Enrolled or In progress (not Waiting or Completed) at the moment the learning evaluation is activated.
Missing enrollments
If user enrollments were missed due to synchronization issues:
- For e-learning courses, unenroll and re-enroll them in the course, but do not use the Archive and re-enroll option when archiving the enrollment.
- For ILT courses:
- Unenroll them from the session and enroll them in a session created after the learning evaluation was enabled.
- Archive the learners’ enrollments in the ILT course by using the Archive and re-enroll option.