Our first assignment is to explore the logic, structure and intent of an analytics problem. I have chosen Course Completions as the key area of exploration for this assignment.
What do you want to do/understand better/solve?
The vocational education and training (VET) sector (career oriented training for those of you who might not understand the Australian Training system) in Australia is undergoing dramatic changes due to government reforms. Many of these reforms are driven by the desire for the VET sector to be a key driver of Australia’s prosperity.
One of the key initiatives offered by the Government is an entitlement model which provides potential students with entitlement payment for publicly funded training places. This payment will be made available to those who do not have an appropriate Certificate III qualification. This has not been well received by all stakeholders in the VET sector who believe reforms such as this will lead to the demise of public education.
Previously, training providers such as TAFE have received funding to offer training places from the Government. An entitlement model changes this creating greater incentive for private providers to seek funding which would normally have gone to the larger public providers. Part of this key reform is a proposed completion model that attempts to correct the low completion rates experienced within the sector. Payments for training places under an entitlement model will mostly be paid on completion. As you can imagine this reform too has been met with mixed emotions and resistance. The Australian Education Union a key stakeholder of the VET sector suggests that an economic reform such as the proposed entitlement model is purely a shift of costs from Government to students. Unfortunately, whether we like or not, these reforms are not going away.
With this in mind, I would like to explore an automated system that provides VET practitioners will data that clearly identifies students that are most at risk of not completing their VET qualification in an online/e-learning environment.
Defining the context: what is it that you want to solve or do? Who are the people that are involved? What are social implications? Cultural?
As a head teacher of a large public college I would like to consider ways in which I can explore a student’s online behaviour and predict those that are most at risk of not completing. This will greatly assist both my own practices and those of the practitioners I work with.
The Australian Education Union explains that completion rates are a problem for the sector and learning analytics offers a potential tool to explore completion rates before the problem escalates and becomes a funding nightmare for the teaching section.
Brainstorm ideas/challenges around your problem/opportunity. How could you solve it? What are the most important variables?
Educational Data Mining enables the VET sector to consider data collected from educational settings and use this data to better understand student behaviour and performance.
All of the students involved in this study are completing a qualification either online or in a blended learning environment using a Learning Management System known as MOODLE. Moodle provides some potential for monitoring student engagement with learning content online. The system will identify login dates and times and in some cases the activities completed and assessments submitted. Unfortunately it cannot provide details of activities completed by the student in an offline environment.
Given that a blended learning environment offers a number of ways for a student to interact with their teacher and other learners the analytics provided in MOODLE should not be considered in isolation. Having said this, the analytics provided in MOODLE can and should be used to assist a practitioner to keep track of learner behaviour if the learning environment has been well planned and due dates for completion of activities and/or assessments can also be taken into consideration. Attendance in other activities such as traditional based classrooms or virtual attendance in a virtual meeting room can also identify if a student is still engaged with the learning environment.
Managing student participation in all of these platforms may present a challenge to the organisation and the practitioners who rely on the systems to provide the information required.
Explore potential data sources. Will you have problems accessing the data? What is the shape of the data (reasonably clean? or a mess of log files that span different systems and will require time and effort to clean/integrate?) Will the data be sufficient in scope to address the problem/opportunity that you are investigating?
Regular records need to be kept to document student attendance, participation and submission of assessment. Educational software such as electronic roll books is used to capture this information. The data collected if interpreted in a timely manner can assist the practitioner to identify factors that might suggest student failure or non-retention in courses however, within the organisation I work for there we do not have access to early warning reporting. Data mining relies heavily on the teaching section or practitioner who must wade through roll books and examine attendance patterns, submissions of assignments and grades. This is time consuming and if not completed by the practitioner themselves it may result in time delays if individual head teachers must analyse hundreds of student’s data and report back to their staff.
Fortunately, MOODLE itself does offer reports that can easily identify student progress and participation. This can highlight potential issues and assist in the process of data mining within individual roll books.
Consider the aspects of the problem/opportunity that are beyond the scope of analytics. How will your analytics model respond to these analytics blind spots?
Given the need to encourage participation and student completions educational data mining methods will provide the teaching section with information that will enable them to determine student attributes in real-time making it possible to identify those most at risk. This is beneficial as teaching can then schedule in additional contact times, support and assistance where necessary.
Learning analytics cannot provide teaching staff with details of which pedagogical support is most effective only knowledge of current teaching theories for the 21st century can accommodate this. The analytics model will need introduce teaching staff to common theories such as heutagogy or peeragogy and guide teachers through a decision process based on analytics and learning theory. A decision tree could be an appropriate solution for this task.
Please note: this post does not include the selection of tools but will be used in later assignments to guide tool selection.