The Proposed DSS
The Open Academic Analytics Initiative’s (OAAI) Early Alert System is an ideal solution given the school’s limited resources. Jayaprakash and Lauria (2014) explained that the OAAI was a two-year project supported by the EDUCAUSE Next Generation Learning Challenges (NGLC) and was primarily funded by the Bill and Melinda Gates Foundation. The two primary goals of the project were to 1) develop and deploy an open-source early alert system and 2) assess the portability of the predictive analytics models developed.
A pilot system was successfully developed at Marist College during the fall of 2011 and deployed during the spring of 201 Results of 10 trials consisting of 7,344 records in the training dataset and 5,101 records in the test dataset revealed that the model was more than 85% accurate at correctly identifying at-risk students. Three algorithms were compared including support vector machines (SVM), C4.5 decision trees, and logistic regression along four performance metrics including accuracy, recall, specificity, and precision (Lauria, Moody, Jayaprakash, Jonnalagadda, & Baron, 2013).
Portability of the model was tested in two phases. First, the results of the Marist model were compared to the results of a Purdue model previously developed by Campbell (as cited in Dawson, Gaŝević, Siemens, & Joksimovic, 2014). Second, the results of the Marist model were compared to the results of pilots deployed at three ethnically diverse institutions including two community colleges (Cerritos College and College of the Redwoods) and one Historically Black College and University (HBCU), a part of Savannah State University. As reported by Lauria, Baron, Devireddy, Sundararaju, and Jayaprakash (2012) and Lauria et al. (2013), the results of both phases revealed that the model was more portable than the researchers had anticipated despite the cultural and ethnic differences of the environments.
How the DSS Can Help
The DSS will help all involved with making retention and intervention decisions do so more effectively. Being able to predict attrition and having a thorough understanding of what influences attrition not only enables stakeholders to take action but to choose the best action at the most optimal time. In addition, knowing which factors are predominant for a given at-risk student helps identify who in the process should do the intervening and to determine if referrals to outside experts are warranted.
Most Helpful Features of the DSS
As the name of the system suggests, the most helpful feature is that faculty will be alerted as soon as a student enrolled in his or her class gets flagged as at-risk. The current practice of waiting to observe a pattern of at-risk behaviors such as missing or late assignment submissions typically does not happen until well into the semester and, as previously mentioned, is oftentimes too late to do much good. Further, with early alerts, faculty will have more time to devise appropriate interventions such as helping the student find a tutor, providing supplemental resources, or recommending freely available open educational resources that an academically challenged student is likely unaware of (e.g. Khan’s Academy videos, Flat World textbooks, etc).
DSS Type
The OAAI Early Alert System is initially data-driven because training of the predictive model is done by extracting and integrating the historical data captured in the school’s operational systems and learning management system (LMS). Its true power however, lies in its predictive model that conceivably will improve in its prediction accuracy over time making it a model-driven DSS.
DSS Design Specification
The software requirements of the OAAI Early Alert System are listed below. It may ultimately be desirable to invest in dedicated hardware to execute business analytic processes depending on the needs of the school. At present however, a pilot system could be deployed using available hardware.
- Microsoft SQL Server 2005/2008
- Pentaho Data Integration (Kettle) available at http://kettle.pentaho.com/
- Weka Data mining Tool
- Weka Scoring plugin for Kettle
- Pentaho Report Designer (http://reporting.pentaho.com/report_designer.php)
The OAAI Early Alert System extracts student aptitude and demographic data from the school’s operational systems and event log data of student activity and gradebook data from the school’s LMS. Data is integrated and fed into a predictive model developed using historical data that identifies at-risk students. An academic alert report is then generated and used to trigger intervention activities. An overview of the framework is shown in Figure 2.
