Exploring Relevance and Contributions in Data Management

EMA

Your critical review must include the following four sections.

1. A description of the topic common to the articles along with an explanation of why this topic is of relevance to the data management community and (if applicable) to your organisation.

500

2. A summary of each article.

1000

3. A comparison, contrastive analysis and evaluation of each article’s contribution to the topic you’ve identified.

1000

4. A discussion of the extent to which the ideas set out in the articles might or might not be used to change policy and/or practice in data management, along with suggestions as to how any change might be incorporated.

250

(word count max – 2500 + 250 = 2750)

(Calvet Liñán and Juan Pérez, 2015)

(Dyckhoff et al., 2012)

(Cope and Kalantzis, 2016)

1. The topic.

Education Data Science (EDS), (Pea & Jacks, 2014), cited by (Cope and Kalantzis, 2016), is the topic shared between the three selected articles:

Calvet Liñán, L. and Juan Pérez, Á. A. (2015) ‘Educational Data Mining and Learning Analytics: differences, similarities, and time evolution’, RUSC. Universities and Knowledge Society Journal, 12(3), p. 98. doi: 10.7238/rusc.v12i3.2515.

Cope, B. and Kalantzis, M. (2016) ‘Big Data Comes to School: Implications for Learning, Assessment, and Research’, AERA Open, 2(2), p. 2332858416641907. doi: 10.1177/2332858416641907.

Dyckhoff, A. L. et al. (2012) ‘Design and implementation of a learning analytics toolkit for teachers’, Educational Technology & Society, 15(3), pp. 58–77.

EDS is an emergent field, which shares the same direction as Business Intelligence (BI), by transforming the data into an asset for management decision support. However, EDS is mainly concerned with the tools and methods to explore the data from educational contexts (Dyckhoff et al., 2012).

EDS, compromises two new subfields of educational research, born from the effort to manage the vast amount of educational data, that is possible to store and compute nowadays through Learning Management Systems (LMS):

Education Data Mining (EDM) – It has more focused on the techniques and methodologies (Calvet Liñán and Juan Pérez, 2015), aiming at the analysis and interpretation of unstructured data such as log files, keystrokes, clickstream data and discussions threads in natural language (Cope and Kalantzis, 2016);

Learning Analytics (LA) – It is more concerned with structured data from data models designed in applications (Calvet Liñán and Juan Pérez, 2015); (Ho, 2015), cited by (Cope and Kalantzis, 2016). Namely, intelligent tutors, games, simulations, and rubric-based peer review (Cope and Kalantzis, 2016);

Both sub-disciplines compute Educational Big Data in terms of:

Purposeful or incidental data;

Varied data types possible to store and analyse;

Accessibility and durability of the data;

Data analytics;

From the chosen organization, school x, point of view, EDS, promises to be of extreme importance by disrupting the more traditional educational landscape. Although, the technology to achieve such outcome, still requires development, and research to comprehend how to better extract pedagogical value out of EDS (Dyckhoff et al., 2012).

By adopting EDM and LA approaches to Big Data (BD), LMS and Virtual Learning Environments (VLE) can serve as Technology Enhancement Learning tools (TEL), by converting educational data into valuable information, which can help the teachers to adapt teaching and learning processes for the best outcome (Dyckhoff et al., 2012).

Concerning the Data Management Community, EDS, is still a young independent scientific subject. However, by being a crossroad between different research fields and technologies, it is deeply interrelated with the data management Knowledge Areas (KA), and it should be of ultimate interest for the data management community.

EDS is the point, where TEL tools converge, by being able to combine BD storage capacity with Cloud Computing (CC) through EDM and LA technologies. Data management KAs are such as Data Architecture; Data Modelling and Design; Data Integration and interoperability; Data Security; Data quality; not to name all KAs. Are profoundly affected and contemplated on Big Data & Data Science as an all; new field of knowledge that aims at new highs but raises at the same time, new concerns regarding Data Handlings Ethics.

2. The articles summary

The article by (Cope and Kalantzis, 2016), offers an overview of the impact, which Big Data technology is having in Education. The study focuses on data, generated by writing as a cross-curriculum medium that evidences the complexities of knowledge representation. It is ideal to be compared between emergent and traditional approaches to education. The study starts by identifying BD in Education in terms of:

Purposeful/incidental recording;

Varied data types;

Data accessibility and durability;

Data analytics;

The authors then introduce the two new emergent subfields of Education Data Science (EDS):

Education Data Mining;

Learning Analytics;

Both, sub extensions of EDS, aim at retrieving valuable information to support knowledge-based decision making.

Three different categories of Educational Data sources in Computer-Mediated Learning Environments (CMLE) are presented:

Machine Assessment;

Structure embedded data;

Unstructured incidental data;

The study then highlights the EDS, potential in designing virtual learning environments, which are able to capture the intricacies of writing learning patterns, disrupting profusely traditional assessment methods by allowing immediate machine feedback. Ending the historical separation between instruction and assessment, pedagogical design becomes “evidence-centred design” (Mislevy et al., 2012; Rupp, Nugent, & Nelson, 2012), cited by (Cope and Kalantzis, 2016).

The authors define classical “assessment argument” as a three-cornered triangle:

Observation;

Interpretation;

Cognition;

However, EDS allows the assessment of both; the artifact of knowledge making, as well as the process, as products. DiCerbo and Behrens (2014), cited by (Cope and Kalantzis, 2016), defines this as a moving from an item paradigm to an activity paradigm.

In order to teach and learn in such environments, there is the need to change professional and pedagogical models. Data analyses will be at some extend a central skill to develop by promoting self-awareness and allowing a better calibration of learning and teaching patterns.

The study introduces the prediction by Warren Weaver. A Third phase of Science, where computers could boost analyses of organized complexity.

In traditionally educational research, mostly within the experimental model, the research design is an independent tool from the object of study. EDS, allows the embedment of both the data collected and the instrument of measurement. Blurring the distinction between data collector and research subject, repositioning the researchers act as data collectors, between instructional designers, teachers and learners.

The article finishes, by addressing three challenges that BG presents for Research and Data Infrastructure:

Data Access and Publishing;

Data Models and Interoperability;

Data Privacy and Research Ethics;

The study by (Calvet Liñán and Juan Pérez, 2015), focuses on the new online approach to education that generate and store large amounts of data, possible to analyse and retrieve information with the input to improve teachers and students performance.

The core of an e-learning environment is the Learning Management System (LMS). An LMS is able to capture enormous amounts of data, from the digital fingertips left by the user. The authors introduce two means to analyse and extract value from such data:

Educational Data Mining (EDM) – more focused on methodologies and techniques;

Learning Analytics (LA) – more directed towards applications;

EDM applies statistical, machine-learning and data-mining methods to dissect educational data to understand and improve the student interaction with the virtual learning environment (VLE). EDM follows a sequence of the steps, Figure 1.

According to Baker et al. (2012), cited by (Calvet Liñán and Juan Pérez, 2015), EDM applications can be categorised into four groups:

Student Modelling;

Modelling to the knowledge structure of the domain;

Pedagogical support;

Scientific research;

One of the issues raised by the authors is the level of expertise, required to master such tools for a professional without a background in data management and data mining.

LMS, incorporate tools that generate customizable statistical reports but often are too basic to offer any valuable insight on their own.

LA is accepted as the measurement, collection, analysis and report of educational data, with the intend of understanding and improving learning outputs by following the same steps as EDM to extract and pre-processed the data into a format, compatible to be analysed and interpreted.

EDM and LA borders are hard to define between both the research fields. There is an overlap of shared interests that focus on improving education quality by analysing huge amounts of data.

Prediction;

Clustering;

Relationship mining;

Siemens & Baker (2012), cited by (Calvet Liñán and Juan Pérez, 2015), recognises five distinctions between EDM and LA:

Discovery;

Reduction and holism;

Origins;

The study finishes by describing some of the challenges regarding the mainstream adoption of EDM and LA to improve learning outcomes:

Lack of data driven culture;

Ethics and personal privacy;

The study by (Dyckhoff et al., 2012), aim at developing a Learning Analytics Toolkit that will equip teachers to correlate learning patterns, user properties and assess and view results using graphical indicators.

Although LMS and VLE are widely used nowadays, that doesn’t mean that learning and teaching output has improved. Often, teachers limit to upload and deliver the same educational content online. Without any pedagogical input, that could truly enhance learning, it is effective to make the content remotely more accessible. The teachers also lack the tools to analyse and retrieve decision-making information, from the huge amount of the data, generated by TEL tools.

Exploratory Learning Analytics Toolkit (eLAT) is the solution proposed by the authors, to support the teachers integrating cyclical research activities; by collecting, integrating and analysing raw data from their VLE, powered by LA and EDM.

For such, the eLAT would make use of indicators, as “a context aware indicator system, which dynamically aligns data sources, data aggregation and data presentation to the current context of a learner (Glahn, 2009), cited by (Dyckhoff et al., 2012). The study then illustrates the Learning Analytical Process, (Appendix 1).

The mining of the pre-processed data;

The visualization of the results;

The eLAT would require as main design goals:

Usefulness;

Usability;

Interoperability;

Extensibility;

Reusability;

Real-time operation;

On a first development stage, the eLAT focused on evaluating different software architectural approaches for LA using different VLE.

On a second stage, on setting the User Interface (UI), using a:

Second iteration – layout and data presentation of the UI;

Third iteration – a qualitative think-aloud study;

First iteration – collection and definition of content;

The UI, in monitoring view, offers the content grouped into four widgets (indicators):

Document usage;

Assessment/performance;

Communications;

The eLAT architecture incorporates three main components:

Indicator framework;

Mining Data Base;

In order to keep the eLAT independent of any particular VLE, a neutral data model was developed that supports all major data types.

The study concludes, by stating that lacks enough evidences to define the pedagogical input, which captured the variables may have. However, the quality of the indicators set, are of great importance for meaningful data visualizations. For such, LA tools need to be designed open, allowing interoperability between different VLE.

3. The articles evaluation

(Cope and Kalantzis, 2016), focus on the impact of BD in Education, by analysing data generated by writing as a cross-curriculum, learning evidence, medium.

(Calvet Liñán and Juan Pérez, 2015), discusses the similarities, differences and time evolution of EDM and LA and their relationship with BD and MOOCs.

(Dyckhoff et al., 2012), offers the plan to develop an eLAT, so that it can bridge some of the many gaps that the teachers, as non-data mining experts, still face nowadays to retrieve meaningful information from raw data.

Conclusion

All three articles focus on BD and the implications that EDS brings with. The disruptive impact of EDM and LA are there in the educational landscape, as paradigm shifters. The great potential that such technologies promise to deliver, such as more personalized teaching, and new educational research perspectives and opportunities, are at the same time, shadowed by security risks and ethical challenges. While all three studies highlight that, more research is still required for BD and EDS to achieve their full potential. It is undeniable, the central role that BD and Data Science (DS) are occupying within the Data Management Community. Therefore, it is not hard to consider that, BD and DS will require in a near future, where a set of Knowledge Areas exclusively devoted to their cause. BD and DS will then become, not just one more KA, but accepted as the next evolutionary step for data management, as all data is Big and possible to make science with.

Continue your journey with our comprehensive guide to Exploring Professional Development Through Digital Reflexive Assignment.

References

Calvet Liñán, L. and Juan Pérez, Á. A. (2015) ‘Educational Data Mining and Learning Analytics: differences, similarities, and time evolution’, RUSC. Universities and Knowledge Society Journal, 12(3), p. 98. doi: 10.7238/rusc.v12i3.2515.

Cope, B. and Kalantzis, M. (2016) ‘Big Data Comes to School: Implications for Learning, Assessment, and Research’, AERA Open, 2(2), p. 2332858416641907. doi: 10.1177/2332858416641907.

Dyckhoff, A. L. et al. (2012) ‘Design and implementation of a learning analytics toolkit for teachers’, Educational Technology & Society, 15(3), pp. 58–77.

International, D., 2017. The DAMA Guide to The Data Management Body of Knowledge (DAMA-DMBOK). 2nd ed.

https://www.dataversity.net/data-governance-and-data-science-what-is-the-intersection/

https://docs.microsoft.com/en-us/azure/architecture/guide/architecture-styles/big-data