Rogers, McEwen and Pond have written a chapter on “The Use of Web Analytics in the Design and Evaluation of Distance Education” (Chapter 12) in the Veletsianos (ed.) book.
Web analytics are defined here as “the measurement, collection, analysis, and reporting of Internet data for the purpose of understanding and optimizing Web usage.” (from Web Analytics Association 2005).
This chapter discusses the ethical standards that apply to tracking usage in this way. A simple “for instance” example that goes back to my earliest days in online education is that it would be wrong to assume that a student who visits a course reading frequently is actually more engaged in the material than another who has visited it only once. Perhaps the second student printed the reading. A much larger picture that doesn’t identify individual users is what is of value in web analytics – a “conglomeration of data from hundreds, thousands, and even millions of users”… allowing for the analysis of trends that can inform alterations to the website being analyzed.
So this is where a careful determination of the Key Performance Indicators (KPIs) is required to ensure they relate to outcomes with meaning (and remember outputs are not outcomes). Key Educational Requirements (KERs) guide this. Know what it is you want to track and know the reasons why this information is of value to you.
The chapter describes a case study out of Brigham Young University. The KERs in this case related to quality and individualization and patterns that emerged related to peak usage times and some navigation issues, with plenty of room for further analysis.
By the way, the first sentence in the abstract of this chapter is “One main challenge that has faced distance education since its inception has been a relative lack of knowledge concerning how students actually interact with the materials”. The authors compare this to the face-to-face learning environment where educators are seen to be more aware of how students interact with materials, but I’d argue that in many cases (large lecture halls in postsecondary come to mind) the things that students do to learn, whether face to face or at a distance, are not well understood.
But new technologies, for instance those which analyze social media, are coming available that can give us insights to any number of trends and I am optimistic about these.
Here are some of my past blog posts on related topics:
This is from the “Elizabeth Tweets” series of chapter-by-chapter blog posts related to the new book “Emerging Technologies in Distance Education“, edited by George Veletsianos and published by Athabasca University Press (including a freely available e-book version).