Extending IMS Caliper AnalyticsTM with Learning Activity Metric Profiles

A LAK16 Workshop

View project on GitHub

The Challenge

Today's digital learning ecosystem is expected to leverage data analytics in order to drive pedagogical innovation. Yet an ecosystem rich in services but lacking a common approach to measuring learning activity will find data collection, aggregation and analysis time-consuming and costly. The IMS Caliper AnalyticsTM specification addresses the need for data and semantic interoperability by providing an extensible information model, controlled vocabularies and an API for instrumenting learning applications and systems that log learning events.

The Caliper information model is composed of a set of extensible learning activity profiles. Each profile constitutes a controlled vocabulary that describes a set of domain-specific concepts, relationships and constraints. Caliper has embraced linked data principles in its adoption of JSON-LD and its profiles aspire to provide both structure and shared meaning when data is exchanged between different agents (both human and machine).

Caliper Event Stream

However, many learning activities have yet to be modeled by the Caliper working group. Engaging the SoLAR community directly in this effort will help ensure that the needs of researchers and other consumers of learning analytics data will inform future versions of the specification. The LAK16 Caliper workshop is being offered with this goal in mind. The half-day session, facilitated by members of Team Caliper, will provide LAK16 participants with an opportunity to extend the Caliper specification by modeling new learning activity profiles.

Objectives

Caliper workshop objectives are fourfold:

  • increase awareness and understanding of the Caliper information model
  • provide practical training on modeling learning activities
  • generate a set of draft metric profiles that can be included in future releases of the Caliper specification
  • encourage the SoLAR community to consider engaging more directly with the project
 

The Caliper working group is looking to LAK16 participants to help enrich the existing set of profiles as well add additional profiles in order to extend the information model and increase its utility. New profiles, new connections and new friendships are expected outcomes.

Format

Members of Team Caliper will lead the half-day workshop. The workshop will commence with a brief overview of the Caliper information model and its constituent profiles, entities and controlled vocabularies. Following the overview, participants will be invited to propose learning activities that they would like to model or they can choose from a list of suggested activities that require modeling.

Existing Profiles

The existing set of Caliper profiles cover a limited number of use cases:

  • session login/logout/timeout
  • page navigation
  • resource viewing
  • annotation
  • rich media interactions
  • assessment
  • outcomes
Caliper Session Profile Caliper Navigation Profile Caliper Reading Profile Caliper Assessment Profile Caliper AssessmentItem Profile Caliper Annotation Profile Caliper Media Profile Caliper Forum Profile Caliper Outcome Profile

Caliper’s annotation profile, for example, models a variety of actions: bookmarked, classified, commented, described, highlighted, identified, linked, questioned, recommended, replied, and tagged.  Annotation events are described using an expanded event model that includes the application context, group/organizational context, an actor’s membership context (e.g., role, status) as well as objects that are the target of the activity and/or generated as a result of the interaction as in the case of a highlight or bookmark created as result of a user annotating a digital resource.

Caliper TagAnnotation

A partial list of learning activities that have yet to be described include content authoring, forums/discussions, problem sets, learner participation and credentialing. Participants may have a local need that requires creation of a custom profile; the facilitators will happily accommodate such interests. Participants will then self-organize into teams of 2-3 in order to design one or more draft Caliper profiles and then report out on their work before the close of the workshop. Draft profiles will be stored in a public Github repository.

Audience

In contrast to the LAK16 Apereo/JISC hackathon, which seeks participants possessing a technical mindset, the Caliper workshop is open to all practitioners and researchers with a professional interest in the advancement of teaching and learning and the practical application of learning analytics. In short, one need not be a developer or data architect to contribute usefully to the evolution of the Caliper specification.

Facilitators

Anthony Whyte (@arwhyte) is a member of Team Caliper and holds a staff appointment at the University of Michigan. Between 2004-2012 he was seconded to the open-source Sakai and OAE projects, serving in a variety of community, technical and leadership roles. In 2013 he "returned home" in order to focus on campus initiatives that leverage technology in the service of teaching and learning. During the fall of that year he began contributing in earnest to IMS Global's Caliper learning analytics initiative. A former Fulbright scholar, Anthony was educated at Michigan State University and Princeton University.

John Johnston (@johnpj) is Product Manager and Lead Analyst for the Information Technology Services Teaching & Learning Team at the University of Michigan. Technologist, author, digital artist, and musician, he has worked in the academic technology field for over 20 years managing the intersections between teaching, technology, and innovation.

Acknowledgements

The Caliper AnalyticsTM specification was developed under the auspices of the IMS Global Learning Consortium, a non-profit organization that fosters collaboration among the world’s leading education providers and technology suppliers. The University of Michigan is an IMS contributing member and an active member of the IMS Caliper working group.