The IAFOR Journal of Education
Volume III – Issue I – Winter 2015

The IAFOR Journal of Education
Volume III – Issue I – Winter 2015

The IAFOR Journal of Education
Volume III – Issue I – Winter 2015

This paper explores the application of canonical gradient analysis to evaluate and visualize
student performance and acceptance of a learning system platform. The subject of evaluation is a
first year BSc module for computer programming. This uses ‘Ceebot’, an animated and
immersive game-like development environment. Multivariate ordination approaches are widely
used in ecology to explore species distribution along environmental gradients. Environmental
factors are represented here by three ‘assessment’ gradients; one for the overall module mark and
two independent tests of programming knowledge and skill. Response data included Likert
expressions for behavioral, acceptance and opinion traits. Behavioral characteristics (such as
attendance, collaboration and independent study) were regarded to be indicative of learning
activity. Acceptance and opinion factors (such as perceived enjoyment and effectiveness of
Ceebot) were treated as expressions of motivation to engage with the learning environment.
Ordination diagrams and summary statistics for canonical analyses suggested that logbook grades
(the basis for module assessment) and code understanding were weakly correlated. Thus strong
module performance was not a reliable predictor of programming ability. The three assessment
indices were correlated with behaviors of independent study and peer collaboration, but were
only weakly associated with attendance. Results were useful for informing teaching practice and
suggested: (1) realigning assessments to more fully capture code-level skills (important in the
workplace); (2) re-evaluating attendance-based elements of module design; and (3) the overall
merit of multivariate canonical gradient approaches for evaluating and visualizing the
effectiveness of a learning system platform.

Keywords: Technology Enhanced Learning; Computer Programming; Research Methods;
Multivariate Analysis.

Bernard Montoneri