Following a recent IT Services review, the IOE has constituted a task group of the Information Strategy Committee (which has academic governance of IT services), which is now in the process of drafting a requirements analysis to inform future activity. This review has drawn on various institutional committees’ work, but has also drawn evidence from the early analysis undertaken in this project. In the current draft’s appendix, one page of the three-and-a-half pages of evidence is drawn from our work. Qualitatively, the priorities we’ve identified have shaped the review. For example, “differentiation” – something that the project has shown is important in supporting our students – is now listed as one of the principles against which IT solutions should be prioritised, designed and selected.
This is, of course, an early indication – the report has yet to be approved or enacted – but it does signal how project outputs are starting to influence institutional activity.
Specifically, I think it provides an interesting example of our theoretical frame in action. We can follow material/digital project texts to show how specific actors (in this case, students and academics), who were previously quite marginal in decisions about IT service provision, are able to use the texts to reconfigure existing arrangements in ways that are more favourable to their needs and priorities.
During today’s webinar, I was struck by a contrast between the work we’re doing and how other projects were talking about their work. It seemed to me that some projects had clear definitions of digital literacy, with things like skills frameworks and assessment tools; others had less precisely pinned-down concepts, and were doing a lot of work to negotiate what this might mean in specific (e.g. disciplinary) contexts; and we seem to be getting ever more tentative about using the term. The contrast, it seems to me, is that other projects are very concerned with how aware people are of the term; reflecting on this, I think that we aren’t.
This post is really an attempt to try and think a little bit about why that’s the case. I think we’re using the phrase as a placeholder to point to specific, situated instances where people have been ‘digitally literate’ – the claims relate to historically specific configurations of people, resources and so on. So we’re trying to identify where such things happen (so we have greater awareness), but also we’re trying to make it easier for such configurations to happen – what we’re not spending time on is getting people talk about whether it happens or not. This has strengths and weaknesses: it might make it harder for us to lay claim to stuff at the end of the project (we don’t have a neat checklist to tick stuff off against), but it does get rid of the problem of ’embedding’, because we’re not creating some new, separate practice that then needs to be incorporated into what already happens: we’re focused on changing what happens anyhow.
Lesley summarised this nicely towards the end of the evaluation session: “we’re not seeking to create an undifferentiated set of expectations about our students”. Students’ digital literacy practices are incredibly diverse and that’s ok; it’s entirely appropriate given the various contexts (personal, professional, studious) that they work in. We don’t want or need them to converge on a list we’ve created. But we should be better at understanding that diversity and working to support it. It’s helped us to approach this problem by using the phrase “digital literacies” to frame the work, but it may not be as helpful to the students or teaching staff to have this phrase added to their vocabulary.
Today, we took part a webinar about the project’s evaluation work.
One important part for us was the idea that Jay referred to, that “evaluation has to be supported by some kind of theory of change”. This is something we’re still grappling with. Our account of change has to look at the kind of day-to-day work that goes on in the institution, because we want to improve the day-to-day experiences of students. (In this sense, we certainly feel comfortable with what Jay described about the evolutionary processes aiming at “an iterative deepening of understanding”; I think we’d agree we’re less clear about structural measures of accountability.) What our early work has shown us is that structures don’t determine these experiences, because students are creative, adaptive and resourceful. So we can’t rely on claims about changing structures; we have to be able to talk about changes to what people do, too. We’re drawing on sociomaterial accounts of change to help us with this. We expect these to be able to help us provide convincing accounts of the ways that the project has helped to reconfigure structures, resources and practices – but we’re aware that we’ll need to provide convincing accounts of this.
The accountability side of things is harder to talk about – but not because it isn’t happening. Rightly, Jay challenged us to answer the “so what?” question: does the work contribute to ‘big picture’ outcomes; is the work evaluative; will the answers be direct, explicit and actionable?
This is a fair challenge. Over the last few months, we’ve been discussing how our close analysis of practices helps us to do these things. Here’s some ideas that Lesley and I have been exchanging that shows our thinking; they’re all tentative and provisional, but are an indication of our general direction.
- Theoretically, we’re developing the idea that being digitally literate involves being able to create places where you can complete academic/professional/personal tasks by coordinating spaces, things (particularly digital things) and people. (This needs work; it’s not elegant. But it’s pointing where we’re thinking right now.)
Building on this, and our baseline work:
- We can make our students more ‘digitally literate’ if we can identify areas where they struggle to coordinate people/places/things and ‘black box’ them. An example of where this would work well would be single sign-on authentication, but library access to electronic journals via library sign-on is still a success story in this respect: students don’t need to manage personal authentication for a dozen different journals. Sociomaterially, we coordinate networks that students can readily co-opt to their ends. It adds an environmental, infrastructure element that we can tackle at an institutional level.
- We can explain how some students demonstrate their digital literacy in the ways that they can overcome shortcomings in the coordinated environment we provide them with. If they can’t work through our networks but can work around it – by co-opting other people, places, things – they prove themselves digitally literate and suggest areas where we need to black box more effectively. (Students’ decision to use printers in another institution rather than here might be one example of this.)
- Combining the above, ‘Black boxing’ may create risks: if a system is disaggregated it might be possible to work with some and work around others. For example, setting the printers at the IOE so that you can’t edit it to double-sided printing black-boxes in a way that might be helpful for the least able students, but is actively unhelpful for those who want more control over printer settings. We should think about the design of boxes in such a way that they can be opened gracefully. Email forwarding might be another example. Student frustration over lack of control might be a signal for black boxes that we should make more amenable to opening.
On the basis of this, we’ve started using words like “resilience” when talking about students’ digital literacy, and thinking about how we can help make students’ digital literacy more “resilient” by helping them to think about strategies they could use to reconfigure networks when things go wrong.
Lesley and I have just completed two presentations at the Networked Learning conference in Maastricht that draw on the work of the project, both of which were well received. Slides are available from the papers & talks link. Both of these have linked the project’s work to some wider debates – about understanding students’ authorship practices, and about understanding technology. Sadly we were scheduled against each other so I couldn’t see Lesley’s session, but the discussion in my session was certainly lively, raising some important questions about how we can and should go about researching these areas.