This Tuesday, we took part in the final programme meeting in Brimingham – quite a momentous milestone. It feels like a long journey from the initial meeting 18 or so months ago! The close of the meeting involved making an object or giving a presentation about the journey of our project. Here’s Lesley making our thing:
And here’s what it looked like at the end:
Is it merely a bunch of stuff and some string? Or is it an heterogeneous network of actants that has been engineered to achieve project completion? Either way, the cash has strings attached, digital devices are central an potentially oppositional binaries of new and traditional technologies (and texts) have been shown as linked.
Lesley & I have just had a meeting with Nazlin Bhimani and Barbara Sakarya about the work this year focusing on students’ use of LibGuides. There’s lots of exciting possibilities for this strand of work, which may involve us scaling up the scope of it. There may now be multiple focus groups, covering different groups of students. Barbara & Nazlin also updated us about work they’ve already completed, and have blogged about elsewhere.
Here’s our timetable for today:
09.30 - 10.00 Coffee, arrivals
10.00 - 10.15 Welcome (Carol)
10.15 – 12.30 Project updates – 10mins presentation and 20mins discussion
(With a short comfort break 11.15)
12.30 – 14.00 Lunch and networking
14.00 – 16.00 Planning for year 2
16.00 – 17.00 Planning and action points
17.00 onwards ‘cultural hour’.
Yesterday, we took part in a meeting with our cluster partners. Kudos to Greenwich for providing a fabulous venue for the event.
It was a useful opportunity to review progress (including the videos each project had made), talk frankly about some of the challenges we’ve been engaging with and also to go into more detail about ideas around dissemination, in particular.
This week, we took part in the programme meeting, bringing together all the projects working on digital literacies. It was a really useful opportunity to hear what the other projects have been up to, and to hear more about the cross-project themes and issues that have emerged. Helen Beetham’s overview of the baseline reports was particularly useful, giving some programme-level context for what we’re doing.
We also took part in several smaller discussions, around topics such as disciplinarity, change management and evaluation. This was a chance to present and discuss some of our thinking about evaluation, as well as to listen to what others are doing. There was a pleasantly surprising level of similarity between what we are doing and how the project at Reading are undertaking evaluation, for example. It’s reassuring to know it’s not just us thinking along those lines.
All projects were required to prepare videos in advance of the meetings; these are to be made public soon, at which point we’ll link to ours.
Following a recent IT Services review, the IOE has constituted a task group of the Information Strategy Committee (which has academic governance of IT services), which is now in the process of drafting a requirements analysis to inform future activity. This review has drawn on various institutional committees’ work, but has also drawn evidence from the early analysis undertaken in this project. In the current draft’s appendix, one page of the three-and-a-half pages of evidence is drawn from our work. Qualitatively, the priorities we’ve identified have shaped the review. For example, “differentiation” – something that the project has shown is important in supporting our students – is now listed as one of the principles against which IT solutions should be prioritised, designed and selected.
This is, of course, an early indication – the report has yet to be approved or enacted – but it does signal how project outputs are starting to influence institutional activity.
Specifically, I think it provides an interesting example of our theoretical frame in action. We can follow material/digital project texts to show how specific actors (in this case, students and academics), who were previously quite marginal in decisions about IT service provision, are able to use the texts to reconfigure existing arrangements in ways that are more favourable to their needs and priorities.
During today’s webinar, I was struck by a contrast between the work we’re doing and how other projects were talking about their work. It seemed to me that some projects had clear definitions of digital literacy, with things like skills frameworks and assessment tools; others had less precisely pinned-down concepts, and were doing a lot of work to negotiate what this might mean in specific (e.g. disciplinary) contexts; and we seem to be getting ever more tentative about using the term. The contrast, it seems to me, is that other projects are very concerned with how aware people are of the term; reflecting on this, I think that we aren’t.
This post is really an attempt to try and think a little bit about why that’s the case. I think we’re using the phrase as a placeholder to point to specific, situated instances where people have been ‘digitally literate’ – the claims relate to historically specific configurations of people, resources and so on. So we’re trying to identify where such things happen (so we have greater awareness), but also we’re trying to make it easier for such configurations to happen – what we’re not spending time on is getting people talk about whether it happens or not. This has strengths and weaknesses: it might make it harder for us to lay claim to stuff at the end of the project (we don’t have a neat checklist to tick stuff off against), but it does get rid of the problem of ‘embedding’, because we’re not creating some new, separate practice that then needs to be incorporated into what already happens: we’re focused on changing what happens anyhow.
Lesley summarised this nicely towards the end of the evaluation session: “we’re not seeking to create an undifferentiated set of expectations about our students”. Students’ digital literacy practices are incredibly diverse and that’s ok; it’s entirely appropriate given the various contexts (personal, professional, studious) that they work in. We don’t want or need them to converge on a list we’ve created. But we should be better at understanding that diversity and working to support it. It’s helped us to approach this problem by using the phrase “digital literacies” to frame the work, but it may not be as helpful to the students or teaching staff to have this phrase added to their vocabulary.
Today, we took part a webinar about the project’s evaluation work.
One important part for us was the idea that Jay referred to, that “evaluation has to be supported by some kind of theory of change”. This is something we’re still grappling with. Our account of change has to look at the kind of day-to-day work that goes on in the institution, because we want to improve the day-to-day experiences of students. (In this sense, we certainly feel comfortable with what Jay described about the evolutionary processes aiming at “an iterative deepening of understanding”; I think we’d agree we’re less clear about structural measures of accountability.) What our early work has shown us is that structures don’t determine these experiences, because students are creative, adaptive and resourceful. So we can’t rely on claims about changing structures; we have to be able to talk about changes to what people do, too. We’re drawing on sociomaterial accounts of change to help us with this. We expect these to be able to help us provide convincing accounts of the ways that the project has helped to reconfigure structures, resources and practices – but we’re aware that we’ll need to provide convincing accounts of this.
The accountability side of things is harder to talk about – but not because it isn’t happening. Rightly, Jay challenged us to answer the “so what?” question: does the work contribute to ‘big picture’ outcomes; is the work evaluative; will the answers be direct, explicit and actionable?
This is a fair challenge. Over the last few months, we’ve been discussing how our close analysis of practices helps us to do these things. Here’s some ideas that Lesley and I have been exchanging that shows our thinking; they’re all tentative and provisional, but are an indication of our general direction.
- Theoretically, we’re developing the idea that being digitally literate involves being able to create places where you can complete academic/professional/personal tasks by coordinating spaces, things (particularly digital things) and people. (This needs work; it’s not elegant. But it’s pointing where we’re thinking right now.)
Building on this, and our baseline work:
- We can make our students more ‘digitally literate’ if we can identify areas where they struggle to coordinate people/places/things and ‘black box’ them. An example of where this would work well would be single sign-on authentication, but library access to electronic journals via library sign-on is still a success story in this respect: students don’t need to manage personal authentication for a dozen different journals. Sociomaterially, we coordinate networks that students can readily co-opt to their ends. It adds an environmental, infrastructure element that we can tackle at an institutional level.
- We can explain how some students demonstrate their digital literacy in the ways that they can overcome shortcomings in the coordinated environment we provide them with. If they can’t work through our networks but can work around it – by co-opting other people, places, things – they prove themselves digitally literate and suggest areas where we need to black box more effectively. (Students’ decision to use printers in another institution rather than here might be one example of this.)
- Combining the above, ‘Black boxing’ may create risks: if a system is disaggregated it might be possible to work with some and work around others. For example, setting the printers at the IOE so that you can’t edit it to double-sided printing black-boxes in a way that might be helpful for the least able students, but is actively unhelpful for those who want more control over printer settings. We should think about the design of boxes in such a way that they can be opened gracefully. Email forwarding might be another example. Student frustration over lack of control might be a signal for black boxes that we should make more amenable to opening.
On the basis of this, we’ve started using words like “resilience” when talking about students’ digital literacy, and thinking about how we can help make students’ digital literacy more “resilient” by helping them to think about strategies they could use to reconfigure networks when things go wrong.
Just completed a presentation about the project at the institution’s annual Learning and Teaching conference. (The slides from this are available on slideshare.)
Turnout was great (a full room) and feedback was very positive – several comments about how interesting the project was, and how important it was to take a position that assumes and builds upon the diversity of student practice. I thought it was particularly interesting how the project’s work was seen as an important part of so many different peoples’ areas of practice. Discussion of PGCE and distance students was particularly lively.
At the end of the session I asked people to think about the students they work with, and how these issues might relate to work there – this sparked off some lively discussion in small groups. I tried to get some of this fed back in plenary, and managed to note the following before everyone rushed off to get lunch:
- That it was good to see the project’s purpose was not to classify whether people are competent or not, but to understand and support people in the alternative ways of negotiating what you have to deal with day-to-day.
- That this contributes in to ongoing dialogue with students about their academic literacies – it was described as intensifying such discussions “in quite a good way”.
- It is important to help teachers find ways to work that recognise this diversity of practice are so are more inclusive than a ‘one-size-fits-all’ approach. (For example, how should tutors work with groups in classes where some people have connected digital devices and others don’t?)
- There was discussion about how hard it is to get students to collaborate online – they only do what’s compulsory, not what’s offered, but were able to engage when it was made compulsory. It was suggested that what teachers may need to do differently is not just expect use, but to do more to support this and raise expectations about digital participation.
- Students may face important issues decisions balancing having to pay for printing against how they might want to read.
- There were repeated requests to undertake similar work with staff – to understand the diversity of their practices and the challenges they face.
- Memoranda of Agreement and Memoranda of Understanding ask about resources in other institutions – as we use more digital resources, local institutions’ lack of resources may matter less. (Note: we did discuss the differences there may be between offering resources to students and them being able to use them successfully.)
Repeatedly, I was also asked whether we could explore whether these constraints and expectations identified amongst students were replicated for teaching and support staff. This was seen as a really important issue, for several reasons. Firstly, these issues have ramifications for students (e.g. the resources available in classrooms can limit what we are able to model to students during classes). Secondly, there was concern that decisions about infrastructure and IT provision are being made on the assumption that staff are homogeneous, with similar needs and patterns of work; it would be helpful to have evidence that demonstrates this isn’t the case. Thirdly, there are training and support issues that would be identified by doing this: if we have a better understanding of how staff work, we can work with them in more appropriate ways.
It may be worth discussing with the JISC whether any aspects of this could be taken up under the ‘institutional readiness for change’ work.
The last term has been incredibly busy, so here’s a quick review of some of our activities, which might otherwise be invisible. There’s a lot going on, but some of it only gets seen at the point of presentations, reports or publications.
- We released our baseline report in February, describing current practice and issues. This will provide us with a point of reference for evaluations throughout the project.
- We conducted the last of the focus groups, involving students learning at a distance.
- We completed a pilot of our methodology for the journalling work, which involved us documenting our own practices and then being interviewed by other team members. This gave us confidence that we could generate useful and interesting evidence about our students’ digital literacy practices, confirmed that we could use iPods for data collection and allowed us to practice the tasks, interviews and interpretation we needed for the full journalling study.
- We recruited 12 students to take part in the journalling project, three from each of the groupings with whom we held focus groups (PGCE students, MA students, PhD students, distance students). We’ve held initial interviews and inductions with almost all of them, and most have undertaken initial journalling tasks and met with us for a second time so that we discuss what they’ve gathered to date. We have already gathered some good data, and reviewed this together informally, but have yet to start a formal analysis of this.
- We’ve started sharing this – for example, via the presentation at Lancaster, at the IOE’s learning and teaching conference later today (slides already up on the “Papers, reports & talks” page) and two presentations that feature the project scheduled for the Networked Learning conference in Maastricht next week.
We’ve also got some tasks coming up – the interim reporting to JISC, for example; the talks at Networked Learning to deliver; and some other writing tasks, which will help document and share what we’ve been doing.
An update on the IOE’s conference should follow later today.