LSE Library and a REF call-out: lessons

In the second of his two guest posts, Neil Stewart identifies lessons from managing a call-out for publications for REF assessment at the London School of Economics.

If you would like to contribute a guest-post to the UKCoRR blog, please contact a member of the committee.
___________________________________________________________

At the recent RSP event on Readiness for REF (which I blogged about on this blog), my former LSE colleague Dave Puplett and I presented on our experience of managing a call-out for publications for REF assessment. The presentation was qualitatively different from the other presentations at the event, because it was on the managerial and organisational issues surrounding management of publications for REF purposes.

The slides from the presentation, which detail LSE Library's experience (full disclosure: I have now moved on to City University London, where I manage City Research Online) of managing the call-out and subsequent influx of publications, can be downloaded from the RSP site (Powerpoint link). Instead of re-hashing the whole presentation, I thought I would take the opportunity to detail some of the lessons learnt, which are hopefully of general applicability to repository people.

Lesson 1: dealing with REF matters puts you at the heart of things
When LSE Research Online (LSERO) was chosen as the de facto method of managing REF data, LSERO became a much higher strategic priority for the School. This is of course excellent for the service, and it had been the case that the LSERO team and Library management had been plugging away to make this happen for a very long time. However, it's also an opportunity that must be seized, because missing it could have meant that the repository would have been side-lined, and new methods to manage this process would have been found. At LSE, this meant that resources to adequately manage things had to be found, which meant diverting resources from elsewhere to allow this to happen. The REF is too important to ignore: get it right, by allocating adequate resource and managerial effort, and the repository gains profile and prestige; but getting it wrong could be disastrous.

Lesson 2: if you didn't talk to the Research Office before, you soon will
The REF call-out at LSE was instigated by the Research Office. While that team had been close allies during the RAE in 2008, the call-out meant that we really had to start working with them more closely well in advance of the REF. This soon fostered a productive relationship, and allowed us to use the Research Office's channels of communications with which to talk to departments. It also gave us the authority to standardise the way in which publications data was reported upon, since the combined weight of the LSERO team and Research Office left departments with little choice!

Lesson 3: it's possible to use the ePrints (and presumably DSpace) back-end to perform database query magic
If you're lucky enough to have a friendly IT person who can run SQL database queries (or if you have that skill yourself) then get in touch with them when you have to start thinking about REF matters. Being able to access then manipulate data direct from the repository's database is invaluable, because it allows you to create customised reporting data based upon any criteria you might wish to include.

Lesson 4: issues of disambiguation get thrown into sharp relief
Dealing with REF publications data brought up those old librarianship questions which are probably familiar to all of us. Two in particular came into relief particularly strongly:

• Which department do academics really live in? Academics can have multiple allegiances, to their department(s), research centre(s) and other parts of the university (e.g. the senior management team). Where, for REF purposes, should an academic be placed? If "units of assessment" do not correlate with departments, how does the repository map this? These are of course as much questions for the Research Office as they are for repository teams, but nevertheless they must be tackled.
• When do academics start (and finish) their careers with parent institutions? How much data from before (and after) these dates should the repository hold, for REF purposes?

The above points (and I'm sure other people can think of others) points to the need to have a CERIF-ied repository system, which links into other university-wide systems, and which may be able to solve these problems of ambiguity.

Lesson 5: in-press and submitted publications are hard to deal with
Be very careful about how forthcoming publications are dealt with. The problem here is one of recording this data in a non-public forum, which can still be reported back to departments in a useful fashion. Many academics will feel that you are jeopardising their chances of publication by including a citation to an in press item in the live repository without their say-so.

Lesson 6: don't let Open Access be forgotten about!
All of the above sounds like work that could usefully be done by a CRIS, and makes no mention of the primary goal of (most) repositories, which is providing openly accessible research. There is a great danger, in my view, that open access can be overwhelmed by the needs of REF reporting, particularly if the repository team has to devote extra resource to dealing with this. How to balance open access and REF is an open question, and one that the LSERO team are still pondering. One benefit of the REF exercise is that it has made LSERO "complete" (regarding citations, at least), which might be a way of further pushing the open access agenda from a position of strength.

I'm sure there are plenty of other lessons that people could add to this list, judging by discussions at this event and elsewhere- please add them (or any other comments) in the comments section below.

RSP Readiness for REF (R4R) workshop, 5th September 2011

In the first of two guest posts, Neil Stewart reflects on the RSP Readiness for REF workshop.

If you would like to contribute a guest-post to the UKCoRR blog, please contact a member of the committee.
___________________________________________________________

My name is Neil Stewart, and I'm the repository manager for the newly minted City Research Online repository, at City University London. I normally blog at City Open Access, if you want to keep an eye on developments of a repository which is still on a project footing, rather than a fully-fledged service.

The reason you find me writing here is because I was recently the recipient of two invitations: to present at the RSP Readiness for REF (R4R) workshop, held on Monday 5th September in London, and to blog about that event here at UKCORR's blog. I was happy to take up both invitations. What follows summarises some thoughts about the workshop, and on the Research Excellence Framework (REF) as it relates to repositories in general. It should be noted that the opinions below are my own, and do not necessarily reflect those of UKCORR. I'll be writing another post soon, which outlines the R4R workshop presentation which I delivered with my former LSE colleague Dave Puplett.

The event's emphasis (apart from the presentation which Dave & I delivered) can fairly be characterised as macro-level, since it discussed REF data submission in very general terms, and with specific reference to the CERIF metadata standard. CERIF is a flexible, extensible model for metadata about research-producing institutions, including (but not limited to) publications data. It is possible, using the CERIF schema, to model an institution's structure, then show how researchers and research outputs relate to that structure. This has obvious benefits for an exercise like the REF, which will (amongst other things) require submission of data on "REF-able" (i.e. high quality) publications at a department (or at least department-like "unit of assessment") level.

The morning sessions dealt with laying out the details of how CERIF and CERIF-compliant repositories could assist with REF submissions. The first session was an overview of the JISC-sponsored Readiness4REF (R4R) Project, and was delivered by Richard Gartner of Kings College London. R4R, just in the process of closing, has looked at the way in which repositories and other data management tools can provide CERIF-compliant data for REF submission purposes. Second was Keith Jeffery from euroCRIS, who gave the bigger picture as to how CERIF was developed and what it can do. This was followed by a panel discussion on the Measuring Impact Under CERIF (MICE) project, which is attempting to build in research impact data into the CERIF schema, and hence make it readily submissible for REF.

After Dave & my presentation and lunch, there were demonstrations of R4R plug-ins for the three major repository software types (Fedora, DSpace and ePrints). As an ePrints user, I was interested to see a demonstration of ePrints v. 3.3, which is to be released in the next few weeks, and contains some "CRIS-like" functionality. This is a kind of CERIF-lite approach, by which it is easy to create associations between researchers, research grants, research centres etc. to express CERIF-like linkages between them. These linkages can then be exposed in useful ways using ePrints web pages, but also exported as CERIF data to be re-used in other systems, or for a REF submission. This seems to me an interesting development, and one we may have to look at here at City.

The final session featured the obligatory break-out groups. I was assigned to a group which discussed the question: "Do you think CERIF is now a more viable option for your institution to use for its REF submission?" A variety of subjects were covered, as ever with these type of discussions. The two main points I took from it were the fact that CERIF provides the opportunity to provide an "open" citation model by modelling linkages (including positive and negative citations) between publications, outside of the "walled gardens" provided by Scopus and Web of Science; and that, for CERIF to work within my institution, there is the somewhat intractable problem of knowing to whom to speak to find out if, for example, the HR database can be plugged into the repository to transfer CERIF-formatted data between the two systems.

All in all, an interesting and timely event. Keep an eye out for my post on LSE Library's experiences of conducting a mini-REF, coming soon!

CRIS + Repositories at UK Universities

On the back of this recent post I've been invited to speak at the RSP Autumn School about whether the developing CRIS / repositories landscape at UK Universities might present an opportunity to re-focus on Open Access.

Since the RAE2008, the uptake of integrated electronic research administration systems (CRIS, ERA, RMAS...choose your favourite acronym / abbreviation) has been dramatic with the primary driver being to oil the administrative wheels of the REF in 2013.

As the ever-growing UKCoRR membership attests (currently 254 members) Institutional Repositories are now well established across the HE sector and there are several approaches that institutions are taking to utilise this existing repository infrastructure for research administration and/or embed their repository as a component of a broader research administration infrastructure with many either implementing additional commercial software or developing a bespoke solution in-house.

A significant initiative in this area is the JISC funded RePosit project (final report due in October 2011) which aims to "increase uptake of a web-based repository deposit tool embedded in a researcher-facing publications management system" and comprises a consortium of 5 institutions (University of Leeds , Keele University, Queen Mary University of London, University of Exeter and University of Plymouth) in partnership with Symplectic Ltd as a commercial partner. The project runs a Google group at http://groups.google.com/group/reposit where there has been a very active thread recently discussing this developing environment; I have used the thread to collate a list of CRIS + repositories at UK institutions and set up a public Google doc if anyone would like to add their institution (I won't post the link here but it's already been shared via the RePosit and UKCoRR mailing lists.)

As can be seen from the list so far, the most common solutions are commercial software implementations of Atira Pure (11 instances) and Symplectic Elements (16 instances) [+4 instances of Avedas Converis and 1 bespoke]. Though this hastily compiled document almost certainly reflects the membership of the RePosit Google group* with only 32 institutions so far represented out of 142 Institutional Repositories in the UK listed on OpenDoar I think it's still a big enough sample to be significant, especially as Symplectic, Atira and Avedas are arguably the only real options currently in the market-place (notwithstanding the ongoing development of CRIS-like functionality into EPrints itself which is by far the most popular repository platform in the UK (run by 71 UK institutions listed on OpenDoar - exactly half of the total.)

* Similar information is also being captured on the new RSP wiki -http://www.rsp.ac.uk/pmwiki/index.php?n=Institutions.HomePage

Both Symplectic and Pure are designed to enable research staff to manage their research profile both manually and by pulling data from online databases via their APIs (Web of Science, SCOPUS, Mendeley etc) but one observation that is worth commenting upon is the different core functionality of the two systems with Symplectic providing a modular solution designed to integrate with an existing repository whereas Pure is arguably more fully featured software, capable of managing full-text and with full version control, functionality to manage embargo, visibility of items based on business rules, and fully indexable by search engines. Indeed, the comprehensive nature of Atira Pure raises the intriguing possibility that it could effectively replace a repository altogether; at this stage, however, I believe the majority of institutions running the software have chosen to integrate with an existing repository in the Symplectic model – a good example is the University of St Andrews who are running Pure – research portal here - alongside their DSpace repository - http://research-repository.st-andrews.ac.uk/ - which is just used for full-text.

There are some very interesting perspectives on the RePosit thread on why we should or should not maintain two systems (CRIS + repository) and it is clear that the decision will depend to a large extent on the particular systems at a given institution and their specific configuration. Janet Aucock of St Andrews suggests that the integration of the best features of both CRIS and IR will evolve over time going on to say that an important consideration should be "not to lose flexibility and options. Teams across research offices and libraries can be well coordinated, communicate well and have regular contact and debate"..."But even then...there can be differences in emphasis in what the CRIS is about and what service it offers. The Library tends to emphasise open access and discovery. Research office is undoubtedly more focused on research assessment. (Also see James Toon’s comment on this blog).

Arguments in favour of retaining repositories include the issue of creative arts research outputs and Jackie Wickham of the RSP (and now UKCoRR secretary) points out that “there has been considerable investment in developing repositories (focused on EPrints, KULTUR plug in) to enable them to showcase this type of research e.g. UAL - http://ualresearchonline.arts.ac.uk/. The visual impact of the repository is critical to arts researchers and many institutions are using the KULTUR plug in – and not just the specialist arts ones”. I think this is a persuasive argument, also for those using their repositories for "non-REF" output / grey literature / Open Educational Resources (though I have posted elsewhere that I’d actually like to investigate the CRIS model for managing OER as there seems no fundamental reason why such a system could not be used to support the workflow for both OA research and OER.

As to whether all of this does indeed present an opportunity to re-focus on Open Access I think is still a moot point. There is perhaps a danger that the administrative burden of the REF will overshadow the objective of providing Open Access to research but there is also the opportunity to integrate the various infrastructural components in such a way to facilitate what are ultimately complementary objectives; to increase visibility of institutional research, improve awareness and advocacy initiatives around OA (and OER) and to more effectively link institutional research administration with access to the actual research outputs.