LSE Library and a REF call-out: lessons

In the second of his two guest posts, Neil Stewart identifies lessons from managing a call-out for publications for REF assessment at the London School of Economics.

If you would like to contribute a guest-post to the UKCoRR blog, please contact a member of the committee.
___________________________________________________________

At the recent RSP event on Readiness for REF (which I blogged about on this blog), my former LSE colleague Dave Puplett and I presented on our experience of managing a call-out for publications for REF assessment. The presentation was qualitatively different from the other presentations at the event, because it was on the managerial and organisational issues surrounding management of publications for REF purposes.

The slides from the presentation, which detail LSE Library's experience (full disclosure: I have now moved on to City University London, where I manage City Research Online) of managing the call-out and subsequent influx of publications, can be downloaded from the RSP site (Powerpoint link). Instead of re-hashing the whole presentation, I thought I would take the opportunity to detail some of the lessons learnt, which are hopefully of general applicability to repository people.

Lesson 1: dealing with REF matters puts you at the heart of things
When LSE Research Online (LSERO) was chosen as the de facto method of managing REF data, LSERO became a much higher strategic priority for the School. This is of course excellent for the service, and it had been the case that the LSERO team and Library management had been plugging away to make this happen for a very long time. However, it's also an opportunity that must be seized, because missing it could have meant that the repository would have been side-lined, and new methods to manage this process would have been found. At LSE, this meant that resources to adequately manage things had to be found, which meant diverting resources from elsewhere to allow this to happen. The REF is too important to ignore: get it right, by allocating adequate resource and managerial effort, and the repository gains profile and prestige; but getting it wrong could be disastrous.

Lesson 2: if you didn't talk to the Research Office before, you soon will
The REF call-out at LSE was instigated by the Research Office. While that team had been close allies during the RAE in 2008, the call-out meant that we really had to start working with them more closely well in advance of the REF. This soon fostered a productive relationship, and allowed us to use the Research Office's channels of communications with which to talk to departments. It also gave us the authority to standardise the way in which publications data was reported upon, since the combined weight of the LSERO team and Research Office left departments with little choice!

Lesson 3: it's possible to use the ePrints (and presumably DSpace) back-end to perform database query magic
If you're lucky enough to have a friendly IT person who can run SQL database queries (or if you have that skill yourself) then get in touch with them when you have to start thinking about REF matters. Being able to access then manipulate data direct from the repository's database is invaluable, because it allows you to create customised reporting data based upon any criteria you might wish to include.

Lesson 4: issues of disambiguation get thrown into sharp relief
Dealing with REF publications data brought up those old librarianship questions which are probably familiar to all of us. Two in particular came into relief particularly strongly:

• Which department do academics really live in? Academics can have multiple allegiances, to their department(s), research centre(s) and other parts of the university (e.g. the senior management team). Where, for REF purposes, should an academic be placed? If "units of assessment" do not correlate with departments, how does the repository map this? These are of course as much questions for the Research Office as they are for repository teams, but nevertheless they must be tackled.
• When do academics start (and finish) their careers with parent institutions? How much data from before (and after) these dates should the repository hold, for REF purposes?

The above points (and I'm sure other people can think of others) points to the need to have a CERIF-ied repository system, which links into other university-wide systems, and which may be able to solve these problems of ambiguity.

Lesson 5: in-press and submitted publications are hard to deal with
Be very careful about how forthcoming publications are dealt with. The problem here is one of recording this data in a non-public forum, which can still be reported back to departments in a useful fashion. Many academics will feel that you are jeopardising their chances of publication by including a citation to an in press item in the live repository without their say-so.

Lesson 6: don't let Open Access be forgotten about!
All of the above sounds like work that could usefully be done by a CRIS, and makes no mention of the primary goal of (most) repositories, which is providing openly accessible research. There is a great danger, in my view, that open access can be overwhelmed by the needs of REF reporting, particularly if the repository team has to devote extra resource to dealing with this. How to balance open access and REF is an open question, and one that the LSERO team are still pondering. One benefit of the REF exercise is that it has made LSERO "complete" (regarding citations, at least), which might be a way of further pushing the open access agenda from a position of strength.

I'm sure there are plenty of other lessons that people could add to this list, judging by discussions at this event and elsewhere- please add them (or any other comments) in the comments section below.

RSP Readiness for REF (R4R) workshop, 5th September 2011

In the first of two guest posts, Neil Stewart reflects on the RSP Readiness for REF workshop.

If you would like to contribute a guest-post to the UKCoRR blog, please contact a member of the committee.
___________________________________________________________

My name is Neil Stewart, and I'm the repository manager for the newly minted City Research Online repository, at City University London. I normally blog at City Open Access, if you want to keep an eye on developments of a repository which is still on a project footing, rather than a fully-fledged service.

The reason you find me writing here is because I was recently the recipient of two invitations: to present at the RSP Readiness for REF (R4R) workshop, held on Monday 5th September in London, and to blog about that event here at UKCORR's blog. I was happy to take up both invitations. What follows summarises some thoughts about the workshop, and on the Research Excellence Framework (REF) as it relates to repositories in general. It should be noted that the opinions below are my own, and do not necessarily reflect those of UKCORR. I'll be writing another post soon, which outlines the R4R workshop presentation which I delivered with my former LSE colleague Dave Puplett.

The event's emphasis (apart from the presentation which Dave & I delivered) can fairly be characterised as macro-level, since it discussed REF data submission in very general terms, and with specific reference to the CERIF metadata standard. CERIF is a flexible, extensible model for metadata about research-producing institutions, including (but not limited to) publications data. It is possible, using the CERIF schema, to model an institution's structure, then show how researchers and research outputs relate to that structure. This has obvious benefits for an exercise like the REF, which will (amongst other things) require submission of data on "REF-able" (i.e. high quality) publications at a department (or at least department-like "unit of assessment") level.

The morning sessions dealt with laying out the details of how CERIF and CERIF-compliant repositories could assist with REF submissions. The first session was an overview of the JISC-sponsored Readiness4REF (R4R) Project, and was delivered by Richard Gartner of Kings College London. R4R, just in the process of closing, has looked at the way in which repositories and other data management tools can provide CERIF-compliant data for REF submission purposes. Second was Keith Jeffery from euroCRIS, who gave the bigger picture as to how CERIF was developed and what it can do. This was followed by a panel discussion on the Measuring Impact Under CERIF (MICE) project, which is attempting to build in research impact data into the CERIF schema, and hence make it readily submissible for REF.

After Dave & my presentation and lunch, there were demonstrations of R4R plug-ins for the three major repository software types (Fedora, DSpace and ePrints). As an ePrints user, I was interested to see a demonstration of ePrints v. 3.3, which is to be released in the next few weeks, and contains some "CRIS-like" functionality. This is a kind of CERIF-lite approach, by which it is easy to create associations between researchers, research grants, research centres etc. to express CERIF-like linkages between them. These linkages can then be exposed in useful ways using ePrints web pages, but also exported as CERIF data to be re-used in other systems, or for a REF submission. This seems to me an interesting development, and one we may have to look at here at City.

The final session featured the obligatory break-out groups. I was assigned to a group which discussed the question: "Do you think CERIF is now a more viable option for your institution to use for its REF submission?" A variety of subjects were covered, as ever with these type of discussions. The two main points I took from it were the fact that CERIF provides the opportunity to provide an "open" citation model by modelling linkages (including positive and negative citations) between publications, outside of the "walled gardens" provided by Scopus and Web of Science; and that, for CERIF to work within my institution, there is the somewhat intractable problem of knowing to whom to speak to find out if, for example, the HR database can be plugged into the repository to transfer CERIF-formatted data between the two systems.

All in all, an interesting and timely event. Keep an eye out for my post on LSE Library's experiences of conducting a mini-REF, coming soon!

CRIS -> Repository...full-text only...or metadata records too?

Just a quick post to hopefully stimulate some discussion ahead of the RSP event ‘Repositories and CRIS: working smartly together. Conference and Software Exhibition’ that is taking place at Nottingham University Park Conference Centre on the 19th of July. (Places still available!)

From the RSP publicity: "With the REF taking place in 2014 (http://www.hefce.ac.uk/research/ref/) CRIS systems and repositories are becoming a higher priority and their functionality more visible. Smooth interaction between the two will be vital for a smooth and painless submission in 2014. Now is the perfect time to find out how others have managed this process! This event will look at how repositories and CRISs can work together to meet this goal. Findings of the RePosit project, which has developed case studies around the integration of Symplectic Ltd and repositories in 5 institutions, will be shared."

I've had several conversations recently with repository managers from different institutions in the process of or planning soon to implement CRIS and integrate with their repositories with many reporting a driver to transfer not only full-text to an institutional repository (the current functionality supported by Symplectic for example) but also metadata-only records - presumably to ensure their repositories remain the locus for research management. However, does it perhaps make more sense to separate our research database in a CRIS from full-text only in a repository?

Arguably, one of the limitations of Open Access repositories from an original conception (in the arXiv mould) of holding, disseminating and preserving full-text research outputs is that they have, in effect, become "diluted" by metadata records for which it has not been possible to procure full-text or copyright does not permit deposit. Developing a "hybrid" model that separates full-text from bibliographic records in this way rather than pushing everything into a repository might enable repositories to return to an antedeluvian world where they are focused once again on preservation of full-text material rather than also including bibliographic data.

The very term "CRIS" is perhaps problematic and probably better conceived of as an infrastructure derived from a set of software and services - HR systems, Finance systems, repository etc as well as 3rd party commercial software like Symplectic Elements, Atira Pure and Converis (Avedas). Moreover, leading repository software providers like EPrints are looking to extend the functionality of repositories themselves (see: Carr, L. (2010) EPrints: A Hybrid CRIS/Repository. In: Workshop on CRIS, CERIF and Institutional Repositories, 10-11th May 2010, Rome, Italy.); I have no idea whether such a "hybrid" approach would necessarily be achievable in every - or even most - institutions; as emphasised at "Learning how to play nicely: Repositories and CRIS", institutions are all different and there is no one-size-fits-all solution. The availability of human and other resources, in-house expertise and the existing infrastructure will all have an impact on the most appropriate course of action. The tentative conclusion from that event was that if starting with a blank slate, it probably makes sense for a CRIS to be the central system with the repository as a linked peripheral component; but, of course, very few are actually starting from this point and different models can be just as effective.