Altmetrics in the last year and what’s next

Guest blogger for this session is Ruth Wells, Senior Publication Manager at Highwire.

Ruth has experience on both the publisher and vendor side of scholarly publishing, and was formerly Head of Engineering and acting Product Manager at Publishing Technologies.

Ruth has spent time working for several leading publishing organizations including Taylor & Francis, SAGE, and Cambridge University Press in IT and operations. Ruth is a qualified ScrumMaster and PRINCE II accredited project manager.

Altmetrics in the Last Year and What’s Next:

Martin Fenner from PLOS started the fast paced session to tour the last year and the future of the altmetrics universe.  An exciting session with much to offer from the 7 speakers who each had 7 minutes in which to tell their story.

Euan Adie from Altmetric kick started the discussion with a rapid review of Altmetric, plus the roadmap they are considering based on the use cases originating from pilot projects. One of the key outcomes of this research is the indication that altmetrics are not necessarily an indicator of the articles that will attract citations. Altmetric want to look at impact within and beyond the scholarly use – and have launched an institutional edition (which displays metrics by department) to help academics do this.  It is not just about twitter, although that is a useful data source.  It is more about looking at the influence of research on outcomes such as policy decisions by auditing data sources and establishing what this data means. In this environment a single data score is not useful across the board and actually we need to look at the flavour of impact for different types of articles.  There is also a question of whether research influences practice as well – the number of citations don’t guarantee implementation or readership – it needs human analysis to see whether someone has acted on that data.

Then Andrea Michalek from Plum Analytics spoke next. Plum was founded in 2012 by a librarian and a technologist. In the last year it was acquired by EBSCO and this opened up opportunities for the company. Measures used include usage, captures, mentions, social media and citations.  Key findings so far include that Mendeley can demonstrate leading indicators for future citations and that social media is a great way of measuring the level of promotion around particular research articles.  Existing features include Single Sign On, IP authentication, user login and UI customisation. As part of the acquisition Plum Analytics have been able to include an increasing data exhaust with access to the discovery service and aggregate, agnostic usage data. This has lead to a visualisation service, book metrics on Amazon and funder insights into particular subject areas.

Jennifer Lin from PLOS works with Martin Fenner on the article level metrics product. It has been compared to a cottage industry alongside other developments happening in this space. The work this year is more related to a warehouse! The plan is to extend out across publishers, building out sources and refactoring the code base while thinking about technical and business needs e.g. how to get and refresh data on a frequent basis preferably in real time. PLOS are increasingly looking at visualisation of reporting including a date time frame, geography and collating data beyond one publisher which is far more fun! Periscope is launching next week with CrossRef data which overlays information from internal systems and the CrossRef API. This will include subject areas, disciplines, research collaboration groups etc. The  ultimate aspiration is to create a community eco-system with trusted parties for article level metrics available to all with no barriers to access.

So on to David Sommer from Kudos, who felt that altmetrics could become the predominant metrics for the future as it’s all about discoverability and impact is increasingly important.  Kudos offers tools for authors to share research and measure impact using altmetrics. An initial survey of authors revealed that 80% said authors should be promoting themselves and at an initial pilot authors saw a 19% increase in usage per day using these tools.  Kudos’ aim is to Explain Enrich Share and Measure using a simple interface for authors to link up promotional text. There is a dashboard for authors to summarise promotional output and the results.  ORCID is now a partner adding to Ringgold and CrossRef and a new user interface is on its way this year with new tools, suggestions and the Altmetric donut. This is all to encourage authors in their quest – “You did something, something good happened, do some more”. Kudos is about to aggregate statistics at an institutional level with small pilot, and invites librarians to be included, plus is also working with A&I’s. The question posed by the floor was will Kudos scale or will only early adopters get the benefits? David’s response: that this is a long term issue and the tools are likely to change and become increasingly sophisticated. Watch that space!

Then William Gunn from Mendeley (@mrgunn) summarised his talk with – API. Scopus integration. Snowball altmetrics.  That’s all you need to know! Each presence of a document in Mendeley user library is a read and comes with metadata such as academic status and geography which makes for valuable data. There is proof within of an impact over a broader spectrum of people outside academia – the non-author population. There is a new development site with extended methods with all disciplines and tags and Mendeley have eased restrictions. Mendeley are looking at identity, privacy, attribution, gaming and filtration but there is no gold standard as yet. Note that a very popular paper could be wrong!  Mendeley want to identify who’s doing the best work, even in obscurity, by trying to minimise the bias of who’s on the site.  Obviously they cannot see what someone is thinking when it comes to impact on the broader scholarly environment but they do know if the reader went to a PDF viewer, scrolled to the end of the article and shared it.

Stacy Konkiel from ImpactStory got up very early this morning to present remotely so thank you to her.  Impact Story are aiming to incentivise open science researchers through altmetrics and remain open, to avoid commercialisation of article impact.  They place alot of importance on contextualising metrics, for example “how many is alot of citations?” through identifying percentiles and exposing underlying data to the community. Mendeley metrics are inserted as well as GitHub and Scopus. Researcher profiles are included to build a professional tool for researchers which launched in last few weeks and is subscription based after free 30 day trial.  Impact Story are growing fast, the team grew by 33% this year with NSF funding, and new features include maps, GitHub and Twitter impact statistics, better software tracking, embedded articles and metrics on blog posts.

Finally, Mark Patterson from eLife spoke about DET which helps to create open infrastructure to enable altmetrics and researcher communication and evaluation as a whole. The prevailing paradigm in judging researchers is that it puts a brake on innovation so they want to lessen the affect of that and move away from focussing on the container to the actual research. DET are not trying to reinvent the wheel and compete with other metrics services, its based on the PLOS tool and collects data plus makes data openly available.  It relates to institutions, researchers, funders and service providers and there are use cases for both quantitive and qualitative measures. There’s a Technical Working group and a Governance one for policy and sustainability to establish a pilot in the next 6 months.

A whistle stop tour through the current providers who are certain of one thing, that there will continue to be change in this space.