Responsible Use of Research Metrics

Source: https://www.ncl.ac.uk/research/research-governance/research-metrics/ Parent: https://www.ncl.ac.uk/undergraduate/

Logo

Who we Are

Who we Are - Excellence and impact - Research Excellence - Teaching and Student Experience - Graduate Employability - League Tables and Rankings - Our Strengths - Engagement and Place - Research Strategy - Education Strategy - Social and Environmental Justice - Social Justice - Sustainability - University of Sanctuary - Our Campuses - Faculties, Schools and Research Units - Newcastle Campus - NUMed Malaysia - Singapore Campus - Our People - Global - Structure and Governance - Transparency - Office for Students Transparency Data - Access and Participation - Brand Hub - Deans - Key Dates - Our Partnerships - History of Newcastle University - Contact Us - Maps and Directions - Study with Us

Study with Us - Undergraduate - Find a Degree - Subject Areas - Applications and Offers - Undergraduate Application Advice - Step-by-Step Guide for UK Students - Step-by-Step Guide for International and EU Students - Applying through UCAS - Entry Requirements - A and AS Levels - Personal statements - Application Decisions - Access Schemes and Pathway Programmes - PARTNERS - Policies and Procedures - Applicants with Disabilities - Mature Applicants - Deferred Entry - Fees and Funding - Scholarships and Bursaries - Vice-Chancellor's Excellence Scholarships - Vice-Chancellor’s EU Scholarships – Undergraduate - Vice-Chancellor's International Scholarships - International Foundation Scholarships - International Family Discounts - Opportunity Scholarships - Subject Scholarships - Sports Scholarships - St Nicholas’ Educational Trust Scholarship - NU Sanctuary Scholarships - Tuition Fees and Costs - Additional Costs - Student Loans - International Student Finance - Global Scholarships - Undergraduate Open Days - Sign up and Discover - School and College Outreach - Information for Parents and Supporters - Request a Prospectus - Chat to Our Students - Postgraduate Taught - Find a Degree - Why Choose Newcastle? - Your Study Options - Qualifications Explained - Fees & Funding - Search for Funding - Funding Your Studies - Global Scholarships - Applications & Offers - Application Help - Advice & Resources - Your Offer Guide - Postgraduate Open Days - Chat to Our Students - Postgraduate Research - Find a Degree - Why Choose Newcastle? - Qualifications Explained - Research at Newcastle - Is a PhD for me? - Fees and Funding - How to Apply - How to apply for Studentships - How to Propose your Research - How to Write a Research Proposal - Occasional Students - Student Support - Careers - Our Doctoral College - Postgraduate Open Days - Chat to Our Students - Global Scholarships - Distance Learning - Continuing Professional Development (CPD) - Why Choose Newcastle University? - Our Campus - Libraries - IT - Study Support - Sport - Campus Tours - Student Safety - Student Life - Life in Newcastle - City Suburbs - Sports and Fitness - Food and Drink - Local Services - Things to do - Our North East Region - Cost of Living - Support and Wellbeing - Teaching and Learning - Student Stories - Chat to Our Students - Student Blog - Belong - Accommodation - Our Accommodation - Accommodation Types - Types of Rooms - Accessible accommodation - Staff and Visitors - Bedrooms we offer - Booking - Applying for Accommodation - Accommodation Guarantee - Applying for Undergraduate Accommodation - Applying for Postgraduate Accommodation - Applying for Accessible Accommodation - Part Year Student Accommodation - Clearing and Insurance - Living with Us - Safety and Security - Moving In - Leaving Us - Returning Next Year - Extending Your Stay - Room Changes - Current Residents - Insurance - Parking & Bicycle Storage - Post and Parcels - Guest Visitors and Going Away - Energy & Recycling - ResLife - ResLife Find a Flatmate - Your ResLife Team - Student Support - Policies - Accommodation Payments - Payment Methods - Payment Schedules - Managed Partnerships - Rent Adjustments - Contact Us - Student Village Receptions - Your Accommodation Team - Report a Fault - Feedback and Complaints - Internet Connection - Your Future - Work Placements - Careers Service - About the Careers Service - Careers Service Events - Work for Yourself - Career Planning - Careers Modules - Making Applications - Interviews, Tests & Assessment Centres - Internships, Placements & Shadowing - Finding Jobs - Handling Job Offers - Researching Employers - Making Contacts - Further Study - Awards, Competitions & Project Funding - Volunteering - Boost Your CV - Defence Technical Undergraduate Scheme (DTUS) - Meet Us - Getting Here - Undergraduate Open Days - Undergraduate Virtual Event - Undergraduate Offer Holder Days - Undergraduate Pop Up Event London - Campus Tours - Self-Guided Campus Tours - Postgraduate Open Days - Postgraduate Schools & Supervisors - International Students - Find your Country/Region - International Student Experience - Fees and Funding - English Language Requirements - Visa and Immigration - Chat to Our Students - International Pathway Courses - Newcastle University International Study Centre - Student Exchange and Study Abroad - Request a Prospectus - Key Dates - Contact Us - Your Academic Experience - Our Research

Our Research - Research Impact - Research Strengths - Centres of Research Excellence - Centre for Ageing - Centre for AI Safety - Centre for Behaviour - Centre for Biomedical Engineering - Centre for Cancer - Centre for Children and Youth - Centre for Climate and Environmental Resilience - Centre for Cyber Security and Resilience - Centre for Data Science and AI - Centre for Energy - Centre for Healthier Lives - Centre for Heritage - Centre for Industrial Biotechnology - Centre for Landscape - Centre for Mobility and Transport - Centre for Rare Disease - Centre for Researching Cities - Centre for Transformative Neuroscience - Centre for Water - Research Culture - Research Culture Action Plan - Working Together on Research Culture - Doctoral College - Global Challenges Academy - Global Partnerships - Let's Work Together - Research Excellence Framework (REF) 2021 - Research Directory - Research Governance - Research Integrity - Code of Good Practice in Research - University Research and Innovation Committee - Ethics at Newcastle - Ethics Forms and Processes - GDPR Information for Research - Ethics Toolkit - Responsible Research Innovation - Animals and Research - Security and Defence Research - Research Metrics - Export Control - Open Research - Policies and Guidance - Research and Innovation Support - People - Equipment - Researcher Development - Technicians' Community - Research Facilities - Research Funding - Work with Us

Work with Us - Case Studies - Expert Solutions - Continuing Professional Development - CPD Courses - Collaborative Research - Company Creation - Consultancy - Corporate Partnerships - Facilities and Equipment - Intensive Industrial Innovation Programme - Knowledge Transfer Partnerships - Clinical Trials & Research - Procuring Goods & Services - Working with Newcastle - Tender Opportunities - Submitting an Invoice - Sustainable Procurement - Code of Conduct & Terms and Conditions - Contact Us - Business and Partnerships - Arrow - Degree Apprenticeships - Expert Solutions - Technology Transfer and Licensing - Contact Us - Health & Social Challenges - Creative Collaborations - Alumni

Alumni - Connect with alumni - Develop your career - Discover lifelong learning opportunities - Support future generations - News - Events - Contact us

Search for something

Search... Search...

Background

Newcastle University recognises that the responsible use of metrics can support the delivery of our Research Strategy’s ambition to catalyse transformative research within and between disciplines, and build a positive and thriving research culture. As such, we have produced this Policy Statement and associated guidance on appropriate use of quantitative research metrics as part of responsible research assessment.[1] The University became a signatory to DORA in 2017 and reaffirmed a commitment to the principles of DORA in the REF code of practice published in 2019. The use of research metrics falls within the context of the University Code of Good Practice in Research.

This policy statement builds on these commitments and a number of other prominent external initiatives, including the Leiden Manifesto for Research Metricsand Metric Tidereport. The latter urged UK institutions to develop a statement of principles on the use of quantitative indicators in research management and assessment, where metrics should be considered in terms of robustness(using the best available data); humility(recognising that quantitative evaluation can complement, but does not replace, expert assessment); transparency(keeping the collection of data and its analysis open to scrutiny); diversity(reflecting a multitude of research and researcher career paths); and reflexivity(updating our use of metrics to take account of the effects that such measures have had).

Implementation of this statement supports our strategic ambitions, and our institutional commitment to the Concordat for Research Integrityand the Concordat to Support the Career Development of Researchers, and is supported (and in some cases, mandated) by research funders in the UK (e.g., UKRI, Wellcome Trust etc.). This statement will evolve as we develop the University’s Research Culture Action Plan in consultation with students, colleagues and external partners.

This statement is integral to our commitment to developing all members of the research community across career stages and job families. It will support our aim to cultivate a research culture that recognises diverse contributions to research, and to build a research culture that values inclusive collaboration, integrity, transparency, and global visibility.

Implementation of the Responsible Use of Metrics Statement

We recognize that Newcastle University is a diverse university, and metrics will be used in ways that are sensitive to disciplinary and local contexts. This Policy Statement is deliberately flexible to allow for the diversity of contexts, and is not intended to provide a comprehensive set of rules.

We recognize that metrics are often more appropriate to evaluate the collective contribution of teams, and are less useful in the evaluation of individual contributions which may be diverse. At the University, we endeavour to use metrics to provide a collective picture of how we are performing against our strategy.

To help put this Statement into practice, we will provide an evolving set of guidance material to include more detailed discussion and examples of how these principles could be applied. Newcastle University is committed to valuing research and researchers based on their merits, not the merits of metrics. In the future, we will provide relevant training for all members of our research community through the Newcastle University Skills Academy.

The implementation of this Statement will include establishing a review and monitoring process, enabling regular institutional review of adherence to these principles, as well as mechanisms for reporting and addressing breaches of the principles.

Key progress:

Responsible use of Research Metrics policy statement and guiding principles published.
Creation of the Responsible use of Research Metrics Special Interest Group with members from Professional Services, expanded to include academics in 2025.
Creation of an implementation plan and signed off by the University Research Innovation Committee.
Appointment of Research Integrity Project Officer to provide professional services support in the responsible use of research metrics.
Appointment of an Associate Dean for Good Research Practice to provide academic leadership in the responsible use of research metrics.
Short statement created and agreed to be embedded in key University documents.

Planned for 2025/26:

The consideration of the merits and opportunities in becoming a signatory of the Coalition for Advancing Research Assessment (CoARA).
Continue to embed a short statement and signposting to our responsible use of research metrics commitments in all research staff and student recruitment, induction, appraisal, promotion and REF Internal and External Quality Assessment processes.
Develop training and guidance on the responsible use of research metrics that can be used by all colleagues.
Create a glossary of terms about the responsible use of research metrics and what they relate to at Newcastle University.

Guiding Principles for the Responsible Use of Metrics at Newcastle University

This guidance is provided to ensure that research assessment activities are carried out in line with the three principles below.

Metrics should:

Support expert judgement

Research assessment should always be based on expert judgement with metrics used to support and inform this assessment.
Quality, influence, excellence and impact of research are typically abstract and subjective concepts that prohibit direct measurement. There is no simple way to measure research quality, and quantitative approaches can only be interpreted as indirect proxies for quality. Superficial inferences of the quality of research by use of research metrics alone in research evaluations can be misleading.
Irresponsible use of metrics alone can pose the risk of incentivizing undesirable behaviours, such as focusing on number rather than quality of publications, chasing publications in journals with a high Journal Impact Factor (JIF) regardless of whether this is the most appropriate venue for publication, or discouraging the use of open research approaches such as pre-prints or data-sharing.

Be fit for purpose

Data should only be obtained from reliable and accurate sources and due consideration should be given to ensure data quality.
Metrics that we use should align with what we value as important. For example, categories that we value include: (a) research quality, (b) research volume, income, and activity levels (critical mass), (c) research impact, and (d) research environment.
Disciplinary differences should be accounted for. Researchers from different disciplines have different perspectives of what characterises research quality, and different approaches for determining what constitutes a significant research output (for example, the relative importance of book chapters vs journal articles). As some metrics lend themselves to certain discipline areas to the detriment of others, we will work hard with the wider sector to ensure that research in all our Faculties is understood and its value represented, across STEM and SHAPE disciplines.
All research outputs must be considered on their own merits, in an appropriate context that reflects the needs and diversity of research fields and outcomes. Metrics should be used in the correct context and users should be mindful of changes in information that might affect the underlying assessment.
Quantitative indicators should be selected from those that are widely used and easily understood to ensure that the process is transparent and they are being applied appropriately. Likewise, any quantitative goals or benchmarks must be open to scrutiny.
If goals or benchmarks are expressed quantitatively, care should be taken to avoid the metric itself becoming the target of research activity at the expense of research quality.
Metrics that are useful for aggregate assessment may not be appropriate for use at a different scale, for example in assessment of an individual output. For example, the JIF of a journal is a useful measure of the whole content of the journal, but cannot be extrapolated to infer the quality of every article within the journal.

Be transparent

When metrics are used to form an assessment, the data sources and assumptions should be acknowledged. New and alternative metrics are continuously being developed to inform the reception, usage, and value of all types of research output. Any new or non-standard metric or indicator must be used and interpreted in keeping with the other principles listed here for more traditional metrics. Additionally, the sources and methods behind such metrics should be considered as to whether they are vulnerable to being gamed, manipulated, or fabricated.
Metrics (in particular bibliometrics) are available from a variety of services, with differing levels of coverage, quality and accuracy, and these aspects should be considered when selecting a source for data or metrics. Where necessary, such as in the evaluation of individual researchers, a source that allows records to be verified and curated should be used to ensure records are comprehensive and accurate.

[1] The term “research assessment” refers to the evaluation of research quality and measurements of research inputs, outputs and impacts, and embraces both qualitative and quantitative methodologies, including the application of bibliometric indicators and mapping, and peer review. Research assessment can take place across a wide range of contexts, from responding to national research assessment exercises to evaluating individual candidates during recruitment exercises.\ \ Further Resources\ \

Who to contact

Research Policy, Intelligence and Ethics, Research and Innovation

Policy Approved by Senate 30th June 2021

Responsible Use of Research Metrics

On this page

On this page

On this page

Background

Implementation of the Responsible Use of Metrics Statement

Key progress:

Planned for 2025/26:

Guiding Principles for the Responsible Use of Metrics at Newcastle University

Metrics should:

Who to contact