Research Impact Challenge 6: What I do, what’s important to me, and what “counts”?

Welcome to Challenge 6 of WSU’s Research Impact Challenge! 

The Challenges 1–5 focused on managing your scholarly identity online. Starting with Challenge 6, we’ll shift gears to talk about how scholarly work makes a difference, and how those contributions are measured, reported, and valued. There are very specific ways to address different aspects of research impact—measures such as the Journal Impact Factor and the author h-index come to mind—and we’ll talk about some of these in the coming Challenges.

But Challenge 6 invites you to think broadly and inclusively about the work you’re doing, why it matters, and how you can effectively communicate that message to someone else, whether a member of the public, a scholar in a different field, or an administrator evaluating your productivity.

Rather than jumping right into the metrics and tools used to evaluate research, today’s activity focuses on you and your work.

Let’s get started.

Here’s how to do it: 

1. Use the Challenge 6 Worksheet to indicate all the types of scholarly work that you do (probably much more than you realize!), and to what extent that work is valued by you, your peers, and those who may be in a position to evaluate you.
   • There are a few caveats as you tackle this challenge:
     • Not every item on the worksheet will apply to you. This is a very broad list that attempts to cover many disciplines and specialties. We hope that some of these types of work will resonate with each person.
     • As you complete the challenge, try to focus roughly on the last five years, rather than on your entire career, or on the distant future
     • The types of work described on the worksheet are grouped into categories of research, teaching, service, and engagement, but you may find that for you, some activities align more with one area than another.
     • Finally, you’ll find this is a long list but it is certainly not an exhaustive one—if a type of work you do doesn’t appear in the list, feel free to add it in a blank row.
   • The goal of this exercise is to arrive at a clear, current, and comprehensive awareness of your scholarly labor, as well as your own assumptions about what work is important, that you can bring to bear on the rest of the week’s exercises.

2. From the worksheet, pick one area of work that is important to you, but that you’ve chosen to exclude from your CV, or that you anticipate won’t help you in an evaluation. In an ideal world, how would you express the impact of this work? Consider what indicators of success you would be looking for, how you would capture them, and how you would communicate them in a systematic way. 

Bonus Challenge

Locate the guidelines or evaluation criteria for the next major milestone at which you expect to be evaluated (application for a postdoc position, third year review, application for promotion, etc.). Read the requirements with fresh eyes, reflecting on your assessment of your own work. Is there alignment between what’s important to you, what you’re doing, and what’s expected?

What next?

Across disciplines, groups of scholars are working together to establish new norms for evaluating scholarly work. Here are a few examples:

• The HumanitiesHSS Initiative, launched in 2016, has created a framework for defining “a scholarly life well lived.”
• Designed to be read by scholars who are used to the monograph and the journal article, the MLA Guidelines for the Evaluation of Digital Scholarship aim to provide clear guidance for evaluating the quality and rigor of scholarship that takes the form of a website or other digital resource.
• Checkoway, Barry. “Professionally Related Public Service as Applied Scholarship: Guidelines for the Evaluation of Planning Faculty.” Journal of Planning Education and Research. June 1, 1998. DOI: 10.1177/0739456X9801700410.
  • Though now more than 20 years old, this report describes concrete metrics/data points that scholars should provide, and evaluators should look for, to assess certain types of service as applied scholarship.
• The Becker Medical Library Model for the Assessment of Research Impact, established in 2009 and updated in 2011, provides a framework for tracking how research outputs affect science and society at large.
  • See also, The Becker List: Impact Indicators
  • For background information on the Becker Model, see “Beyond citation analysis: a model for assessment of research impact,” published in the Journal of the Medical Library Association in 2010.
• The Research Impact Zotero library is a collection of readings related to assessment of research impact with an emphasis on biomedical research.

Learn more:

• “The concept of research impact pervades contemporary academic discourse – but what does it actually mean?” LSE Impact Blog. January 9, 2018.
• This reflection from a political scientist speaks to the frustration of balancing work that is important to him, work that makes a difference to the world, and work that counts for academic credit: “Understanding academic impact: fear and failure, stealth and seeds.” OUPBlog. August 8, 2018.

Preparing for the next challenge: 

Congratulations! You’ve completed Day 6 of the Research Impact Challenge! Today we’ve reflected the work we do, and what gets counted. Tomorrow we’ll start to dig into how scholarly impact is measured, how to identify useful metrics, and how to apply them appropriately.