Med Comms eROI Calculator

Behaviour change: The probability that behaviour will change post initiative was distilled into 4 primary elements: 

1. Was a needs assessment completed? Research shows that behaviour change is more probable if the HCPs believe the initiative will help them solve for a challenge in their day-to-day.
   • The multiplier is 1.5 
2. Does the initiative include real-world scenarios? Whether this is case-based discussions, clinic simulations, or in some way situationally-oriented, behaviour change is more probable if the initiative is directly applicable to the clinic. 
   • The multiplier is 2.0
3. Was there active participation and collaboration? Peer-to-peer learning is a powerful mover of behaviour change. HCPs trust their colleagues and are likely to apply what they hear from their colleagues. In small group settings (<20 participants at one event), this is much easier to obtain.
   • The multiplier is 2.5
4.  Was this initiative part of multiple tactics? Research has demonstrated that the application of reinforcing strategies increases the probability that behaviour will change. If the initiative is a singular event, it is less likely behaviour will change. 
   • The multiplier is 2.0
     

The maximum score for behaviour change is a 4.5fold multiplier. A well-designed medical education program will score well in this section. Advisory boards can score depending on the agenda design. Market research does not score in this section. 

Insights: Insights from clinicians support the development of strategic imperatives. A multiplier is assigned to each stepwise increase in the number of questions asked, data collected for and analyzed. To score on this section, the questions posed should provide strategic value, and the data analyzed and interpreted. 

The maximum score in this section is a 4fold multiplier. Advisory boards and market research will often score well, while medical education typically scores lower. To score in this section with medical education programs, a seamless integration of pre/post questionnaires, outcomes assessments, polling questions, and evaluations will help. 

Goodwill: The probability that goodwill is provided was distilled into 3 primary elements: 

1. Was honoraria provided to the participant? 
   • The multiplier is 1.5 
2. Was there in-person peer interaction?
   • The multiplier is 1.5 
3. Were CME credits provided? 
   • The multiplier is 1.2 
     

An in-person accredited Medical Education program will score two of these three elements. An in-person Advisory Board will also score two of these three elements. Market research will score only the honoraria element. 

eROI Simulation Data 

Simulation Parameters: 

Medical Education Simulation Parameters 

• Cost (CAD): $50,000 → 300,000
• Participants: 50 → 200
• Avg Minutes per HCP: 60 → 120
• FMV ($/hour): 400 (fixed)
• Behaviour Multiplier: 1.0–4.0 (no constraint)
• Insights Multiplier: 1.0–1.6 (constrained)
• Goodwill Factor:  
• CME credits (1.2) on/off
• In-person (1.5) on/off
• Possible goodwill values: 1.0 / 1.2 / 1.5 / 1.8 (no honoraria)
• Simulations: 5,000 
  

Advisory Board Simulation Parameters 

• Cost (CAD): $50,000 → 220,000
• Participants: 10 → 20
• Avg Minutes per HCP: 180 → 360
• FMV ($/hour): 550 (fixed)
• Behaviour Multiplier: 2.5 if “Active participation and collaboration” included, else 1.0
• Goodwill Factor: product of honoraria (1.5) on/off and in-person (1.5) on/off (→ 1.0, 1.5, or 2.25)
• Insights Factor: randomly 3.4 or 4.0
• Simulations: 5,000 

Market Research Simulation Parameters 

• Cost (CAD): 50,000 → 300,000
• Participants: 50 → 200
• Avg Minutes per HCP: 30 → 60
• FMV ($/hour): 550 (fixed)
• Goodwill: Honoraria ON (1.5); CME credits & In-person OFF
• Insights: randomly 2.8, 3.4, or 4.0
• Behaviour: No behaviour multiplier (1.0)
• Simulations: 5,000 

• Medical education has high upside but has a large standard deviation. In other words, a well-designed med ed program will score a high eROI, but there is also higher risk of low eROI.
• Advisory boards are a relatively safe investment. The standard deviation is lower and tends to score well. However, the upside is more restricted vs medical education.
• Market research doesn’t have access to the behaviour change elements so these tactics are heavily reliant on the number of participants and overall cost. There is a large standard deviation so these programs can score well as long as recruitment is solid and cost are in check.
• With any med comms tactic, cost per engaged minute is the most sensitive eROI determinant. Generally, once a piece of content is developed for any med comms tactic, the more people that genuinely engage, the higher the eROI. 

Cervero RM, Gaines JK. The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. J Contin Educ Health Prof. 2015 Spring;35(2):131-8.  

Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995 Sep 6;274(9):700-5.  

Forsetlund L, O'Brien MA, Forsén L, Reinar LM, Okwen MP, Horsley T, Rose CJ. Continuing education meetings and workshops: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2021 Sep 15;9(9):CD003030. doi: 10.1002/14651858.CD003030.pub3. PMID: 34523128; PMCID: PMC8441047. 

Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007 Jan;41(1):23-31. doi: 10.1111/j.1365-2929.2006.02644.x. PMID: 17209889. 

Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002 Sep 4;288(9):1057-60. doi: 10.1001/jama.288.9.1057. PMID: 12204068. 

Thistlethwaite JE, Davies D, Ekeocha S, Kidd JM, MacDougall C, Matthews P, Purkis J, Clay D. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23. Med Teach. 2012;34(6):e421-44. doi: 10.3109/0142159X.2012.680939. PMID: 22578051. 

Pfeffer J, Sutton RI. (2006). “Evidence-based management.” Harvard Business Review, 84(1):62–74. 

Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009 Apr 15;2009(2):CD003030. doi: 10.1002/14651858.CD003030.pub2. 

Campbell EG, Gruen RL, Mountford J, Miller LG, Cleary PD, Blumenthal D. A national survey of physician-industry relationships. N Engl J Med. 2007 Apr 26;356(17):1742-50. doi: 10.1056/NEJMsa064508. PMID: 17460228.