6. Has the HEARTSMAP tool’s inter-user reliability been evaluated?

Yes. HEARTSMAP’s inter-user reliability was evaluated among pediatric ED clinicians locally at BC Children’s Hospital in 2014, and among ED clinicians in general EDs in 2016.

At BC Children’s Hospital, data from 104 ED mental health presentations was extracted to construct clinical narratives. A panel of ED clinician evaluators reviewed each of these narratives. The panel included: 2 pediatric emergency medicine physicians, 1 nurse practitioner, and 1 bedside nurse. The results below summarize the agreement among the clinician reviewers with regard to consulting psychiatry in the ED:

Comparisons Kappa Value Interpretation¹
Pediatric ED physician vs. Pediatric ED physician 0.7 Substantial Agreement
Pediatric ED physician vs. Nurse practitioner 0.6 Moderate Agreement
Pediatric ED physician vs. Bedside nurse 0.5 Moderate Agreement

Eleven ED clinicians from 5 health authorities – Provincial Health Services Authority, Providence Health Care, Vancouver Coastal Health, Fraser Health Authority, and Interior Health Authority validated the HEARTSMAP tool in 2016 (3). These ED clinicians, including physicians, social workers, and psychiatric liaison nurses, were asked to use the HEARTSMAP tool to evaluate 50 fictional narratives describing a variety of pediatric mental health cases. Their agreement with regards to consulting psychiatry in the ED or referring to a community based program such as a Crisis Response Team, a Child and Youth Mental Health Team (CYMHT), social services through the Ministry of Child and Family Development, or a youth health clinic, is summarized below:

Resource Kappa Value Interpretation²
Psychiatry consultation in the ED
Outpatient Crisis Response Team
Outpatient CYMHT
MCFD Social Services
Youth Health Clinic/Substance & addiction services
  1. Viera AJ, Garrett JM. Understanding Interobserver Agreement : The Kappa Statistic. Fam Med. 2005;37(5):360-363.
  2. Hallgren KA. Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutor Quant Methods Psychol. 2012;8(1):23-34.
  3. Virk, P., Stenstrom, R., & Doan, Q. (2018). Reliability testing of the HEARTSMAP psychosocial assessment tool for multidisciplinary use and in diverse emergency settings. Paediatrics & Child Health, 1–6. https://doi.org/10.1093/pch/pxy017

62 Replies to “6. Has the HEARTSMAP tool’s inter-user reliability been evaluated?”

  1. What i don’t understood is if truth be told how you are not really much more neatly-liked than you might be right now. You’re very intelligent. You know thus considerably in terms of this topic, made me personally imagine it from a lot of numerous angles. Its like women and men aren’t involved unless it is one thing to do with Woman gaga! Your own stuffs great. All the time handle it up!

  2. Hi there would you mind sharing which blog platform
    you’re using? I’m looking to start my own blog soon but I’m having a difficult time selecting between BlogEngine/Wordpress/B2evolution and Drupal.
    The reason I ask is because your design and style seems
    different then most blogs and I’m looking for something unique.
    P.S Apologies for being off-topic but I had to ask!

  3. Hey there! I understand this is kind of off-topic however I needed to ask.
    Does building a well-established website like
    yours require a large amount of work? I’m completely new to running a blog however
    I do write in my journal daily. I’d like to start a blog so I
    can share my experience and feelings online. Please let me know if you have any kind of suggestions or tips
    for new aspiring blog owners. Appreciate it!

  4. Great blog! Do you have any suggestions for aspiring
    writers? I’m hoping to start my own website soon but I’m a little lost
    on everything. Would you suggest starting with a free
    platform like WordPress or go for a paid option? There are
    so many choices out there that I’m completely overwhelmed ..

    Any suggestions? Thank you!

  5. My spouse and I stumbled over here by a different page and thought I may as well check things out.
    I like what I see so i am just following
    you. Look forward to looking at your web page again.

  6. Wonderful article! This is the type of information that should be shared across the net.
    Disgrace on the search engines for no longer positioning this post upper!
    Come on over and discuss with my website . Thank you =)

  7. I constantly spent my half an hour to read this webpage’s articles or reviews all the
    time along with a cup of coffee.

  8. Terrific article! That is the type of information that are meant to be shared across
    the web. Shame on the seek engines for no longer positioning this submit higher!
    Come on over and seek advice from my site . Thank you =)

  9. I was wondering if you ever considered changing the layout of your blog?
    Its very well written; I love what youve got to
    say. But maybe you could a little more in the way of
    content so people could connect with it better. Youve got an awful
    lot of text for only having one or 2 images. Maybe you could space it out better?

  10. Thanks for your personal marvelous posting! I genuinely enjoyed reading it,
    you could be a great author. I will be sure to bookmark your blog and will often come back at some point.
    I want to encourage continue your great job, have a nice evening!

  11. Heya! I just wanted to ask if you ever have any trouble with hackers?

    My last blog (wordpress) was hacked and I ended up losing a few
    months of hard work due to no backup. Do you have any methods to protect against hackers?

  12. Have you ever thought about publishing an ebook or guest authoring
    on other blogs? I have a blog based upon on the same subjects you discuss and would love to have
    you share some stories/information. I know my audience would
    enjoy your work. If you’re even remotely interested,
    feel free to shoot me an email.

  13. Hello there! Quick question that’s entirely off topic.
    Do you know how to make your site mobile friendly?

    My website looks weird when viewing from my iphone4.
    I’m trying to find a template or plugin that might be able to fix this problem.
    If you have any suggestions, please share. Thanks!

  14. I’m really impressed with your writing skills and also
    with the layout on your blog. Is this a paid theme or did you modify it yourself?
    Either way keep up the excellent quality writing, it’s rare to see a nice blog like this
    one these days.

  15. Please let me know if you’re looking for a author for your blog.
    You have some really great posts and I feel I would be a
    good asset. If you ever want to take some of the load off, I’d love
    to write some articles for your blog in exchange for a link back to mine.
    Please blast me an e-mail if interested. Cheers!

  16. I like the valuable info you provide in your articles. I’ll bookmark your blog and check again here
    frequently. I’m quite sure I’ll learn plenty of new stuff
    right here! Good luck for the next!

  17. You really make it seem so easy with your presentation but I find this topic to be really something which I think I would never understand.

    It seems too complex and extremely broad for me.
    I’m looking forward for your next post, I will try to get the hang of it!

  18. I’m really impressed with your writing skills as well as with the layout on your blog.

    Is this a paid theme or did you modify it yourself?
    Either way keep up the nice quality writing, it’s rare to see a nice blog like this one nowadays.

  19. Hi! I know this is kinda off topic nevertheless I’d figured I’d
    ask. Would you be interested in exchanging links or
    maybe guest authoring a blog article or vice-versa? My website addresses a lot of the same topics
    as yours and I think we could greatly benefit from
    each other. If you are interested feel free to send me an email.
    I look forward to hearing from you! Awesome blog by the way!

  20. I believe that is among the so much important information for me.

    And i’m satisfied reading your article. But should remark
    on few normal things, The website style is great, the articles is in point of
    fact excellent : D. Good job, cheers

Leave a Reply

Your email address will not be published. Required fields are marked *