6. Has the HEARTSMAP tool’s inter-user reliability been evaluated?

Yes. HEARTSMAP’s inter-user reliability was evaluated among pediatric ED clinicians locally at BC Children’s Hospital in 2014, and among ED clinicians in general EDs in 2016.

At BC Children’s Hospital, data from 104 ED mental health presentations was extracted to construct clinical narratives. A panel of ED clinician evaluators reviewed each of these narratives. The panel included: 2 pediatric emergency medicine physicians, 1 nurse practitioner, and 1 bedside nurse. The results below summarize the agreement among the clinician reviewers with regard to consulting psychiatry in the ED:

Comparisons Kappa Value Interpretation¹
Pediatric ED physician vs. Pediatric ED physician 0.7 Substantial Agreement
Pediatric ED physician vs. Nurse practitioner 0.6 Moderate Agreement
Pediatric ED physician vs. Bedside nurse 0.5 Moderate Agreement

Eleven ED clinicians from 5 health authorities – Provincial Health Services Authority, Providence Health Care, Vancouver Coastal Health, Fraser Health Authority, and Interior Health Authority validated the HEARTSMAP tool in 2016 (3). These ED clinicians, including physicians, social workers, and psychiatric liaison nurses, were asked to use the HEARTSMAP tool to evaluate 50 fictional narratives describing a variety of pediatric mental health cases. Their agreement with regards to consulting psychiatry in the ED or referring to a community based program such as a Crisis Response Team, a Child and Youth Mental Health Team (CYMHT), social services through the Ministry of Child and Family Development, or a youth health clinic, is summarized below:

Resource Kappa Value Interpretation²
Psychiatry consultation in the ED
Outpatient Crisis Response Team
Outpatient CYMHT
MCFD Social Services
Youth Health Clinic/Substance & addiction services
  1. Viera AJ, Garrett JM. Understanding Interobserver Agreement : The Kappa Statistic. Fam Med. 2005;37(5):360-363.
  2. Hallgren KA. Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutor Quant Methods Psychol. 2012;8(1):23-34.
  3. Virk, P., Stenstrom, R., & Doan, Q. (2018). Reliability testing of the HEARTSMAP psychosocial assessment tool for multidisciplinary use and in diverse emergency settings. Paediatrics & Child Health, 1–6. https://doi.org/10.1093/pch/pxy017

196 Replies to “6. Has the HEARTSMAP tool’s inter-user reliability been evaluated?”

  1. What i don’t understood is if truth be told how you are not really much more neatly-liked than you might be right now. You’re very intelligent. You know thus considerably in terms of this topic, made me personally imagine it from a lot of numerous angles. Its like women and men aren’t involved unless it is one thing to do with Woman gaga! Your own stuffs great. All the time handle it up!

  2. Hi there would you mind sharing which blog platform
    you’re using? I’m looking to start my own blog soon but I’m having a difficult time selecting between BlogEngine/Wordpress/B2evolution and Drupal.
    The reason I ask is because your design and style seems
    different then most blogs and I’m looking for something unique.
    P.S Apologies for being off-topic but I had to ask!

  3. Hey there! I understand this is kind of off-topic however I needed to ask.
    Does building a well-established website like
    yours require a large amount of work? I’m completely new to running a blog however
    I do write in my journal daily. I’d like to start a blog so I
    can share my experience and feelings online. Please let me know if you have any kind of suggestions or tips
    for new aspiring blog owners. Appreciate it!

  4. Great blog! Do you have any suggestions for aspiring
    writers? I’m hoping to start my own website soon but I’m a little lost
    on everything. Would you suggest starting with a free
    platform like WordPress or go for a paid option? There are
    so many choices out there that I’m completely overwhelmed ..

    Any suggestions? Thank you!

  5. My spouse and I stumbled over here by a different page and thought I may as well check things out.
    I like what I see so i am just following
    you. Look forward to looking at your web page again.

  6. Wonderful article! This is the type of information that should be shared across the net.
    Disgrace on the search engines for no longer positioning this post upper!
    Come on over and discuss with my website . Thank you =)

  7. I constantly spent my half an hour to read this webpage’s articles or reviews all the
    time along with a cup of coffee.

  8. Terrific article! That is the type of information that are meant to be shared across
    the web. Shame on the seek engines for no longer positioning this submit higher!
    Come on over and seek advice from my site . Thank you =)

  9. I was wondering if you ever considered changing the layout of your blog?
    Its very well written; I love what youve got to
    say. But maybe you could a little more in the way of
    content so people could connect with it better. Youve got an awful
    lot of text for only having one or 2 images. Maybe you could space it out better?

  10. Thanks for your personal marvelous posting! I genuinely enjoyed reading it,
    you could be a great author. I will be sure to bookmark your blog and will often come back at some point.
    I want to encourage continue your great job, have a nice evening!

  11. Heya! I just wanted to ask if you ever have any trouble with hackers?

    My last blog (wordpress) was hacked and I ended up losing a few
    months of hard work due to no backup. Do you have any methods to protect against hackers?

  12. Have you ever thought about publishing an ebook or guest authoring
    on other blogs? I have a blog based upon on the same subjects you discuss and would love to have
    you share some stories/information. I know my audience would
    enjoy your work. If you’re even remotely interested,
    feel free to shoot me an email.

  13. Hello there! Quick question that’s entirely off topic.
    Do you know how to make your site mobile friendly?

    My website looks weird when viewing from my iphone4.
    I’m trying to find a template or plugin that might be able to fix this problem.
    If you have any suggestions, please share. Thanks!

  14. I’m really impressed with your writing skills and also
    with the layout on your blog. Is this a paid theme or did you modify it yourself?
    Either way keep up the excellent quality writing, it’s rare to see a nice blog like this
    one these days.

  15. Please let me know if you’re looking for a author for your blog.
    You have some really great posts and I feel I would be a
    good asset. If you ever want to take some of the load off, I’d love
    to write some articles for your blog in exchange for a link back to mine.
    Please blast me an e-mail if interested. Cheers!

  16. I like the valuable info you provide in your articles. I’ll bookmark your blog and check again here
    frequently. I’m quite sure I’ll learn plenty of new stuff
    right here! Good luck for the next!

  17. You really make it seem so easy with your presentation but I find this topic to be really something which I think I would never understand.

    It seems too complex and extremely broad for me.
    I’m looking forward for your next post, I will try to get the hang of it!

  18. I’m really impressed with your writing skills as well as with the layout on your blog.

    Is this a paid theme or did you modify it yourself?
    Either way keep up the nice quality writing, it’s rare to see a nice blog like this one nowadays.

  19. Hi! I know this is kinda off topic nevertheless I’d figured I’d
    ask. Would you be interested in exchanging links or
    maybe guest authoring a blog article or vice-versa? My website addresses a lot of the same topics
    as yours and I think we could greatly benefit from
    each other. If you are interested feel free to send me an email.
    I look forward to hearing from you! Awesome blog by the way!

  20. I believe that is among the so much important information for me.

    And i’m satisfied reading your article. But should remark
    on few normal things, The website style is great, the articles is in point of
    fact excellent : D. Good job, cheers

  21. What i do not understood is in truth how you are not really a lot more well-favored than you may be right
    now. You’re so intelligent. You recognize therefore significantly when it comes to this matter, made me for my part imagine it from a lot of various angles.
    Its like women and men are not involved until it’s one
    thing to accomplish with Woman gaga! Your own stuffs outstanding.
    All the time take care of it up!

  22. I will immediately grasp your rss feed as I can not find your email subscription link or newsletter service. Do you’ve any? Please let me recognise in order that I could subscribe. Thanks.

  23. Good day! I could have sworn I’ve been to this site before but after looking at a few of the articles I realized it’s new to me. Anyways, I’m certainly pleased I stumbled upon it and I’ll be bookmarking it and checking back regularly!

  24. Hey there superb blog! Does running a blog similar to this require a lot of work?
    I have virtually no expertise in computer programming however I was hoping
    to start my own blog soon. Anyhow, should you have any ideas or tips for new blog owners
    please share. I understand this is off topic however I simply needed to ask.
    Thanks a lot!

  25. Fantastic beat ! I wish to apprentice whilst
    you amend your site, how could i subscribe for a blog site?
    The account aided me a appropriate deal. I had been a little bit acquainted of this your
    broadcast offered shiny clear idea

  26. Good post. I learn something more challenging on completely different blogs everyday. It is going to always be stimulating to learn content material from other writers and apply a bit of one thing from their store. I抎 desire to use some with the content on my blog whether or not you don抰 mind. Natually I抣l provide you with a link on your internet blog. Thanks for sharing.

  27. you are actually a excellent webmaster. The site
    loading velocity is incredible. It sort of feels that you’re doing any distinctive trick.
    Moreover, The contents are masterwork. you’ve done a magnificent process on this matter!

  28. I just like the valuable information you provide to your articles. I will bookmark your weblog and take a look at once more here frequently. I am relatively certain I’ll learn a lot of new stuff proper right here! Good luck for the next!

  29. My spouse and I absolutely love your blog and find
    almost all of your post’s to be exactly I’m looking for.
    Do you offer guest writers to write content to suit your needs?
    I wouldn’t mind producing a post or elaborating on most of the subjects you write in relation to here.
    Again, awesome website!

  30. Its like you read my mind! You appear to know so much about this, like
    you wrote the book in it or something. I
    think that you could do with some pics to drive the message home
    a bit, but instead of that, this is wonderful blog.
    An excellent read. I’ll definitely be back.

  31. Oh my goodness! an incredible article dude. Thanks However I’m experiencing challenge with ur rss . Don抰 know why Unable to subscribe to it. Is there anyone getting identical rss drawback? Anyone who knows kindly respond. Thnkx

  32. You really make it appear really easy along with your presentation however I to find
    this matter to be actually something that I believe I might by no
    means understand. It sort of feels too complex and extremely wide for me.
    I’m taking a look ahead to your next publish, I’ll attempt to get the dangle of it!
    natalielise plenty of fish

  33. I just like the helpful information you provide on your articles.
    I will bookmark your blog and take a look at once more right here regularly.
    I’m reasonably certain I will be informed many new stuff proper here!
    Best of luck for the next!

  34. First of all I want to say excellent blog!
    I had a quick question in which I’d like to ask
    if you do not mind. I was curious to find out how you center yourself and clear your
    thoughts before writing. I’ve had a difficult time clearing my thoughts in getting my ideas
    out. I do take pleasure in writing however it just seems like the first 10 to 15 minutes are wasted simply just trying to figure out how
    to begin. Any recommendations or tips? Thanks!

  35. Undeniably believe that which you stated. Your favorite
    justification seemed to be on the net the easiest thing to be aware of.
    I say to you, I definitely get annoyed while people think
    about worries that they just do not know about. You managed to hit the
    nail upon the top and defined out the whole thing without having side
    effect , people could take a signal. Will probably be back to get more.
    Thanks

  36. Have you ever considered publishing an ebook or guest authoring on other sites?
    I have a blog based upon on the same information you discuss and would love to have you share some stories/information. I know my subscribers would value your work.

    If you are even remotely interested, feel free to send me an email.

  37. I would like to thank you for the efforts you have put in writing this site.I am hoping to view the same high-grade blog posts by you in the futureas well. In fact, your creative writing abilities has encouraged me to get my own blog now 😉

Leave a Reply

Your email address will not be published. Required fields are marked *