Psychometric properties of the Italian version of the Nurses’ Global Assessment of Suicide Risk (NGASR) scale

Proprietà psicometriche della versione italiana della Nurses’ Global Assessment of Suicide Risk (NGASR)

PAOLO FERRARA1, STEFANO TERZONI1, ARMANDO D’AGOSTINO2, JOHN R. CUTCLIFFE3,
YELISSA POZO FALEN1, SILVANA ESTHER CORIGLIANO4, LORIS BONETTI5, ANNE DESTREBECQ2,
ORSOLA GAMBINI2
*E-mail: paolo.ferrara@asst-santipaolocarlo.it

1San Paolo Bachelor School of Nursing, University of Milan, San Paolo Teaching Hospital, Milan
2Department of Health Sciences for Health, University of Milan
3Cutcliffe Consulting (operating in Canada, USA and Europe), Amherstview, Ontario
4ASST Ovest Milanese, Legnano (MI)
5Nursing Research and Development Unit, Istituto oncologico della Svizzera Italiana EOC - Ente Ospedaliero Cantonale, Bellinzona


SUMMARY. Aim. People with mental disorders have higher risk of suicide compared to the general population. Assessment of risk factors can help nurses reducing suicidal risk. The Nurses’ Global Assessment of Suicide Risk scale (NGASR) has proven valid and reliable in supporting the nursing evaluation of suicidal risk in different studies. The aim of the study was to examine the psychometric properties of the NGASR in the Italian population. Methods. We translated the scale and administered it to a sample of 121 patients admitted to acute psychiatric wards. Results. The Content Validity Index-Scale (CVI-S) was 96.7%, the correlation with the Scale for Suicide Ideation (SSI) score was high (r=.98, p<.001). Inter-rater reliability (rho=.97, p<.001), and test-retest stability (p=.96) were satisfactory. Factor analysis pointed out 5 factors and the 15 items of the NGASR-ita explained 61.29% of total variance. Of the 121 subjects assessed upon admission, 25.62% had average or higher suicidal risk. Discussion and conclusions. The use of valid screening tools in support of Suicide risk assessment is recommended. The NGASR-ita is a valid and reliable tool, suitable for nursing assessment of suicidal risk in the acute psychiatric setting.

KEY WORDS: Risk assessment, suicide, psychiatric nurses, scales.


RIASSUNTO. Scopo. Le persone affette da disturbi mentali presentano un rischio di suicidio superiore rispetto alla popolazione generale. La valutazione dei fattori di rischio può aiutare i professionisti a ridurre la probabilità che l’evento si verifichi. La scala Nurses’ Global Assessment of Suicide Risk (NGASR) si è dimostrata valida e affidabile, in diversi contesti, nel supportare la valutazione infermieristica del rischio di suicidio. L’obiettivo di questo studio è stato quello di testare le proprietà psicometriche della versione italiana della NGASR. Metodi. È stata prodotta la versione italiana della scala, successivamente somministrata a un campione di 121 pazienti ricoverati all’interno di due Servizi Psichiatrici di Diagnosi e Cura. Risultati. Il Content Validity Index-Scale (CVI-S) è risultato pari a 96,7%, la correlazione con i punteggi della Scale for Suicide Ideation (SSI) è forte (r=,98, p<0,001). L’affidabilità intervalutatore (rho=,97, p<,001) e la stabilità del test (p=0,96) sono state soddisfacenti. L’analisi fattoriale esplorativa ha evidenziato la presenza di 5 fattori e i 15 item dell’NGASR-ita hanno spiegato il 61,29% della varianza totale. Dei 121 soggetti valutati al momento dell’ammissione in reparto, il 25,62% presentava un rischio suicidario medio o superiore. Discussione e conclusioni. L’utilizzo di uno strumento di screening a sostegno della valutazione del rischio è altamente raccomandato. La NGASR-ita si è dimostrata valida e affidabile e dunque adatta alla valutazione infermieristica del rischio suicidario nel contesto psichiatrico italiano.

PAROLE CHIAVE: valutazione del rischio, suicidio, infermieri di salute mentale, scale.

INTRODUCTION
Suicide is a major public health issue worldwide. According to the World Health Organization1 suicide rates have increased by 60% in the past 45 years. About 800 000 people commit suicide every year, meaning approximately one person every 40 seconds. Moreover, for every adult who completes suicide, another 20 are estimated to make a suicide attempt. Suicide occurs throughout the lifespan and is the second leading cause of death among people aged 15 to 29 globally, accounting for 1.4% of all deaths worldwide and making it the 17th leading cause of death in 2015 2.
These data highlight the extent of the problem and the need to address it with appropriate prevention strategies3; for this reason, in 2014, the WHO published the first report ‘Preventing Suicide: a global imperative’, aimed at raising awareness of the problem and giving encouragement and support to countries for developing or strengthening global strategies for suicide prevention4. Countries participating in the World Health Organisation’s ‘Mental Health Action Plan 2013-2020’ aim to reduce their national suicide rates by at least 10%5. In Italy, the problem is well documented6 with a rate of 6.5 events per 100,000 inhabitants in 2015, according to the latest data by the National Institute of Statistics (ISTAT). The problem is so urgent that a recommendation has been issued by the Ministry of Health7. Moreover, in hospital settings, the fifth Italian report on sentinel events in 2015 confirmed that suicide is the second most important event reported, with an incidence as high as 15.4%.
Several studies suggest that persons diagnosed with mental health disorders have a higher lifetime risk of suicide attempts than the general population8-10. In particular, people with ‘depressive disorders’ appear to have a 20-fold increase in suicide risk compared to the general population11; in addition, 95% of those who attempt suicide are suffering from mental disorders12. Given that suicide is a complex phenomenon13 and currently, in some cases, it is difficult to predict, and prevent suicide, even in professional care settings despite reasonable attention, efforts in pharmacological and psychotherapeutic care14-17, a rigorous multidimensional risk assessment of suicidal thoughts, feelings and behaviours to identify high-risk individuals is recommended, especially in the field of mental health care18. However, the multifactorial nature of the problem makes assessment very difficult19,20.
Suicidality requires a multidimensional and multidisciplinary response21,22; nursing assessment in this field is aimed at researching specific risk factors. Assessment helps nurses to identify those at high risk, and to implement interventions aimed at reducing the risk of self-injurious behaviour.
The principle of suicidal risk assessment includes a careful examination of the personal, interpersonal and social conditions of each individual, which can support clinical judgement. Furthermore, the use of valid screening tools in support of risk assessment is highly recommended23-25.
There are many assessment tools for suicidal risk described in relevant psychiatric and/or suicidology literature12,26-28, only a limited number of which are generally considered suitable for medical or nursing practice29. Some are focused on specific aspects of suicidal tendency, such as hopelessness or depressive symptoms while others are too time consuming for clinical use. The Nurses’ Global Assessment of Suicide Risk (NGASR) scale, instead, can be used by nurses to assess suicidal tendency29. The NGASR, designed by Cutcliffe23 has been translated into several languages, including German, Mandarin Chinese, Portuguese and Korean. It has been tested in different clinical settings in numerous countries, from psychiatric settings, to Emergency Rooms, to Older Adult care facilities23,30-33, demonstrating good validity and reliability.
Chen et al.31 tested reliability and validity of the NGASR scale translated into Chinese in a sample of 86 hospitalized patients diagnosed with schizophrenia. All participants completed a self-administered Beck Hopelessness Scale34 and were then evaluated through the NGASR. The authors concluded that the NGASR had good reliability and validity in inpatients with schizophrenia. Shin et al.32 translated the scale into Korean and enrolled a sample of 106 psychiatric inpatients in open and closed facilities, finding good construct validity and inter-rater agreement. Similarly to Chen et al.31, Shin et al. concluded that the NGASR is reliable and valid in assessing suicide risk of psychiatric inpatients. For the Dutch version, van Veen et al.33 enrolled a sample of 252 psychiatric inpatients and found good internal consistency and agreement between the total NGASR scores and the Suicide Intent Scale. NGASR total score had a significant and moderately strong association with judgement by a physician on ‘suicidal thoughts’ or ‘suicidal thoughts or plans’.
More recently, Kozel et al.30 presented the German version of NGASR. After translating the original instrument into German and pretesting the German version, they tested inter-rater reliability. Twelve video case studies were evaluated by 13 raters with the NGASR scale in a ’laboratory’ trial. The authors concluded that the German version of the NGASR scale is a reliable instrument for evaluating risk factors for suicide. Similarly, Facanha at al.35 translated, adapted and validated the Nurses Global Assessment of Suicide Risk for the Portuguese population applying the scale to a sample of 109 patients with depressive symptoms. The study showed moderate internal consistency; content validity, assessed by a panel of experts, was consensual. The NGASR index showed strong agreement with the Suicidal Ideation (SIQ), Questionnaire, Beck Depression Inventory (BDI) and Depression Anxiety and Stress Scales (DASS-21). The authors concluded that results were similar to those obtained in previous studies conducted in other countries and support use of NGASR to evaluate the risk of suicidal behaviours.
The NGASR scale has consistently been found to be user-friendly35. It is currently undergoing translation (and testing) into an electronic health record in Ontario psychiatric facilities, and has been included in the best-practice recommendation of the Registered Nurses Association of Ontario22 and in the ‘Clinical practice guidelines for suicide risk assessment’ of the Emergency Nursing Resources Development Committee36, as an adequate tool for assessing suicidal risk. Considering that there is currently no Italian version of this scale, the aim of the study was to evaluate the psychometric properties of the NGASR-ita.
METHODS
Translation into Italian
Upon authorisation from the original author, the NGASR was translated to test its psychometric properties in the Italian population. An Italian version was produced by a specialist in psychiatric nursing (P.F.) and a bilingual clinical nurse (S.T.), independently from each other. The two versions were compared, and a final version was back-translated into English by a bilingual English teacher. The author of the original NGASR scale confirmed the accuracy of the retranslated version. The Italian version was evaluated by seven nurses with at least 5 years of experience in mental health care. Based on their assessment, the authors calculated the content validity index. The index has two components, called CVI-I (regarding each of the items) and CVI-S (regarding the whole scale). The CVI-I is determined by the calculation of the evaluations by a group of experts evaluating the relevance of each item, on a scale of 0 (not relevant) to 10 (very significant). The CVI-S is calculated by averaging the percentage of items classified as ‘relevant’ (≥6) 37.
The Content Validity Index of the scale (CVI-S) was 96.7%, a value confirming the adequate validity of the instrument’s content in accordance with the literature that considers acceptable a CVI-S with a value ≥90%38. Each item on the scale in turn obtained a score (CVI-I) ≥.80; all 15 items were therefore confirmed in the Italian version of the tool (NGASR-ita). The seven experts also confirmed the absence of ambiguities or items to be modified or added and found the NGASR easy to be completed; this pleads in favour of the face validity of the instrument.
Participants
Reliability and validity of the NGASR-ita were assessed in a non-randomized sample of patients admitted to the psychiatric wards after accessing the emergency department of two Italian hospitals, ASST Ovest Milanese – Ospedale Legnano and ASST Santi Paolo e Carlo – Ospedale San Paolo, Milan – Lumbardy, from October to December 2017 (“ASST” meaning “social and healthcare facility on the territory”). All patients gave their informed consent. The study was conducted upon authorisation from the hospital management. The authors complied with the rules of the local ethical committee, the principles contained in the Declaration of Helsinki, and the Italian law on data protection. All data were collected anonymously.
Data analysis
Internal consistency was measured by calculation of Cronbach’s alpha coefficient. Intraclass correlation coefficient was used to evaluate inter-rater reliability (two nurses independently assessed each patient with the NGASR-ita). Pearson’s r coefficient was used to evaluate convergent validity by comparing the NGASR-ita with the Scale for Suicide Ideation (SSI)39 as well as test-retest stability (assessed by comparing data at admission and after 24 hours) after Blom’s data transformation. We chose to transform data because parametric tests have higher statistical power compared to non-parametric. Normality was assessed with three different tests, as per default option of the SAS® software (Kolmogorov-Smirnov, Anderson-Darling, and Kramer-von Mises). Exploratory factor analysis was performed after Kaiser-Meyer-Olkin’s measure of sample adequacy and Bartlett’s test of sphericity. Kaiser’s criterion was used to retain the eigenvalues of the correlation matrix; the factors were extracted with the principal component method. The factor pattern was rotated with the Varimax algorithm, and the factor loadings were retained according to Stevens’ criterion. The statistical significance treshold was set at 5%.
All calculations were conducted with the SAS® University Edition software and reviewed by a statistician.
The tool
The questionnaire was composed of two parts: the first part included The Nurses’ Global Assessment Suicide Risk (NGASR) created by Cutcliffe23 and translated into Italian; the scale is composed by 15 items that investigate the risk factors of suicide. The items are rated with Yes/No, depending on the presence or absence of the described criterion. The items have different scores and the overall score varies from 0 to 25 (Table 1). The levels of risk defined by original authors are: 0 to 5= low level of risk estimated; 6 to 8= intermediate level of risk; 9 to 11= high level of risk; 12 or more= very high level of risk. The second part includes open questions on socio-demographic variables (age, gender, diagnosis, life setting, type and duration of hospitalisation and the reporting of any self-inflicted behaviour that occurred during the stay.




RESULTS
121 subjects were enrolled, 60 males and 61 females; the mean age was 35±5 years. 71 patients (58.68%) were cohabiting or living in a family, 43 (35.54%) lived alone and 7 (5.79%) came from a community home. 105 (86.78%) patients were admitted to voluntary health care, 16 (13.22%) to forced hospitalization. The mean duration of hospitalization was 9±1 days. 45 patients were diagnosed with a psychotic disorder, 23 had Borderline Personality Disorder (BPD), 22 Bipolar Disorder (BD), 19 Major Depressive Disorder (MDD), 5 Anxiety Disorders, 4 Eating Disorders; 3 had no diagnosis at admission.
Interrater reliability
Interrater reliability was calculated as the degree of correlation between the scores assigned to 15 patients by two nurses who simultaneously, and independently, applied the scale. The correlation was very strong - Intraclass correlation (ICC) =.97. This result indicates that the scale provides results with the same clinical significance for the same patient condition, regardless of who employed it.
Test-retest stability
No statistically significant differences were found between the patient scores recorded in two consecutive moments (p=0.96), at the time of hospitalization and after two hours. This evaluation is used to understand whether the scale, evaluating the same patient twice without any factors that could change the score, yields the same result. Figure 1 confirms that the scores are substantially unchanged in both moments, which supports repeatability of the measurement.
Convergent validity
Suicide risk was assessed in 15 subjects at the time of admission, by evaluating concurrent validity between the NGASR-ita and the SSI 39,40, a validated tool used for clinical assessment of suicide risk. As shown in Figure 2, the scores of the two scales are strongly correlated (r=.98) so that high risk scores in NGASR correspond to high risk scores in the SSI. The mathematical model showed a very good adaptation to the data (R2=.98). This data indicates that the results of the analysis are reliable because they reflect the clinical reality of the patients. The relationship between the scale scores is statistically significant (p<.001).
Internal consistency
The value of Cronbach’s alpha coefficient was .66, indicating moderate correlation between the questions that make up the NGASR-ita scale; this supports reproducibility of the measurement.
Construct Analysis
Exploratory factorial analysis was conducted with the aim of verifying the “weight” of the items on the final score, and then to assess how much variability in the characteristics described by each item contributed to the overall variability of the sample’s suicidal intentions. The factorial loads, evaluated after Varimax rotation, were all higher than Stevens’ cut-off47, indicating that all items on the NGASR-ita scale contribute significantly to delineating the overall picture of suicidal intent. As can be seen in Table 2, items load on 5 factors; factor loadings ranged from .48 to .78. The 15 items accounted for 61.29% of the variance.
Assessment of suicide risk according to the NGASR-ita
All 121 enrolled subjects were assessed by all nursing staff members using the NGASR-ita scale at the time of admission to wards, or within the first 2 hours. Table 3 shows the sample’s suicidal risk levels at time of admission to ward.
According to the criteria defined by the original authors, 31 out of 121 subjects (25.62%) had an intermediate or higher level of risk at admission. The differences in the scores between the categories of diagnoses was statistically significant (p=.02). In particular, the category with the highest ratio of subjects at intermediate or higher risk was that of MDD (9 subjects at risk out of 19, 47.37%). An intermediate or superior level of risk was also observed in 16 of the 43 individuals who were living alone (p=0.02).









DISCUSSION
The aim of this study was to produce and attempt to validate the Italian version of the NGASR; a suicide risk assessment instrument that has already been studied and previously validated in other countries. The process of cultural adaptation of the scale revealed no ambiguity, complexity of judgement or misunderstandings; moreover the Content Validity Index, based on the judgement from a panel of expert nurses, confirms the content validity of the scale. The Italian version corroborates the usefulness and appropriateness of the 15 items found in the original version 23. Exploratory factorial analysis confirmed that all items showed high factorial loads, consistently superior to Stevens’ cut-off; 5 factors were revealed; previous works have revealed different structures overall30,33,35.
In line with the relevant literature our results support the hypothesis of a multifactorial nature of suicidal tendencies. The association between total NGASR and SSI was strong and significant, confirming the convergent validity already established in previous studies. Furthermore, the excellent scores concerning interrater reliability (the same patient is evaluated equally by two nurses) – and the test-retest stability – confirm the reliability of the NGASR-ita. The only criterion of reliability that failed to reach high values was internal consistency; the score obtained (alpha=.68) could be defined only moderately satisfactory, although it is higher than scores obtained in validation studies in other contexts.
This work showed a percentage of subjects at intermediate or superior risk of suicide as high as 25.62%. Somewhat unsurprisingly, a significant association emerged with the diagnosis of MDD. This is in line with the literature that frames this diagnostic category as the one most exposed to suicidal risk. Likewise, overall scores confirmed a higher prevalence of suicide-related thoughts in people living alone35,41.
Limitations
We did not study sensitivity and specificity of the scale, so we have no information about the predictive validity of the Italian version of the NGASR. In terms of sample size, although larger samples should be considered highly desirable, only one previous study on the topic was conducted on a larger sample33. The study of larger and more numerous samples will, however, allow us to strengthen the results and to further investigate subsamples of patients with psychiatric diagnoses that have been under- or not represented, such as adolescents or children30.
CONCLUSION AND RELEVANCE FOR CLINICAL PRACTICE
So far, scientific evidence shows that suicide cannot always be predicted with satisfactory confidence. Therefore, it is not always possible to implement effective preventive measures, as no conclusive evidence regarding suicide prevention is currently available17.
However, this study has shown that the Italian version of the NGASR is valid and reliable and therefore able to support nurses in the complex assessment of suicidal intentionality. This may be, as highlighted by Hunt et al.9, especially important in the first days of hospitalization in acute psychiatric setting. Further studies are needed to investigate the sensitivity and specificity of the NGASR-ita. Finally, the validity of the NGASR should be tested in other Italian clinical settings, such as the emergency department and in home care.

Conflict of interests: the authors have no conflict of interests to declare.
references
 1. World Health Organization (2012). Public health action for the prevention of suicide. Available at: https://bit.ly/2UdYkUe (last accessed 23/01/2019).
 2. World Health Organization. Suicide. 2018. Available at: https://bit.ly/1pV1R8w (last accessed 23/01/2019).
 3. Sadovk B. Compêndio de psiquiatria: ciências docomportamento e psiquiatria clínica. Porto Alegre: Artmed Editora, 2007.
 4. World Health Organization. Preventing suicide: a global imperative. 2014. Available at: https://bit.ly/1u4h1Zb (last accessed 23/01/2019).
 5. World Health Organization. Mental Health Action Plan 2013-2020. Available at: https://bit.ly/1d0stzs (last accessed 23/01/2019).
 6. Cardone R, Amore, M, Pompili, M, et al. Suicide in the national protocol for monitoring sentinel events. Ann Ist Super Sanita 2009; 45: 205-12.
 7. Ghirardini A, Murolo G, Palumbo F. The italian strategy for patient safety. Clin Chim Acta 2009; 404: 12-5.
 8. Troister T, Links PS, Cutcliffe J. Review of predictors of suicide within 1 year of discharge from a psychiatric hospital. Curr Psychiatry Rep 2008; 10: 60-5.
 9. Hunt IM, Bickley H, Windfuhr K, Shaw J, Appleby L, Kapur N. Suicide in recently admitted psychiatric in-patients: a case-control study. J Affect Disord 2013; 144: 123-8.
10. Ajdacic-Gross V, Lauber C, Baumgartner M, Malti T, Rössler W. In-patient suicide: a 13-year assessment. Acta Psychiatr Scand 2009; 120: 71-5.
11. Lönnqvist JK. Psychiatric aspects of suicidal behaviour: Depression. In: Keith Hawton K, van Heeringen K (eds).The International Handbook of Suicide and Attempted Suicide. Hoboken, NJ: John Wiley & Sons, 2008.
12. Lyons C, Price P, Embling S, Smith C. Suicide risk assessment: a review of procedures. Accid Emerg Nurs 2000; 8: 178-86.
13. Mitchell AM, Garand L, Dean D, Panzak G, Taylor M. Suicide assessment in Hospital Emergency Departments: implications for patient satisfaction and compliance. Top Emerg Med 2005; 27: 302-12.
14. Tarolla E, Caredda M, Tarsitani L, Maraone A, Biondi M. Fattori predittivi di nuovi tentativi di suicidio in soggetti che accedono in pronto soccorso per un tentativo di suicidio. Uno studio longitudinale a un anno. Riv Psichiatr 2015; 50: 28-33. 

15. Chan MK, Bhatti H, Meader N, et al. Predicting suicide following self-harm: systematic review of risk factors and risk scales. Br J Psychiatry 2016; 209: 277-83.
16. Bolton JM, Gunnell D, Turecki G. Suicide risk assessment and intervention in people with mental illness. BMJ 2015; 351: h4978.
17. Biondi M, Iannitelli A, Ferracuti S. On the unpredictability of suicide. Riv Psichiatr 2016; 51: 167-71.
18. Johnston ME, Nelson C, Shrivastava A. Dimensions of suicidality: analyzing the domains of the SIS-MAP Suicide Risk Assessment Instrument and the development of a brief screener. Arch Suicide Res 2013; 17: 212-22.
19. Barker PJ. Assessment in psychiatric and mental health nursing: in search of the whole person. London: Stanley Thornes, 1997.
20. Ryan T. Managing crisis and risk in mental health nursing. Cheltenham, GB: Nelson Thorners, 1999.
21. Gramaglia C, Feggi A, Bergamasco P, et al. Clinical characteristics associated with suicide attempts in clinical settings: a comparison of suicidal and non-suicidal depressed inpatients. Front Psychiatry 2016; 7: 109.
22. RNAO. Assessment and care of adults at risk for suicidal ideation and behaviour. 2009. Available at: https://bit.ly/2RbBc6K (last accessed 12/01/2018).
23. Cutcliffe JR, Barker P. The Nurses’ Global Assessment of Suicide Risk (NGASR): developing a tool for clinical practice. J Psychiatr Ment Health Nurs 2004; 11: 393-400.
24. Simon RI. Preventing patient suicide clinical assessment and management. Washington, DC: American Psychiatric Publishing, 2011.
25. Santos JC, Cutcliffe J. Suicide and Self-harm: an evidence-informed approach. London: Quay Books, 2012.
26. Cochrane-Brink KA, Lofchy JS, Sakinofsky I. Clinical rating scales in suicide risk assessment. Gen Hosp Psychiatry 2000; 22: 445-51.
27. McPherson A. An overview of the assessment tools available to mental health professionals to help determine patients at risk of suicide. Int J Psychiatr Nurs Res 2005; 10: 1129-42.
28. Range ML. The family of instruments that assess suicide risk. J Psychopathol Behav Assess 2005; 27: 133-40.
29. Abderhalden C, Grieser M, Kozel B, Seifritz E, Rieder P. Wie kan der pflegerische Beitrag zur Einschätzung der Suizidalität systematisiert werden? Psych Pflege Heute 2005; 11: 160-4.
30. Kozel B, Grieser M, Abderhalden C, Cutcliffe JR. Inter-rater reliability of the German version of the Nurses’ Global Assessment of Suicide Risk scale. Int J Ment Health Nurs 2016; 25: 409-17.
31. Chen YX, Ye MJ, Ji XQ, et al. Validity and reliability of Nurses’ Global Assessment of Suicide Risk (NGASR) for schizophrenia inpatients. Medical Journal of Chinese Civil Administration 2011; 3: 271-3.
32. Shin HY, Shin YS, Ju JH, et al. A study on reliability and validity of the Nurses’ Global Assessment of Suicide Risk (NGASR) for psychiatric inpatients. J Korean Acad Psychiatr Ment Health Nurs 2012; 21: 21-9.
33. Van Veen M, van Weeghel I, Koekkoek B, Braam AW. Structured assessment of suicide risk in a psychiatric emergency service: psychometric evaluation of the Nurses’ Global Assessment of Suicide Risk scale (NGASR). Int J Soc Psychiatry 2015; 61: 287-96.
34. Beck A, Weissman A. The measurement of pessimis: the Hopelessness scale. J Consult Clin Psychol 1974; 42: 861-86.
35. Façanha J, Santos JC, Cutcliffe J. Assessment of suicide risk: validation of the Nurses’ Global Assessment of Suicide Risk Index for the Portuguese Population. Arch Psychiatr Nurs 2016; 30: 470-5.
36. Brim C, Lindauer C, Halpern J, et al. Emergency nursing resources development committee: clinical practice guideline suicide risk, 2012.
37. Polit DE, Beck CT. Essentials of nursing research. 6Th Edition. Philadelphia, PA:  Lippincott Williams & Wilkins, 2006.
38. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007; 30: 459-67.
39. Beck A, Kovacs M. Assessment of suicidal intention: a scale for suicide ideation. J Consult Clin Psychol 1979; 47: 343-52.
40. Pettit JW, Garza MJ, Grover KE, et al. Factor structure and psychometric properties of the Modified Scale for Suicidal Ideation among suicidal youth. Depress Anxiety 2009; 26: 769-74.
41. Viana GN, de Moura F, Mamôru T, Tercius Escobar. Prevalence of suicide in the South of Brazil, 2001-2005. J Bras Psiquiatr 2008; 57: 38-43.