The sample consisted of specialists from different disciplines who work in forensic psychiatric institutions (nursing, psychiatry, psychology, social pedagogy, nursing assistance, social work and occupational therapy) and who participate voluntarily. An experience of less than six months in the field of mental health was defined as an exclusion criterion. Non-probability convenience sampling was used.
A total of 153 professionals took part, 61.4% women and 38.6% men. The mean age was 39.92 ± 9.83 years, while the mean number of years worked in forensic units was 7.10 ± 4.94 years. The sample consisted of a variety of professions with 34.6% nursing assistants, 22.9% occupational therapists, 21.6% nurses, 9.2% psychologists, 7.8% psychiatrists and 3.9% social workers. Around 85% of those in employment stated that they described their working environment as good or very good and 15% as bad or very bad. A total of 41.2% stated that they often or always had the opportunity to contribute to improving the quality of medical care, 51.0% sometimes and 7.8% rarely or never.
Professional perception of quality care was assessed using QPC-FIPS Spanish. This version is an adaptation of the original QPC-FIPS questionnaire24.
The Quality of Psychiatric Forensic Inpatient Staff (QPC-FIPS)24, described in the introductory part, is a self-administered questionnaire consisting of 34 items and measuring 7 dimensions of quality of care: Encounter (8 items), Participation (8 items), Discharge (3 items), Support (4 items), Secluded environment (2nd articles), safe environment (3 articles) and forensically specific (6 articles). Each item relates to the statement “I think that…” and the responses are based on 4-point Likert scales, with 1 corresponding to “strongly disagree” and 4 “strongly agree”. All items also have a “does not match” option.
To analyze the convergent validity, the general satisfaction scale NTP 394: Overall Job Satisfaction was created by Warr et al.10 was used. This scale, which assesses working conditions (intrinsic and extrinsic), consists of 15 items that are rated on a scale from 1 to 7 (1 “very dissatisfied” and 7 “very satisfied”).11.
The translation and back-translation process was carried out according to the standards for educational and psychological tests26.
First, the original version was translated into Spanish by two independent native-speaking translators who had no knowledge of the instrument or the objectives of the study. A panel of experts consisting of a psychometrics doctor, three quality experts, five mental health nurses, a psychologist and a psychiatrist specializing in forensic medicine previously assessed semantic equivalence (grammatical difficulties in translation, equivalent meaning of words) and idiomatics (Contextualization) valued the text, colloquialism) and conceptual equivalence. The translation and back-translation process did not present any major difficulties. After the back-translation process, the expert committee proposed to change some terms in points 30 and 33 to adapt them to the Spanish context and to respect the semantic meaning of the original version. Point 30, after re-translating the Swedish version, states: “The staff will help patients, if they wish, to present their wishes and their case to the administrative court.”24, it was considered appropriate to change it in the Spanish version to “Users can access the judge through their lawyer or through the Legal Advice Service” by changing the term “patients” to “user”, as this term is widely used in Spain is common. Point 33 in the Swedish version read: “Patients get help from staff to work out their crime” for “During the prison stay, the professionals help the user to talk about the crime he has committed”.24, therefore the term “patients” was replaced by “users” and “staff” by “professionals”. Also, “talk about the crime” was chosen instead of “working out the crime”.24. Finally, inclusive, non-discriminatory language was used when referring to gender.
A pilot cognitive test was conducted on 30 professionals to assess understanding and processing time. The test included an open-ended question in which participants were asked to indicate whether an item might cause comprehension difficulties. The average time to complete the questionnaire was between 15 and 20 minutes and there were no items that were difficult to understand. After the debriefing, it was not deemed necessary to make any changes to either content format. The Spanish-adapted version of the QPC-FIPS scale was configured with the same number of items and dimensions as the original, with the final version named QPC_FIPS-Spain in Spanish.
The fitting and psychometric assessment process was carried out in four forensic mental health departments at Parc Sanitari Sant Joan de Déu, Barcelona, Spain. The data collection was carried out between February 2019 and December 2021.
To analyze stability over time, it was estimated that at least 60 experts were needed to determine an intraclass correlation coefficient (ICC) of approximately 0.70 between two administrations of the instrument, assuming a 95% confidence level and an 80% power level in a were bilateral comparison22.
The program SPSS Statistics Version 26 was used for the analyzes and the program EQS Version 6.2 for the confirmatory factor analysis (CFA).27.
Construct validity was analyzed by confirmatory factor analysis (CFA) with estimated parameters using the least squares (LS) method, which is similar to the maximum likelihood (ML) method but estimates patterns of relationships between variables by taking the sum of the squares of the Deviation between the hypothetical and observed model is minimized. LS performs better than ML for small samples and provides a better estimate when there is a violation of the assumptions of normality28. The criteria for a good fit were Bentler Bonnet Normed Fit Index (BBNFI), Bentler Bonnet Non-Normed Fit Index (BBNNFI), Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Comparative Fit Index (CFI) > 0.9029,30,31; the ratio of chi-square to degrees of freedom (χ2/df) 32. The value of the root mean square error of approximation (RMSEA) and the standardized root mean square (SRMR) and the root mean square residual (RMR) was ≤ 0.0833.34.
The General Satisfaction Scale NTP 394: Overall Job Satisfaction was used for convergent validity11; Spearman’s correlation coefficient analysis was performed. Furthermore, as an additional method, a Spearman correlation analysis between the factors of the QPC-FIPS instrument was performed with the aim of verifying Fayers & Machin35 Hypothesis suggesting that the correlation between each factor and the general scale was higher than the correlations between the subscales.
To assess the internal consistency of the instrument at the general level and for each of the factors, Cronbach’s alpha and ordinal alpha with a value greater than or equal to 0.70 were used36 considered sufficient reliability. Stability over time, or test-retest reliability, was assessed after 7-14 days by the ICC in a sample of 77 professionals. A value greater than or equal to 0.70 was taken as an indicator of good agreement36. In addition, the composite reliability was calculated. Item analyzes included calculation of item means, standard deviations, and adjusted overall item correlation.
Ethical aspects of research
This study was approved by the Sant Joan de Déu Foundation Research Ethics Committee under CEIC code PIC-73-18. All investigations were carried out in accordance with the relevant guidelines and regulations. All participants were informed about the aim of the study and gave their written consent to participate voluntarily and anonymously.