State University of Pará - UEPA, Marabá, Pará, Brazil.
brennacecilia.ferreira@gmail.com
State University of Pará - UEPA, Marabá, Pará, Brazil.
State University of Pará - UEPA, Marabá, Pará, Brazil.
leonardobrenopessoa@hotmail.com
Federal Technological University of Paraná - UTFPR, Ponta Grossa, Paraná, Brazil.
The evaluation of quality directs efforts to critical areas, especially when talking about public health management, which needs to be efficient to serve the population. In this sense, tools directed to quality evaluation in this sector are pertinent. The objective of this work was to propose a quality evaluation model directed to the Unified Health System, developed based on the Service Quality instrument, and considering aspects concerning the Institute of Medicine, National Health Services Evaluation Program, and the corresponding legislation. Two collection instruments were developed and statistically validated by means of the alpha coefficient for the composition of the referred model. By considering the relative importance of dimensions from the perspective of the three audiences (users, professionals, and managers), the application of the SUSSQUAL model proved to be useful for guiding managers and/or professionals interested in decision making processes, and in outlining strategies that have the qualitative elevation of public health sector attributions as their purposes.
Keywords: Public Health; Unified Health System; Quality Assessment; Service Quality; SUSSQUAL.
With the current scenario of economic crisis in which Brazil finds itself, it is worth emphasizing the increased relevance of the work developed in the search for optimization of resources. In all organizations there is a need for indicators both for external evaluation and for the management process. In addition, it is necessary to draw up performance indicators that respond to the strategic objectives, printing the implementation of the organizational strategy (Vecina Neto; Malik, 2012).
Hospital organizations, in particular, have a certain procedural plurality that configures them as complex organizations and need to be seen as such. As emphasized by Vecina Neto and Malik (2012), this complexity comes from the coexistence of numerous assistance and administrative processes, simultaneous production lines, and decentralization of the decision process. Thus, it is known that the study on qualitative aspects related to the provision of services in the public health sector is relevant, since it is a complex sector. Mezomo (2001) states that this complexity is caused by a series of internal and environmental factors, which end up compromising service quality.
It is worth remembering that the legislation of the Unified Health System (Sistema Único de Saúde - SUS) regulates health as a right of all citizens and duty of the State. However, Mendes (2002) emphasizes that the system that should be unique has become plural, and today is composed of three groups: the public system, the supplementary medical care system, (referring to private plans) and the direct disbursement system. It can be inferred that this segmentation of the system occurred as a result of the inefficiency of public resources to meet the high aggregate demand of the system, which, in turn, needed to gradually migrate to other spheres of health service provision to meet its needs.
A survey conducted by the Trata Brasil Institute and the Brazilian Institute of Public Opinion and Statistics (2012) stipulated that nearly 61% of Brazilians consider health as the most problematic area in the country. This is a referendum on the national public health system, since three out of four Brazilians depend on it (Prates, 2012). According to the Federal Pharmacy Council (2015), Brazil is one of the countries with the lowest investment in public health. The data shows that the Brazilian government invests 4.7% of the Brazilian domestic product (GDP) in health, much lower than expenditures in Canada, France, Switzerland and the United Kingdom, where the percentages of investment range from 7.6% to 9.0%.
In addition to underfunding, a growing number of careful users are identified with the quality of health services. In this sense, tools directed to the evaluation of quality in this sector are pertinent. It is worth mentioning that the topic of quality of services has shown strong growth in recent decades, given its importance to the economy. There are several quality evaluation models that can be used in the services environment, but which require customization (Batalha, 2008).
Service Quality (SERVQUAL), developed by Parasuraman et al. (1988), is conceptualized by Ferreira (2015) as an instrument designed to measure the quality of service provided from customer perceptions. According to Sousa et al. (2011), SERVQUAL emerged to try to solve the following situations: revision of the studies that investigated the quality of services in the purchase of goods and services; review of the reports of the insights obtained in extensive exploratory research of quality in four service companies; development of a model of quality in services; and proposal of conditions for the development of future studies on quality.
The relevance of this study is justified by its contribution in the scientific field, since it suggests a customized quality measurement model to be applied in the sector of services offered by SUS, in order to provide advances in the scientific area and act as a method for the development of future and effective scientific research in the sector for which it is intended, which is little explored by the literature.
This study aimed to propose a quality assessment model for the services offered by SUS, based on the SERVQUAL model. To do so, it is necessary to list the uses of the SERVQUAL model in the health sector and the variations of this tool, besides developing an adapted model with aspects from the Institute of Medicine (IOM), National Program for Health Services Evaluation (Programa Nacional de Avaliação dos Serviços de Saúde - PNASS), pertinent legislations, and validate the model mathematically. IOM was chosen because it is an independent organization that deals with issues relevant to health and medicine, stimulating positive actions to the sector. The organization conducts studies and issues reports that provide impartial and qualified information to policy makers and the population (Allen Jr., 2016). PNASS is a tool to evaluate health establishments and specialized care, seeking to raise relevant indicators to be worked for improvement proposals (Brazil, 2015).
As for the nature of the results, this study is characterized as applied research and assumes the typology of Theoretical Conceptual Model, since it aims to promote a discussion of the literature and, starting from the conclusions drawn in conjunction with the analysis of the data collected, to develop conceptual models that result in a new theory, corroborating for the advancement of science. For the development of the study, a sequence of activities was respected, as described in Figure 1.
Figure 1. Research Stages
The model to be developed is a collection tool, aimed at three target audiences: users, professionals, and managers of municipal public health units. To verify if the proposed model met its objectives, Cronbach's alpha coefficient was calculated and, according to Ganga (2012), it should be submitted to three types of people: colleagues, experts from the studied organization, and target respondents. The calculation of the alpha coefficient is performed according to Equation (1):
The proposed model will follow the survey method, working as an instrument for collecting ordinal categorical variables, which can be defined as qualitative variables that present sorting among the collected categories by using the Likert type scale (Ganga, 2012). However, in this study, the tools used for data collection will be comprised of closed questions, with Likert type scales of importance.
For the construction of the model, a literary review was initially performed aimed to identify variants of the SERVQUAL model and the criteria considered by the IOM, PNASS and SUS legislation. This was necessary in order to identify relevant qualitative aspects to be considered in the quality evaluation in this service sector.
After this investigation, the qualitative attributes to be considered in the model for quality evaluation in the public health sector were defined. This allowed the construction of two questionnaires to apply the proposed model: a) users; b) assistance professionals and managers.
The following topic presents the results achieved with the development of the study until the presentation of the proposed model.
In this section we will discuss uses of the SERVQUAL model for the health sector, present in the current scientific literature. For this, it was necessary to refine the publications to be addressed in this study, given the large number of such productions available for access.
For the purposes of this project, the theses and dissertations considered pertinent were those published in the period from 2013 to 2016, originating from Production Engineering courses offered by Brazilian educational institutions, and which presented the application of the SERVQUAL tool to health service providers. The search was conducted in the thesis and dissertation bank of the Commission for the Improvement of Higher Education Personnel (CAPES). Chart 1 presents a summary of the selected publications.
Chart 1. Publications on SERVQUAL application in health services
After an investigation of SERVQUAL models intended for health evaluation, it was necessary to investigate the quality measurement models that emerged through variation of the SERVQUAL model. A summary of the most usual models in the literature is presented in Table 2.
Chart 2. Summary of SERVQUAL model variations
HOSPQUAL arose from a research project to evaluate the quality of hospital service, with special reference to selected districts in Tamil Nadu, developed in India (Rajaram, 2011). To measure quality, the instrument follows the logic of analysis established by SERVQUAL, by collecting, through the application of questionnaires with Likert scales, the expectations, and perceptions of the consumer public, and establishing the quality indexes of the institutions. However, it inserts specific questions to hospital sectors to point out users, related to the analysis of the behavior of doctors, professionals of the assistance team, the administrative quality of hospitals, and the services provided by hospitals.
Thus, the author states that each question of SERVQUAL should be reformulated to present greater consistency with the area of the specific service to be measured, in addition to emphasizing that the model referred to does not offer a single value attributed to the quality of the service of the organization as a whole, but values related to the five dimensions addressed by it, which, according to it, would weaken the SERVQUAL model. Chart 3 presents the constitution of the HOSPQUAL tool.
Chart 3. Dimensions of HOSPQUAL model
Moreover, among the existing models aimed at quality evaluation in services, the HOSPQUAL developed by Rajaram (2011), although focused on the hospital sector, is just another application adapted from the SERVQUAL tool, in which the author customizes the issues that constitute the instruments of the model mentioned. Table 4 presents the dimensions of SERVQUAL, HOSPQUAL and those proposed by the model of this study - SUSSQUAL.
Chart 4. Comparison of SERVQUAL, HOSPQUAL and SUSSQUAL dimensions
The HOSPQUAL model is an adaptation of SERVQUAL, therefore there are no additions or modifications of qualitative dimensions. The SUSSQUAL model, in turn, presents some additions of information. This model was elaborated with the purpose of making possible the measurement of the qualitative levels of the dimensions of the quality of the services provided by SUS to the population, in a more coherent way with the nature of the service in question.
The SUSSQUAL method adds qualitative dimensions that were disregarded in the formulation of the SERVQUAL model, proper to the health area, revered by IOM, which is responsible for formulating qualitative strategies in international health services and the PNASS Program, which is the policy responsible for auditing public health services in Brazil, besides considering constitutional legislative aspects. For the model proposal, aspects of the IOM and PNASS were considered. Chart 5 presents the relationship between their qualitative dimensions.
Chart 5. Relationship between the qualitative dimensions of PNASS, IOM, SERVQUAL, and SUSSQUAL
Thus, based on the concepts of SERVQUAL, IOM, PNASS, and regulatory legislation, the dimensions of SUSSQUAL defined are tangibility, efficiency, efficacy, effectiveness, patient centrality, safety, empathy, access, and legislation.
Based on the references presented, the SUSSQUAL model appears with the purpose of acting as an auxiliary tool for evaluating the quality of the service provided by the establishments and/or health systems supported by SUS.
In order to proceed with the delimitation of the object to be evaluated, it is necessary to define which levels of reality one wants to study. The health field is made up by public and private institutions that develop actions of promotion, prevention, and cure oriented to the population. SUS presents its particularities, as it is a relatively new system in the country, besides being constitutional. This factor justifies the need to evaluate the service, considering the various aspects of the system and appointing various agents involved in the dynamics of service provision. Thus, SUSSQUAL considers new dimensions and a specific approach based on the Brazilian context.
The proposed model consists of two collection instruments. The first is aimed at users and professionals working in medical assistance, in the area of public health. And the second, optional and complementary, is directed to the managers of each public health institution.
Service users, population representatives, health professionals and service managers have different objectives and perceptions regarding health services, and generally prioritize different aspects when evaluating the quality of health actions. Thus, it is necessary to democratize evaluations that intend to influence decision making processes in services, considering that the involvement of different actors amplifies the power of evaluation to unveil different aspects of the same intervention and its effects (Serapione, 1999 apud Hartz; Silva, 2005, p. 59).
The SUSSQUAL model presents the constitution of each instrument divided into three sections: questions aimed at stratifying the respondent audience; items for qualitative verification of the various dimensions that constitute the process of providing services; and, finally, the section requesting the degree of importance of each qualitative dimension from the perspective of the respondent.
The proposed instrument, however, considers only the assimilation of the perception of the exploited quality aspects, not requesting the expectations of the public in relation to the service. This fact is justified considering that quality can be defined as the "ability of the inherent characteristics of a product, system, or process to meet the needs of customers and other interested parties" (ISO/DIS 9000:2000, 1999). By stipulating the evaluation of the user's perception on a scale from 1 to 5, it is considered that note 5 expresses that a certain aspect has been able to completely satisfy the respondent, by means of a natural comparison between what the respondent understands as a desirable level of quality in a certain aspect and the level of quality they have enjoyed. The level of perception of quality itself already points to the quality level, a factor that makes it unnecessary for the respondent to request expectations.
Thus, the tool proposed in this study is positioned as a propitious means to identify the levels of health services offered, and to continuously improve the attributions that constitute them, besides functioning as a performance control instrument for the organization that makes use of it.
The model consists of two questionnaires (users and managers) related to the public health system. The first one (Appendix 2), aimed at users, is composed of 21 questions subdivided into the nine proposed dimensions. For the managers (Appendix 3), there are 29 items for evaluating the nine dimensions proposed by the model. The model still has an initial survey of the respondent (Appendix 1), serving as the beginning for both questionnaires. Table 1 shows the number of questions proposed in each dimension of the questionnaires.
Table 1. Number of questions in the questionnaires
The items that constitute the tool are affirmations related to the services offered by the health unit under evaluation, for which respondents must indicate their degree of agreement on a 5-point Likert scale, in which 1 expresses total disagreement and 5 complete agreement to the affirmation. In addition, the tool requests the degree of importance of a certain qualitative dimension for the production of a service considered optimal in health. In this section, respondents should indicate how much they consider a certain aspect relevant to the provision of a service considered qualitative, on a 5-point scale of importance, where 1 expresses a criterion of no importance, 2 indicates a criterion of little importance, 3 informs a criterion indifferent to its relevance, 4 expresses a criterion of moderate importance, and 5 indicates a criterion of extreme importance.
The use of this instrument is important for the support and analysis of the results obtained in the application of the "Questionnaire users/professionals of assistance", since it aims to direct the evaluator's gaze on the possible causes of qualitative gaps pointed out by users and assistance professionals.
Statistical analysis was used to validate the construction by applying Cronbach's alpha in order to verify the reliability rate of the instruments developed to constitute the proposed model.
To this end, we estimate the minimum quantity of questionnaires answered by the three audiences of the model proposed in this study to be statistically analyzed. Thus, a value equal to or greater than that established is used for model analysis. Questionnaires from the application in four types of health facilities are used, and two of these are health centers (US-A and US-E), a specialized clinic (US-D), a general hospital (US-C), and a specialized hospital (US-B). In each establishment, 40 users, 20 professionals, and one manager were interviewed. Table 2 presents the results of the coefficient for each investigated unit and a general coefficient for the 200 users and 100 professionals interviewed.
Table 2. Cronbach Alfa per health unit.
According to the classification established by Freitas and Rodrigues (2005), the indicators were between moderate, high, and very high. For these indicators, the levels are consistent for the objective proposed by the questionnaire. For the managers, a general analysis of the five interviewees was made, obtaining a coefficient of 0.971, presenting very high consistency.
This work aimed at developing a model that would better measure the quality of service provided by Brazilian public health institutions. The union of concepts approached in IOM, PNASS and SERVQUAL served as basis for the formulation of the proposed model and, thus, through the application, it was possible to collect data relevant to the identification of qualitative criteria to be prioritized to improve the studied system.
The literature review was carried out in order to theoretically support the study. SERVQUAL models for the health area were investigated, as well as the existing variations of this tool to build the proposed model. The concepts of IOM, PNASS and SUS legislation were integrated and acted as guidelines in the elaboration of the collection instruments, so that it could approach the items and dimensions that were closer to the reality of users and health professionals.
The model can be applied to the reality of the SUS, since it is possible to raise the local reality and direct improvements in a more effective way. The model proved to be robust and valid after being validated by means of Cronbach's alpha coefficient. Therefore, the model is consistent with the proposed objectives. As a future study proposal, it is suggested an application of the model after validated for a pertinent analysis of the surveyed data.
Allen Jr., L.V. 2016. Introdução à farmácia de Remington. University of Sciences in Philadelphia. Porto Alegre: Artmed.
Batalha, M.O. (Org.).2008. Introdução à Engenharia de Produção. Rio de Janeiro: Elsevier.
Batista, D.A. 2013. O uso da Abordagem Fuzzy para a integração das ferramentas QFD e SERVQUAL em serviços de saúde [tese]. Recife: Universidade Federal de Pernambuco.
Brasil. Ministério da Saúde. Secretaria-Executiva. Departamento de Regulação, Avaliação e Controle de Sistemas. 2015. PNASS: Programa Nacional de Avaliação dos Serviços de Saúde. Brasília: Ministério da Saúde.
Caruana, A. and Pitt, L. 1997. INTQUAL—An Internal Measure of Service Quality and the Link between Service Quality and Business Performance. European Journal of Marketing 31, 604-616. https://doi.org/10.1108/03090569710176600
Conselho Federal de Farmácia - CFF. 2015. Brasil é um dos países que menos investem em saúde pública. https://www.cff.org.br/noticia.php?id=2937&titulo=Brasil+%C3%A9+um+dos+pa%C3%ADses+que+menos+investem+em+sa%C3%BAde+p%C3%BAblica
Cronin, J.J., & Taylor, S.A. 1992. Measuring service quality: A reexamination and extension. Journal of Marketing 56, 3:55–68. https://doi.org/10.2307/1252296
Delgado, C.F. et al. 1999. Hotelqual: Una escala para medir calidad percebida em servicios de alojamento. Estudios Turisticos, 139:93-108.
Ferreira, S.P.C. 2015. Modelo de satisfação de clientes do Hospital das Forças Armadas. Relatório Científico do Trabalho de Investigação Aplicada. Lisboa.
Firdaus, A. 2006. The development of HEdPERF: a new measuring instrument of service quality for the higher education sector. International Journal of Consumer Studies 30, 6:569–581.
Freitas, A.L.P.; Rodrigues, S.G. 2005. Avaliação da Confiabilidade de Questionários: uma Análise Utilizando o Coeficiente de Alpha de Cronbach. Anais do XII SIMPEP – Simpósio de Engenharia de Produção, Bauru, SP, Brasil, 7 a 9 de novembro de 2005.
Ganga, G.M.D. 2012. Trabalho de Conclusão de Curso (TCC) na engenharia de produção: um guia prático de conteúdo e forma. São Paulo: Atlas.
Guimarães Júnior, D.S. 2016. Modelo Hierarquico e Multidimensional para mensuração da qualidade percebida no setor de saúde suplementar [tese]. Recife: Universidade Federal de Pernambuco.
Hartz, Z. M. D. A., & Silva, L. M. V. D. 2005. Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz.
Institute of Medicine. 1990. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press.
Instituto Trata Brasil, Instituto Brasileiro de Opinião Pública e Estatística. 2012. A percepção da população quanto ao Saneamento Básico e a responsabilidade do Poder Público. São Paulo: Instituto Trata Brasil, Ibope.
ISO/DIS 9000:2000, Projeto de Norma – Quality Management Systems – fundamentals and vocabulary – ISSO, proposta datada de 25/11/1999.
Maia, M.C.S. 2013. Uma abordagem para avaliação da satisfação dos clientes em empresas de serviços de saúde: Aplicação da integração dos modelos SERVQUAL, KANO e QFD [dissertação]. Recife: Universidade Federal de Pernambuco.
Mendes, E.V. 2002. Sistema Nacional de Saúde no Brasil – SUS e Sistema Complementar. Ano III, nº1.
Mezomo, J.C. 2001. Gestão da Qualidade na Saúde: Princípios Básicos. São Paulo: Manole.
Parasuraman, A. et al. 1988. SERVQUAL: A Multiple-Item Scale for Measuring Consumer Perceptions of Service Quality. Journal of Retailing 4, 1.
Pereira, A.M.R. 2015. Avaliação da qualidade em serviços públicos de saúde em um hospital usando a análise hierárquica [dissertação]. Recife: Universidade Federal de Pernambuco.
Prates, M. 2012. 5 países onde a saúde pública funciona. Publicado 13 jun. 2012. http://exame.abril.com.br/mundo/5-paises-onde-a-saude-publica-funciona/
Queiroz, S.F.A.M. 2014. Estudo da integração do QFD com SERVQUAL em hospitais privados do Recife [dissertação]. Recife: Universidade Federal de Pernambuco.
Rajaram, S. 2011. Service Quality Assessment in Hospitals with special reference to Selected Districts in Tamil Nadu. Indian Institute of Public Administration.
Sousa, T.C.G. et al. 2011. Comparação de modelos de qualidade de serviços: proposição estratégica para instituições de ensino superior. Seminários em Administração, SemeAd, out.
Vecina Neto, G.; Malik, A.M. 2014. Gestão em Saúde. Rio de Janeiro: Guanabara Koogan.
Wegner, R.S. 2016. Aplicação do SERVQUAL e Analytic Hierarchy Process: Ferramentas para melhoria dos serviços prestados pelo Hospital Universitário de Santa Maria, RS [dissertação]. Rio Grande do Sul: Universidade Federal de Santa Maria.
Yildiz, S.M.; Kara, A. 2009. The PESPERF scale: An instrument for measuring service quality in the School of Physical Education and Sports Sciences (PESS). Quality Assurance in Education 17, 4:393-415.
Appendix 1. Characterization of users - Questionnaire "users/professionals of assistance”
Appendix 2. Questionnaire “users / assistance professionals”
Appendix 3. Questionnaire “managers”
Received: 11 Jan 2020
Approved: 17 Nov 2020
DOI: 10.20985/1980-5160.2020.v15n3.1605
How to cite: Amoury Neta, A.S.; Ferreira, B.C.S.; Bezerra, R.R.R.; Silva, L.B.P. (2020). Proposal of a quality evaluation model directed to the Unified Health System. Revista S&G 15, 3, 250-262. https://revistasg.emnuvens.com.br/sg/article/view/1605