Skip to main content

Measuring accuracy of plethysmography based respiratory rate measurement using pulse oximeter at a tertiary hospital in India

Abstract

Background

Childhood pneumonia continues to be a major infectious killer in India. WHO recommended respiratory rate and oxygen saturation (SpO2) measurements are not well implemented in Indian public health outpatient facilities with the result that treatment decision-making rely on subjective assessments from variably trained and supervised healthcare providers. The introduction of a multi-modal pulse oximeter (POx) that gives reliable measurements would mitigate incorrect diagnosis. In light of future potential use of pulse oximeter in peripheral health centres, it becomes important to measure accuracy of respiratory rate and oxygen saturation of such an instrument. The current study measures accuracy of plethysmography based respiratory rate (RR) using a pulse oximeter (Masimo Rad-G) by comparing it with a gold standard (pediatrician) measurement.

Study design

A cross sectional study was conducted in the OPD and emergency ward of Kalawati Saran Children’s Hospital over a 2 week period wherein a convenience sample of 97 children (2 to 59 months) were assessed by a pediatrician as part of routine assessment alongside independent measure by a consultant using pulse oximeter. The level of agreement between plethymography based RR and pediatrician measure was analyzed along with sensitivity and specificity of fast breathing of plethymography based RR measure.

Results

Both methods of measurement show strong association (97%, p < 0.001) and observed values, falling on line of unity, obtained either from pulse oximeter or by pediatrician are very close to each other. Fast breathing measured by POx has a sensitivity of 95% and specificity of nearly 94%.

Conclusion

The current study provides evidence of the accuracy of a plethysmography based RR using a pulse oximeter which can potentially be of use in planning of pneumonia management in public health facilities.

Background

Globally, the number of episodes of clinical pneumonia in young children decreased by 22% from 178 million in 2000 to 138 million in 2015. Yet in the same year, India, Nigeria, Indonesia, Pakistan, and China contributed to more than 54% of all global pneumonia cases, with 32% of the global burden from India alone [1]. Childhood Pneumonia continues to be the topmost infectious killer among under-five children, contributing to 15% of under five deaths (Approximately 1.4 lakhs children) annually in India [2].

While over the past two decades the implementation of World Health Organization (WHO) Integrated Management for Childhood Illness (IMCI) guidelines during primary healthcare (PHC) in low-middle income countries (LMIC) has made substantial contributions to child mortality reductions, considerable residual mortality remains [2, 3]. WHO IMCI guidelines recommend counting respiratory rate for 1 min and checking SpO2 in sick children with cough or difficult breathing [4]. Recording respiratory rate and oxygen saturation (SpO2) are central to delivering high quality integrated care for sick infants and children, yet they are not well implemented in public health facilities in India, especially in peripheral health centres for a myriad of reasons including overburdened healthcare providers and non-availability of pulse oximeter.

Hypoxemia defined by WHO as a SpO2 < 90%, is associated with increased mortality in children [5]. As hypoxemia is cited as a key mortality risk factor among children with pneumonia, effectively identifying hypoxemic children is fundamental to reducing pediatric mortality globally, yet pulse oximetry (POx) is not available in primary health care (PHC) facilities. Modeling estimates of disease progression of pneumonia in the top 15 countries with the highest disease burden found that POx has the potential to avert 148, 000 deaths among children with pneumonia, and that, when combined with IMCI guidelines, the prognostic tool is highly cost-effective [6].

Many new ‘multi-modal devices’ that measure SpO2 as well as at least one other vital sign such as respiratory rate and/or temperature are in the developmental pipeline. Since few infants and children have these core vital signs collected, the end result of IMCI application at PHC is that referral and treatment decision-making rely on subjective assessments from variably trained and supervised healthcare providers. For example, in Malawi, it was found that of nearly 700 patient encounters with possible malaria or pneumonia only 16 and 24% had a respiratory rate (RR) or SpO2 measured. Most concerning was the observation that more than 40% of children eligible for hospitalization were not correctly referred [7]. A recent study in Malawi confirmed the potential utility and feasibility of POx at PHC. Malawian outpatient providers successfully measured SpO2 and found providers were two-fold more likely to correctly refer a child to the hospital when the SpO2 was low. Notably, > 60% of hypoxemic children would not have been referred to the hospital if providers had applied the 2014 WHO IMCI guidelines in the absence of a SpO2 measurement [8]. This premise can be extended to PHC where respiratory rate and SpO2 measurements are likely key barriers to optimal management of pneumonia. In fact, preliminary results of an assessment of pneumonia management practices among health workers conducted by the USAID supported Vriddhi project suggest a low awareness of the use of POx in the diagnosis of pneumonia except in the state of Haryana (internal report) [9]. In its absence, physical measurement of breathing rate and recognition of danger signs become essential in the diagnosis of ARI and/or pneumonia. Under the Ayushman Bharat scheme, sub centres and primary health centres (PHC) are being strengthened as Health and Wellness centers (HWC) to provide preventive and curative care for an expanded range of services including reproductive and child health services [10]. The services are to be provided by mid-level health care provider including Auxiliary Nurse Midwife (ANM), Ayush doctors, Community Health Officer (CHO) placed at a HWC and Medical Officer at PHC (Rural/Urban). However, we found that out of 45 providers in such centres, six reported not knowing how to measure RR and nine reported not knowing how to check labored breathing [9]. The introduction of POx would mitigate such issues, however, we would need a device which not only replaces clinical assessment but does so with a high level of precision.

It is within this overall context of improving the effectiveness of measuring the core vital signs of respiratory rate and SpO2 that we seek to evaluate a multi-modal device that records both. This is likely to have the added value of serving as a diagnostic for pneumonia and can help distinguish children with ARI into pneumonia who require referral or drugs or only home management with counselling. The reliability of pulse oximeter vis a vis other devices such as capnography has been studied, in which a high overall agreement was found with clinician reviewed capnography [11]. In addition, available evidence points to the accuracy of Masimo pulse oximeter in comparison with other pulse oximteres [12]. Pulse oximeters have been introduced in many newborn and pediatric care units in India, in tertiary health centres. With future plans to introduce it in the primary health care settings for diagnostic purpose in pneumonia, there is a need to investigate its reliability and add to the current knowledge. The current study compares clinician’s gold standard respiratory rate with device recorded respiratory rate based on plethysmography (Masimo Rad-G).

Methods

This study compared respiratory rate measured through a pulse oxmeter (Masimo Rad-G) with the gold standard which we consider here as a pediatrician’s clinical measurement of respiratory rate. The study was conducted in the pediatric OPD and emergency unit of Kalawati Saran Hospital. The device used is the Masimo Rad-G which is a multimodal pulse oximter designed for use in pneumonia screening and spot checking of oxygen saturation in low resource settings. It uses measure-through Motion and Low Perfusion™ SET® pulse oximetry technology to measure SpO2, respiration rate from the Pleth (RRp™), pulse rate (PR), and perfusion index (Pi).

Sample size

To determine the sample size, n, we used the relative error r and the difference between the overall agreement probability Pa and the chance-agreement probability Pe as follows

$$ \mathrm{n}=\left(\mathrm{n}\ast \right)/\left(1+\mathrm{n}\ast /\mathrm{N}\ \right)\kern2.5em \mathrm{n}\ast =1/\left(\mathrm{r}2\ \left(\mathrm{pa}-\mathrm{pe}\right)2\right) $$

The pulse oximeter, overall, provides an accurate measurement of respiratory rate and oxygen saturation compared with other devices. Therefore, we consider a low relative error between measurements (0.20), and an assumed level of agreement: 0.50 (as this is an unknown value). With the total population of children admitted to the pediatric ward at any given time being an average of 50 (N), we get a sample size of 50. We oversampled to include a total of 97 children as we aimed toward precision of the measures.

Data collection

A convenience sample of 97 children between 2 months to 59 months were assessed over a period of 2 weeks by the pediatrician. Measurements were done at the first hospital contact when children were admitted to the OPD or emergency. Every day, 6–10 children were administered the POx – they were selected as and when they came into the OPD or the emergency ward. Children with shock, toe deformities or inconsolability were excluded. For every selected child, the pediatrician measured the respiratory rate and wrote it in a column of the data collection sheet. A study consultant (junior resident) then used the multimodal device on the child, took an independent measurement and wrote it against the previous column.

Ethical clearance

The study did not require any ethical review as use of POx is a common practice in the children’s inpatient wards and is a standard practice management according to WHO IMCI guideline. The Masimo POx was used as it is a globally approved and standardized equipment.

Statistical analysis

We used three different types of analysis to measure linear association between the POx derived RR and physician derived RR: 1) Pearson Correlation Coefficient, 2) Bland Altman method, and 3) Sensitivity and Specificity analysis.

Advanced new equipments and instruments used in medical diagnosis need a few essential assessment of measurement’s acceptability theory. We used two broad types of measurement errors i.e. systematic errors (Bias) and random errors (Standard Deviation). Bias is mean difference between measured (pulse oxymeter) and true values (pediatrician measurement reference as accuracy) while random error is expressed as the standard deviation of measured values. We also stated root mean square deviation (RMSD) that combines both mean bias and random error to give a total error measure, calculated as follows:

$$ \mathrm{RMSD}=\surd \left(\left[\sum \left\{\mathrm{RRPulse}\ \mathrm{oxymeter}-\mathrm{RRpediatrician}\right\}2\right]/\mathrm{n}\right), $$

Further, Pearson correlation coefficient is represented as an indication of linear association between the measured and reference at p < 0.001. A linear fit and line of unity fit was tested to observe the data appropriateness for linearity (Fig. 1) Another test of association Lin’s concordance correlation coefficient was obtained as a single-magnitude summary of agreement. We assumed a normal approximation interval (α = .05) to compute the 95% confidence interval (95% CI) for these coefficients.

Fig. 1
figure1

Correlation between RR values obtained from both the measurements

Additionally, analysis by components of variance technique was followed, as described by Bland and Altman. This method compensates for multiple measurements being taken from the same subject when the true reference value is changing. A modified Bland–Altman diagram was used to graphically represent the data, displaying mean bias and limits of agreement. Here, the mean of the 2 independent observations was replaced by one of the reference measurements. That is, the reference is assumed to have negligible error and so forms the x-axis of the plot. We assumed a normal approximation of limit of agreement of measuring device (α = 0.05) to compute the 95% confidence interval (95% CI) for these coefficients. Further, sensitivity and specificity of pulse oximeter RR over reference was statistically analyzed. Clinically, fast breathing is recorded as more than or equal to 50 for 2 to 12 months old infants whereas for children above 12 months, it is more than or equal to 40. RR diagnosis result of children obtained from pediatrician was taken as reference hence RR measured by advance equipment Pulse Oxymeter was compared with this existing method (Table 2).

Analyses were performed using STATA 12.0 version.

Results

Total 97 subjects were obtained under study and all the data were analyzed. Selected demographic characteristics and respiratory diagnosis results using two measurements are presented in Table 1. About 75% children were male and average age of children was 15 months. Out of total 97 children, the majority (72%) were diagnosed to be suffering from pneumonia, most with some associated complications including TB and SAM while the rest were admitted for some form of sepsis, abscess, cellulitis and anemia. Average RR observed using POx was a bit lower (48.1) than reference measure (48.8). Average systematic error of measurement was observed by Pulse Oximeter less than 0 (0.40, SD = 3.31) i.e. on average test measurement showed lower values than reference results. Overall divergence from the reference measurement was found to be about 3 (RMSD) (Table 1).

Table 1 Characteristics of sampled children under study (N = 97)

Both methods of measurement on RR shows significant strong association (97%) over the RR values (Fig. 2) at 0.1% level of significance (p < 0.001) and again line of Linear fit on observed values lies on line of unity, hence, values obtained either from Pulse Oximeter or By Pediatrician (Gold Standard) are very close to each other (Fig. 3). While, the Lin’s concordance correlation coefficient indicates an overall agreement of 0.967 (0.958, 0.981) (Table 1).

Fig. 2
figure2

Line of fit of RR values obtained from both the measurements

Fig. 3
figure3

Bland and Altman Plot for difference in two measurements vs mean

From the sampled data, 95% limits of agreement (mean bias ±1.96 × SD) were 6.08 and − 6.89 breaths per minute. Hence the results measured by method POx are 7 units below or 6 units above method of pediatrician (Table1, Fig. 3).

High sensitivity (95%) and specificity (94%) of fast breathing as detected by POx with 95% of accuracy was obtained. There is 95% probability of a diagnostic test using POx that a child is correctly identified as having pneumonia. Similarly, higher specificity represents the 94% probability of a test diagnosis of a child without fast breathing who will be identified as negative in this study. Additionally, a considerably high level of agreement (Kappa = 0.85) was found between the POx and reference values of fast breathing and the difference in values is by chance alone (Table 2). The Masimo POx, thus, allows for significantly integrated result of RR and SPO2 with high sensitivity and accuracy in health care settings, as well as the possibility of rapid detection.

Table 2 Sensitivity and Specificity of RR of Pulse Oximeter

Discussion

The results show a high level of agreement between pleth based RR using a POx and physician measured RR, indicating a high accuracy of RR reading of the POx. Accurate measurement of respiratory rate is an essential step toward correct diagnosis of pneumonia [13]. RR is recognized as an important risk marker in community acquired pneumonia [14]. In the absence of doctors, front line health workers screen and diagnose diseases. In a study in Zambia, despite moderately high level of agreement between CHW classification of children with fast and normal breathing with that of experts, the limits of agreement were wide enough to lead the authors to conclude that further use of diagnostic tools such as POx would greatly help in assessing pneumonia in remote settings in LMIC [15].

On the other hand, there have been studies where inter observer agreement was found to be poor in RR measurements [16, 17]. Variability was found even when measurements were performed using WHO recommended methods leading the authors to suggest development of objective technological solutions [16]. Evidence shows POx to give the most accurate and objective measurement to identify hypoxemia in children and WHO recommends it as a standard diagnostic tool [13].

In our study, pleth based RR using a POx had a high level of agreement with that of the gold standard, indicating a high level of accuracy. The sensitivity analysis, in addition, points to the reliability of the device in correctly identifying fast breathing, a major symptom of the disease, in 95% of the cases. Using it in primary health care centers by staff who are not clinically qualified, like ANM and AYUSH doctors would strengthen the pneumonia management program by facilitating correct diagnosis of cases of pneumonia. Currently, health care workers are trained on IMCI guidlelines, the effect of which has been mixed- while a low evidence was found on fewer deaths among children from birth to 5 years in a Cochrane review, there was insubstantial evidence on the performance of health care workers in treating common illnesses [18]. Another review found that in 13 of 21 studies, health workers prescribed incorrect medicines, thus highlighting the need for correct diagnosis and prescription [19]. In our own project assessment, only 24% of ANM, CHO, and AYUSH doctors in health and wellness centers had complete knowledge of how to measure respiratory rate and recognizing all the danger signs, while only 11% had complete knowledge of the IMCI based signs of labored breathing [9]. The introduction of a reliable and accurate instrument in the diagnosis of pneumonia would not only ease the diagnosis process, but is expected to lead to simplified IMCI guidelines in pneumonia management. In the absence of clinicians, health workers diagnose pneumonia based on chest indrawing and fast breathing which is difficult to measure. The use of a multi-modal device could potentially simplify algorithms for pneumonia case management. Our study has limitations as the two measures were administered simultaneously- one after another, and the data collectors were not blinded, which may potentially carry recording bias. Another limitation is that we conducted the study in a tertiary hospital but we wish to carry the implication of the findings on to periphery facilities which has different levels and quality of infrastructure. Nevertheless, we believe that the evidence generated from a real life clinical situation has potential value in the application of pulse oximeter with pleth based RR measurement in periphery centres for pneumonia screening and management.

Conclusion

There is a high degree of agreement between pleth based RR using a POx and physician measured RR, indicating that the former provides reliable and accurate measurement. Current diagnosis and management of pneumonia in primary health care is based on variably trained health providers despite IMCI guidelines. The use of pulse oximeter, also recommended by WHO, which can provide reliable measurement would streamline pneumonia case management in these settings. The current study provides evidence of the reliability of a pulse oximeter.

Availability of data and materials

The raw data will be made available upon request.

Abbreviations

POx:

Pulse Oximeter

RR:

Respiratory Rate

PHC:

Primary health care

IMCI:

Integrated Management for Childhood Illness

WHO:

World Health Organization

LMIC:

Low and Middle Income Countries

Spo2:

peripheral capillary oxygen saturation

TB:

Tuberculosis

SAM:

Severe Acute malnutrition

USAID:

United States Agency for International Development

ANM:

Auxilliary Nurse Midwife

AYUSH:

Ayurvedic, Yoga and Naturopathy, Unani, Siddha and Homeopathy

CHO:

Community Health Officer

OPD:

Outpatient Department

References

  1. 1.

    Liu L, Oza S, Hogan D, Chu Y, Perin J, Zhu J, Lawn JE, Cousens S, Mathers C, Black RE. Global, regional, and national causes of under-5 mortality in 2000-15: an updated systematic analysis with implications for the sustainable development goals. Lancet. 2017;388(10063):3027–35.

    Article  Google Scholar 

  2. 2.

    McAllister DA, et al. Global, regional, and national estimates of pneumonia morbidity and mortality in children younger than 5 years between 2000 and 2015: a systematic analysis. Lancet Glob Health. 2018.

  3. 3.

    Lancet Volume 17, November 2017, Estimates of the global, regional and national morbidity, mortality, and aetiologies of lower respiratory tract infection in 195 countries: a systematic analysis for the Global Burden of Disease Study 2015.

  4. 4.

    Integrated management of childhood illness: chart booklet. Geneva: World Health Organization; 2014. Available from: http://apps.who.int/iris/bitstream/10665/104772/16/9789241506823_Chartbook_eng.pdf.

  5. 5.

    Lazzerini M, Sonego M, Pellegrin MC. Hypoxaemia as a Mortality Risk Factor in Acute Lower Respiratory Infections in Children in Low and Middle-Income Countries: Systematic Review and Meta- Analysis. PLoS One. 2015;10(9):e0136166.

    Article  Google Scholar 

  6. 6.

    Floyd J, Wu L, HayBurgess D, Izadnegahdar R, Mukunga D, Ghani AC. Evaluating the impact of pulse oximetry on childhood pneumonia mortality in resource-poor settings. Nature. 2015;528 (7580):S53–S59. doi: 10.1038/nature16043.

    Article  Google Scholar 

  7. 7.

    Olson D, Preidis G, Milazi R, Spinler J, Lufesi N, Mwansambo C, Hosseinipour M, McCollum ED. Task shifting an inpatient triage, assessment and treatment program improves the quality of care for hospitalised Malawian children. Tropical Med Int Health. 2013;18(7):879–86.

    Article  Google Scholar 

  8. 8.

    McCollum ED, King C, Deula R, et al. Pulse oximetry for children with pneumonia treated as outpatients in rural Malawi. Bull World Health Organ. 2016;94(12):893–902.

    Article  Google Scholar 

  9. 9.

    IPE Global Vriddhi project (2019). Baseline assessment of pulse oximetry. Internal assessment report. Unpublished.

  10. 10.

    Ministry of Health and Family Welfare. Government of India. Ayushman Bharat. https://ab-hwc.nhp.gov.in/document/6.

  11. 11.

    Sergio D. Bergese, Michael L. Mestek, Scott D. Kelley, Robert McIntyre, Jr, Alberto A. Uribe, Rakesh Sethi, James N. Watson, Paul S. Addison. Multicenter Study Validating Accuracy of a Continuous Respiratory Rate Measurement Derived From Pulse Oximetry: A Comparison With Capnography. Anesth Analg. 2017; 124(4): 1153–1159. Published online 2017 Jan 17. doi: 10.1213/ANE.0000000000001852.

    Article  Google Scholar 

  12. 12.

    Harris BU, Char DS, Feinstein JA, Verma A, Shiboski SC, Ramamoorthy C. Accuracy of pulse Oximeters intended for hypoxemic pediatric patients. Pediatr Crit Care Med. 2016 Apr;17(4):315–20.

    Article  Google Scholar 

  13. 13.

    WHO. Integrated management of childhood illness: caring for newborns and children in the community. In: Manual for the community health worker. Geneva: World Health Organization; 2002.

    Google Scholar 

  14. 14.

    Strauß, R., Ewig, S., Richter, K., König, T., Heller, G., & Bauer, T. T. (2014). The prognostic significance of respiratory rate in patients with pneumonia: a retrospective analysis of data from 705,928 hospitalized patients in Germany from 2010-2012. Deutsches Arzteblatt Int, 111(29-30), 503–v. doi:10.3238/arztebl.2014.0503.

  15. 15.

    Sinyangwe, C., Graham, K., Nicholas, S., King, R., Mukupa, S., Källander, K., … Hamade, P. (2016). Assessing the quality of Care for Pneumonia in integrated community case management: a cross-sectional mixed methods study. PLoS One, 11(3), e0152204. doi:10.1371/journal.pone.0152204.

    Article  Google Scholar 

  16. 16.

    Daw, W. J., Kingshott, R. N., & Elphick, H. E. (2017). Poor inter-observer agreement in the measurement of respiratory rate in children: a prospective observational study. BMJ Paediatr Open, 1(1), e000173. doi:10.1136/bmjpo-2017-0001.

  17. 17.

    Simoes EA, Roark R, Berman S, Esler LL, Murphy J. Respiratory rate: measurement of variability over time and accuracy at different counting periods. Arch Dis Child. 1991;66(10):1199–203.

    CAS  Article  Google Scholar 

  18. 18.

    Gera T, Shah D, Garner P, Richardson M, Sachdev HS. Integrated management of childhood illness (IMCI) strategy for children under five. Cochrane Database Syst Rev 2016, Issue 6. Art. No.: CD010123. DOI: 10.1002/14651858.CD010123.pub2.

  19. 19.

    Nguyen D.T.K., Leung K.K., McIntyre L., Ghali W.A., Sauve R. Does Integrated Management of Childhood Illness (IMCI) Training Improve the Skills of Health Workers? A Systematic Review and Meta-Analysis. (2013) PLoS One, 8 (6) , art. no. e66030.

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We are grateful to Kalawati Saran Hospital for permitting us to conduct the study. We acknowledge the efforts of Dr. Rachana Patel who conducted the data analysis.

Funding

This work was made possible by the support of the American people through the United States Agency for International Development and its VRIDDHI (scaling up RMNCH + A Interventions) Project, implemented by IPE Global Ltd. under the terms of Cooperative Agreement Number AID 386 A 14 00001. The contents of this paper represent the views of the authors and do not reflect the views of the US Government.

Author information

Affiliations

Authors

Contributions

VA and PK designed and conducted the study. VA wrote the introduction, and discussion. ES structured the manuscript, advised on sampling and data analysis and wrote major portions of the article. HK, PS, SG and AK contributed to results and discussion. HK critically reviewed the manuscript and contributed additionally to the background. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Enisha Sarin.

Ethics declarations

Ethics approval and consent to participate

We did not obtain any ethical review as use of POx is a common practice in the children’s inpatient wards and is a standard practice management according to WHO IMCI guideline. The Masimo POx was used as it is a globally approved and standardized equipment. We obtained permission from hospital authorities.

Consent for publication

Not applicable.

Competing interests

None declared.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Alwadhi, V., Sarin, E., Kumar, P. et al. Measuring accuracy of plethysmography based respiratory rate measurement using pulse oximeter at a tertiary hospital in India. Pneumonia 12, 4 (2020). https://doi.org/10.1186/s41479-020-00067-2

Download citation

Keywords

  • Under-five pneumonia
  • Respiratory rate
  • Pulse oximeter
  • Child health
  • India