Temporal Patterns of Meningitis in Hamadan, Western Iran: Addressing and Removing Explainable Patterns

AUTHORS

Hastyar Pazhouhi 1 , Manoochehr Karami 2 , * , Nader Esmailnasab 3 , Abbas Moghimbeigi 2 , Mohammad Fariadras 1

1 Department of Epidemiology, Hamadan University of Medical Sciences, Hamadan, IR Iran

2 Modeling of Non-Communicable Diseases Research Center and Department of Biostatistics and Epidemiology, School of Public Health, Hamadan, IR Iran

3 Department of Epidemiology and Biostatistics, Kurdistan University of Medical Sciences, Kurdistan, IR Iran

How to Cite: Pazhouhi H, Karami M, Esmailnasab N, Moghimbeigi A, Fariadras M. Temporal Patterns of Meningitis in Hamadan, Western Iran: Addressing and Removing Explainable Patterns, Arch Clin Infect Dis. 2016 ; 11(3):e31532. doi: 10.5812/archcid.31532.

ARTICLE INFORMATION

Archives of Clinical Infectious Diseases: 11 (3); e31532
Published Online: July 16, 2016
Article Type: Research Article
Received: July 14, 2015
Revised: July 4, 2016
Accepted: July 4, 2016
Crossmark

Crossmark

CHEKING

READ FULL TEXT
Abstract

Background: Meningitis is one of the most disturbing infectious diseases due to mortality, morbidity and its ability to cause epidemic.

Objectives: The current study aimed to detect and remove explainable patterns of fever and neurological symptoms as suspected meningitis occurred in Hamadan province, West of Iran.

Materials and Methods: Monthly and daily data of suspected cases of meningitis of Iranian national surveillance system from 21st March 2010 to 20th March 2013 were used. explainable patterns of syndrome were identified using autocorrelation and partial autocorrelation functions, mean differences and nonparametric Mann-Kendall statistics. Besides moving average (MA) smoothing methods, Holt-Winters (HW) exponential smoothing and the Poisson regression model were used to remove such patterns.

Results: The study findings indicated the presence of explainable patterns including day-of-the-week (DOW), weekend, holiday effects, seasonality and temporal trend in the syndromic data of fever and neurological symptoms. Overall, HW exponential and regression method had better performances to remove explainable patterns.

Conclusions: Addressing and removing explainable patterns of syndromic data on meningitis is necessary to timely and accurately detection of meningitis epidemics. It was concluded that decomposition methods had better performance compared to the model based ones.

Keywords

Trend Outbreak Smoothing Meningitis Surveillance

Copyright © 2016, Infectious Diseases and Tropical Medicine Research Center. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/) which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.

1. Background

Meningitis disease typically results from contagious infections (1). Meningitis disease outbreaks can inflict damage and spread quickly (2). Controlling this disease is possible only though establishing a dynamic surveillance system, and accurate and updated reporting (3). Public health surveillance systems aim at detecting outbreaks using syndromic data (4). Routinely syndromic surveillance systems use outbreak detection algorithms for real time detection of outbreaks (5, 6). Data on fever and neurological symptoms of meningitis such as all pre-diagnostic data might have two patterns including: explainable patterns, i e, temporal dependency, day-of-the-week (DOW) effect, holiday effect and weekend effect; and unexplainable patterns which encompass actual decrease or increase in the data (4, 7).

The meningitis’s surveillance system in Iran established by the ministry of health and medical education in 1981 uses syndromic data, i e, fever and neurologic syndrome, as a pre-diagnostic data source to detect meningitis outbreaks and monitoring its trends (3). Monitoring tools or outbreak detection methods used for this purpose have two main assumptions i e, independent and identical distributions. Generally, the presence of explainable patterns in the pre-diagnostic data sources such as syndromic data in the meningitis surveillance system lead to some issues true alarms or outbreaks (8). One of the issues is violation from the above assumptions and the second issue is masking the unexplainable patterns such as outbreaks or producing false alarms (9).

Therefore, monitoring such pre-diagnostic data as suspected meningitis cases requires an initial detection and removal of such explainable patterns. The process of removing explainable patterns in the literature is called pre-processing methods (10). These methods fall under the general umbrella of model-based and data-driven methods. Holt-Winters (HW) exponential smoothing is a data-driven method presented in detail in the methods section (11).

Applying outbreak detection methods to the raw and syndromic data without removing the explainable patterns incapacitates the surveillance systems to detect and respond to the potential alarms in a timely manner by considering the challenge of applying outbreak detection algorithms to the fever and neurologic syndromic data as a pre-diagnostic data source with neither removing such patterns nor assessing violations from the independent and identical distribution in the meningitis surveillance system.

2. Objectives

The current study aimed to detect and remove the explainable patterns of the daily counts of suspected cases of meningitis in Hamadan province, Iran.

3. Materials and Methods

3.1. Data

Data on daily counts of the fever and neurologic symptoms of meningitis were obtained from national surveillance system from 21st March (the first day of Iranian solar Hijri-Shamsi calendar and Nowruz, the Iranian traditional New Year) 2010 to 20th March 2013 in Hamadan province, West of Iran. Also data on monthly counts suspected cases of meningitis were obtained from national surveillance system from March to April (the first month of Hijri-Shamsi calendar) 2004 and February - March 2013 in Hamadan province. In Iran, working days are Saturday to Wednesday; Thursday and Friday are weekends. Meningitis is considered as notifiable diseases and by definition when a suspected case of meningitis is detected by a health care provider, it must be reported as a clinical case. Suspected cases of meningitis are defined as individuals with clinical findings including an illness with sudden onset of fever (> 38.5°C rectal or 38.0°C axillary) and one of the following signs: neck stiffness, altered consciousness or other meningeal signs (neurologic signs). Such cases are reported to upper levels of the surveillance system daily. During the study period, from 21st March 2010 to 20th March 2013, 1506 meningitis suspected cases on daily basis were reported to the national surveillance system in Hamadan province.

3.2. Tools to Detect Explainable Patterns

To detect explainable patterns including DOW, holiday and weekend effects, different methods such as autocorrelation function (ACF), partial autocorrelation function (PACF), mean differences and Mann-Kendall test were used.

Autocorrelation function and partial autocorrelation function: ACF, PACF and related autocorrelogram at various lags were used to highlight cyclical effects and dependence time. Possible daily, weekly, monthly and annual cyclic patterns were shown at lag 1, 7, 30 and 365, respectively. Larger values of autocorrelation indicate the existence of the DOW effect and the presence of temporal pattern. Statistical significance of such patterns and white-noise are tested using the portmanteau (Q) statistics (12, 13).

3.3. Mann-Kendall Nonparametric Statistics

The statistics was used to determine the linear, non-linear and seasonal trends (14).

Deviation from the normal assumptions was investigated using the Skewness and Kurtosis values. Greater deviations from zero for the statistics indicate abnormal distribution.

3.4. Tools to Remove Explainable Patterns

HW exponential smoothing: HW exponential smoothing is thought to consist of three components including the level (Lt), trend (Tt) and seasonality (St) as shown in Equation 1.

Equation 1.

Where M is the number of seasons (for a period determined by the data a week being equal to seven) and three-component level (Lt), trend (Tt) and seasonality (St) are calculated as follows:

Equation 2.
Equation 3.
Equation 4.

In the Equation 2, α, β and γ are smoothing parameters that will take values between zero and one.

HW cumulative and HW multiplicative exponential smoothing were implemented using the suggested parameter values α = 0.04, β = 0, and γ = 0.15 (9).

3.5. Regression Model

To remove explainable patterns including DOW and seasonal effects, the generalized linear models (GLM) with the Poisson estimation were applied to the syndromic data of fever and neurological symptoms.

Equation 5.

[Σi ci × Ii(d)]: To get the DOW effect where Σi ci values are one to seven for weekdays and Ii (d) the values zero and one.

[C8 + C9 × d]: To get a long term effect.

[c10 × cos (2πd/365.25) + c11 × sin (2πd/365.25)]: To catch the seasonal effect, 365.25 average numbers of days of the year, C11 and c10 indicate the magnitude and amplitude of the seasonal effect.

Variables in the final model were selected using one of the stepwise approaches according to their own values i e, the smallest Akaike information criterion (AIC) (15, 16).

3.6. Moving Average Method

MA statistics, as explained above in details, is applied for smoothing removing explainable patterns from data (11).

All of the applied statistical methods to detect and remove explainable patterns were computed using the Microsoft excel Add In software entitled XLSTAT (17). Illustrations were drawn using Stata software version 11. All analyses were conducted using Microsoft excel Add In software entitled XLSTAT and Stata software version 11.

4. Results

The presence of DOW and temporal trend in the daily counts of suspected cases of meningitis using the ACF and PACF values with corresponding portmanteau (Q) statistics which reveals temporal dependency in the meningitis’ daily time series data are shown in Table 1. According to Table 1, higher autocorrelations in lags 1 to 3 and 12 indicate seasonal and yearly patterns in monthly data. In addition, all of the autocorrelation values in the above mentioned lags are statistically significant which favour the temporal dependency in the data. Visual inspection reveals higher autocorrelations on autocorrelogarms portend DOW effects, seasonal patterns and annual pattern (Figures 1 and 2).

Table 1. Values of Autocorrelation and Partial Autocorrelation Lag Delays of Daily Meningitis Suspected Cases From 2010 to 2013 and Delays of Monthly Meningitis Suspected Cases From 2004 to 2013
DataLagACPacPortmanteau (Q) StatisticsP-Value
Daily10.230.2399.62< 0.001
70.230.09330.21< 0.001
140.210.09511.15< 0.001
210.250.1604.11< 0.001
280.280.13619.57< 0.001
300.250.09683.62< 0.001
Monthly10.90.9390.58< 0.001
30.660.13206.61< 0.001
120.480.12463.17< 0.001

Mean values and corresponding 95% confidence intervals (CI) for daily reported number of suspected cases of meningitis according to days of week, workdays and weekends are presented in Table 2. After excluding holidays, comparison between the mean and 95% CI for the workday (1.49 (1.38 - 1.60) and the corresponding value for the weekends (1.14 (0.99 - 1.29) revealed the presence of weekend effect (Table 2). In other words, numbers of daily reported suspected cases of meningitis decreased during weekends. Overall, the trend of daily frequency peaks during the week on Tuesdays and then decreases toward the weekend. Regarding the holiday effects, the current study found that the average number of suspected cases of meningitis on holidays (1.09) was lower than those of workdays and weekends.

Table 2. The Mean and 95% Confidence Interval of Reported Cases of Suspected Meningitis on Weekdays From 2010 to 2013
Day of WeekNumber of DaysMean (95% Confidence Interval)
Saturday1431.41 (1.16, 1.66)
Sunday1461.4 (1.17, 1.62)
Monday1461.55 (1.28, 1.82)
Thursday1471.43 (1.21, 1.66)
Wednesday1471.66 (1.41, 1.92)
Tuesday1471.14 (0.94, 1.3)
Friday1561.15 (0.94, 1.36)
Workdays (Saturday to Wednesday)7291.49 (1.38, 1.6)
Weekends (Tuesday and Friday)2961.14 (0.99, 1.29)
Holidays (Fridays and other holidays)2201.14 (0.97, 1.3)
Holidays (except for Friday)641.09 (0.83, 1.35)

The trend analysis tests with the Mann-Kendall method are summarized in Tables 3 and 4. Significant positive trend and seasonal pattern in daily and monthly data series of suspected cases of meningitis can be observed. All of tau Kendall values were statistically significant which presented seasonality and temporal trend in the suspected cases of meningitis data.

Some evaluation metrics to measure the efficacy of smoothing methods to remove the explainable patterns (Tables 3 and 4) are presented. According to Table 3, pre-processing (smoothing), skewness and kurtosis values for daily counts of suspected cases of meningitis indicate violation from normality assumption. However, at the best results after pre-processing using seasonal multiplicative HW exponential smoothing, skewness value of 1.21 and kurtosis value of 4.35 changed to 0.15 and 4.84, respectively. Histograms and normal probability plot in columns 2 and 3 of Figures 1 and 2 revealed the violation normality assumption in the daily and monthly counts of suspected cases of meningitis. However, after pre-processing normality assumption improved. In addition, after implementing pre-processing using smoothing methods, there was a decline in the autocorrelation values in lags 7, 30 days in the daily counts of suspected cases of meningitis and autocorrelation values at lags 1, 3 and 12 months in the monthly counts of suspected cases of meningitis compared to the initial values. Tables 3 and 4 that reported Mann-Kendall test showed that positive trend and seasonal pattern in data before pre-processing was significant but after pre-processing Mann-Kendall test in any method smoothing was not significant. This report involved explainable patterns after pre-processing. This finding reflected the good performance of smoothing techniques to eliminate seasonal patterns such as DOW effects, monthly and yearly patterns. It was found that the HW exponential smoothing had better performance in removing DOW and monthly effects (Table 3). The GLM smoothing showed the best performance in removing the trend (Table 4). Plots of the daily counts of suspected cases of meningitis before and after pre-processing were shown using various smoothing techniques (Figures 1 and 2).

Table 3. Comparative Statistics for Daily Cases of Suspected Meningitis, Before and After Preprocessing Different Methods
IndexRaw DataMoving AverageHolt-Winter AdditiveHolt-Winter MultiplicativeGLM
Window (3)Window (7)
Mean (standard deviation)1.37 (1.44)0.00 (1.43)0.01 (1.34)0.00 (1.3)- 0.08 (1.53)1.27 (1.33)
Mean (weekend Standard deviation)1.37 (1.27)- 0.31 (1.35)- 0.23 (1.27)- 0.17 (1.28)- 0.38 (1.46)3.61 (0.59)
Skewness1.210.560.780.180.151.11
Kurtosis4.354.314.34.94.844.53
Autocorrelation values
Lag (7)0.23- 0.03-0.1- 0.010.040.11
Lag (30)0.250.060.060.060.020.16
Lag (365)N.AN.AN.AN.AN.AN.A
Mann-Kendall Test
Seasonal test
Kendall's tau0.220.01- 0.03- 0.05- 0.06- 0.04
P Value< 0.0010.510.90.990.990.46
Trend test
Kendall's tau0.220.02- 0.020.05- 0.06- 0.04
P Value< 0.0010.430.870.990.990.98

Abbreviation: GLM, generalized linear models.

Table 4. Comparison of the Statistics for Monthly Cases of Suspected Meningitis, Before and After pre-processing Different Methods
IndexRaw DataMoving Average-Window (2)Holt-Winter AdditiveHolt-Winter MultiplicativeGLM
Mean (standard deviation)15.31 (22.85)- 0.04 (7.9)0.41 (9.46)1.29 (11.1)0.07 (0.76)
Skewness1.421.511.382.191.59
Kurtosis3.7113.7612.4412.8814.83
Autocorrelation values
Lag (1)0.90.390.110.57- 0.04
Lag (3)0.65- 0.46- 0.01-0.240.05
Lag (12)0.480.30.290.260.18
Mann-Kendall Test
Seasonal test
Kendall tau0.68- 0.06- 0.010.09- 0.3
P Value< 0.0010.80.540.191
Trend test
Kendall tau0.62- 0.040.040.08- 0.34
P Value< 0.0010.670.330.371

Abbreviation: GLM, generalized linear models.

Plots to Detect Explainable Patterns and Comparing Pre-Processing Methods for Daily Cases of Suspected Meningitis, for the Raw Data (Top Row) and After Pre-Processing Different Methods
Figure 1. Plots to Detect Explainable Patterns and Comparing Pre-Processing Methods for Daily Cases of Suspected Meningitis, for the Raw Data (Top Row) and After Pre-Processing Different Methods
Plots to Detect Explainable Patterns and Comparing Pre-Processing Methods for Monthly Cases of Suspected Meningitis, for the Raw Data (Top Row) and After Pre-Processing Different Methods
Figure 2. Plots to Detect Explainable Patterns and Comparing Pre-Processing Methods for Monthly Cases of Suspected Meningitis, for the Raw Data (Top Row) and After Pre-Processing Different Methods

5. Discussion

Generally, examining the daily counts of suspected cases of meningitis as a pre-diagnostic data source showed the presence of explainable patterns and there was a great need to remove such patterns. To better explain the value of the study findings, first the key findings regarding the detection of the explainable patters and assessment of normality assumption were discussed. In the latter part of the discussion the pre-processing methods implemented to remove explainable patterns from the raw data were debated. As noted in the findings, the tools used to detect explainable patterns discovered the presence of seasonal patterns especially DOW and holiday effects in the daily counts of suspected cases of meningitis. A study with similar results had been performed on suspected cases of measles in Iran (18). Also, similar results were reported in both sales of throat lozenges and emergency department visits for gastroenteritis (13). The study by Mathes (19) in 2011 on the Evaluation of chief complaint data reported for near-real time tracking of communicable diseases to detect outbreaks or other unexpected disease clusters using emergency department daily visits indicated that DOW effect, Holiday effect and seasonal trends affected the number of visits. Furthermore, evidence of DOW effects on the bio- surveillance and pre-diagnostic data by different studies (15, 20-23) indicated the necessity of considering the role of seasonal patterns such as the month and strengthened the rationale.

One expected finding of the study was the trend of average number of suspected cases of meningitis and the inconsistency found in practice. According to Table 3, the daily counts of cases in the early part of the working week (Saturday, Sunday, Monday and Wednesday) were high. They reached their peak on Wednesday. On the weekend (Thursday and Friday), the number of reported cases reduced. One of the surprising findings from the large standard deviations of the reported cases from 2004 to 2013 was suspected meningitis. There seems to be a main reason regarding the inconsistency between this finding and expectations. The national surveillance system for meningitis implemented the routine activities and changes in the approach of electronically collecting reports that lead to the increase since 2010.

As is clearly mentioned above, the current study found that normality assumption for daily counts of suspected cases of meningitis was not met. This finding reflects the necessity of assessing normality assumption before applying outbreak detection algorithms such as cumulative sum control chart (CuSums) and exponentially weighted moving average (EWMA) to the pre-diagnostic data. Hence, those involved should take care when using such algorithms to avoid false alarms. Results of the studies by Lotze et al. (13), aimed to identify and eliminate explainable patterns of emergency department visits for gastroenteritis and Karami et al. (18) aimed to detect and remove explainable patterns of suspected cases of measles in Iran were consistent with the current study findings.

Regarding the efficacy of pre-processing techniques, after implementing smoothing in the daily counts of suspected cases of meningitis, both skewness and kurtosis changed. The result was consistent with that of a similar study (13), which implemented pre-processing using data-driven techniques and also model-based methods. The current study, similar to the study by Lotze et al. (13) demonstrated that seasonal multiplicative HW exponential smoothing technique was more efficient than the seasonal additive one to remove seasonal patterns such as DOW, holiday and weekends effects. In addition, the role of the techniques to eliminate temporal dependency in the daily frequency of suspected cases of meningitis was evident. There were two similar studies in the literature. One reported similar results (22), but was vague on the details of the applied methods. Lotze TH (9) suggested parameter values α = 0.04, β = 0, and γ = 0.15. As explained in the methods, HW exponential smoothing technique was implemented to both optimized smoothing parameter values and suggested parameter values α = 0.04, β = 0, and γ = 0.15.

Finally, according to the characteristics of the data it seems that implementing data-based smoothing methods such as HW exponential smoothing were simpler and better than model-based smoothing methods such as regression Poisson. For optimal performance of the model-based smoothing method, long time data should be driven. Besides this strength, it seems that applying confounding variables such as person and place in the pre-processing of the data should be considered in the future.

Lack of access to daily data of suspected meningococcal disease 2010 years ago, according to the Health Department of the portal has been launched in 2010. The limitations of the analysis, especially has been effective on methods based model.

5.1. Conclusions

Generally, daily counts of suspected cases of meningitis as pre-diagnostic data in the national surveillance system in Iran tend to include explainable patterns such as seasonal pattern, DOW effect, holidays and weekend effects. According to the prevalence of such patterns, it is highly recommended to implement pre- processing methods before applying outbreak detection algorithms in pre-diagnostic data.

Acknowledgements

Footnotes

References

  • 1.

    de Louvois J, Halket S, Harvey D. Neonatal meningitis in England and Wales: sequelae at 5 years of age. Eur J Pediatr. 2005; 164(12) : 730 -4 [DOI][PubMed]

  • 2.

    Yaro S, Lourd M, Traore Y, Njanpop-Lafourcade BM, Sawadogo A, Sangare L, et al. Epidemiological and molecular characteristics of a highly lethal pneumococcal meningitis epidemic in Burkina Faso. Clin Infect Dis. 2006; 43(6) : 693 -700 [DOI][PubMed]

  • 3.

    Ataee RA, Mehrabi-Tavana A, Izadi M, Hosseini SM, Ataee MH. Bacterial meningitis: a new risk factor. J Res Med Sci. 2011; 16(2) : 207 -10 [PubMed]

  • 4.

    Henning KJ. What is syndromic surveillance? MMWR. 2004; 53 : 5 -11

  • 5.

    Buehler JW, Hopkins RS, Overhage JM, Sosin DM, Tong V, C. D. C. Working Group . Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC Working Group. MMWR Recomm Rep. 2004; 53 : 1 -11 [PubMed]

  • 6.

    Yih W, Caldwell B, Harmon R, Kleinman K, Lazarus R, Nelson A, et al. National bioterrorism syndromic surveillance demonstration program. Morbid Mortal Weekly Report. 2004; : 43 -6

  • 7.

    Rothman KJ. Approaches to statistical analysis. 1998;

  • 8.

    Karami M. Validity of evaluation approaches for outbreak detection methods in syndromic surveillance systems. Iran J Public Health. 2012; 41(11) : 102 -3 [PubMed]

  • 9.

    Lotze T. Anomaly detection in time series: Theoretical and practical improvements for disease outbreak detection. DRUM. 2009;

  • 10.

    Fienberg SE, Shmueli G. Statistical issues and challenges associated with rapid detection of bio-terrorist attacks. Stat Med. 2005; 24(4) : 513 -29 [DOI][PubMed]

  • 11.

    Shmueli G, Burkom H. Statistical challenges facing early outbreak detection in biosurveillance. Technometrics. 2012;

  • 12.

    Kirkpatrick B, Reeves DS, MacGowan AP. A review of the clinical presentation, laboratory features, antimicrobial therapy and outcome of 77 episodes of pneumococcal meningitis occurring in children and adults. Infection. 1994; 29(2) : 171 -82

  • 13.

    Lotze T, Murphy S, Shmueli G. Implementation and comparison of preprocessing methods for biosurveillance data. Adv Dis Surveillance. 2008; 6(1) : 1 -20

  • 14.

    Hirsch RM, Slack JR. A nonparametric trend test for seasonal data with serial dependence. Water Resource Res. 1984; 20(6) : 727 -32

  • 15.

    Brillman JC, Burr T, Forslund D, Joyce E, Picard R, Umland E. Modeling emergency department visit patterns for infectious disease complaints: results and application to disease surveillance. BMC Med Inform Decis Mak. 2005; 5 : 4 [DOI][PubMed]

  • 16.

    Erbas B, Hyndman RJ. Statistical methodological issues in studies of air pollution and respiratory disease. 2001;

  • 17.

    Fahmy T. Commercial statistical and multivariate analysis software. 2011;

  • 18.

    Karami M, Soori H, Mehrabi Y, Haghdoost AA, Gouya MM. Detecting and removing the explainable patterns of the daily counts of suspected cases of measles as a prediagnostic data source in Iran. Iran J Epidemiol. 2012; 8(3) : 12 -21

  • 19.

    Mathes RW, Ito K, Matte T. Assessing syndromic surveillance of cardiovascular outcomes from emergency department chief complaint data in New York City. PLoS One. 2011; 6(2)[DOI][PubMed]

  • 20.

    Akhtar S, Mohammad HG, Michael E. Temporal epidemiology of microfilaraemia among migrant workers entering Kuwait. BMC Res Notes. 2008; 1 : 8 [DOI][PubMed]

  • 21.

    Brownstein JS, Kleinman KP, Mandl KD. Identifying pediatric age groups for influenza vaccination using a real-time regional surveillance system. Am J Epidemiol. 2005; 162(7) : 686 -93 [DOI][PubMed]

  • 22.

    Lee SL, Wong WH, Lau YL. Association between air pollution and asthma admission among children in Hong Kong. Clin Exp Allergy. 2006; 36(9) : 1138 -46 [DOI][PubMed]

  • 23.

    Michelozzi P, Forastiere F, Fusco D, Perucci CA, Ostro B, Ancona C, et al. Air pollution and daily mortality in Rome, Italy. Occup Environ Med. 1998; 55(9) : 605 -10 [PubMed]

  • COMMENTS

    LEAVE A COMMENT HERE: