Low Sensitivity of Malaria Rapid Diagnostic Tests Stored at Room Temperature in the Brazilian Amazon Region

Introduction: In remote areas of the Amazon Region, diagnosis of malaria by microscopy is practically impossible. This study aimed to evaluate the performance of two rapid diagnostic tests (RDTs) targeting different malaria antigens stored at room temperature in the Brazilian Amazon Region. Methodology: Performance of the OptiMal Pf/Pan test and ICT-Now Pf/Pan test was analyzed retrospectively in 1,627 and 1,602 blood samples, respectively. Tests were performed over a 15-month period. Kits were stored at room temperature in five community health centres located in the Brazilian Amazon Region. RDT results were compared with thick blood smear (TBS) results to determine sensitivity, specificity, and accuracy of the RDT. Results: The sensitivities of the OptiMal Pf/Pan test were 79.7% for Plasmodium falciparum malaria diagnosis and 85.7% for non-P. falciparum infections. The results showed a crude agreement of 88.5% for P. falciparum, and 88.3% for non-P. falciparum infections (Kappa index = 0.74 and 0.75, respectively). For the ICT-Now Pf/Pan test (CI 95%), the sensitivities were 87.9% for P. falciparum malaria diagnosis and 72.5% for non-P. falciparum infection. Crude agreement between the ICT-Now Pf/Pan test and TBS was 91.4% for P. falciparum and 79.7% for non-P. falciparum infection. The Kappa index was 0.81 and 0.59 for the final diagnosis of P. falciparum and non-P. falciparum, respectively. Higher levels of parasitaemia were associated with higher crude agreement between RDT and TBS. Conclusions: The sensitivities of RDTs stored at room temperature over a 15-month period and performed in field conditions were lower than those previously reported.


Introduction
In 2009, over 300,000 cases of malaria were reported in Brazil [1]; most of them were caused by Plasmodium vivax.Although less frequently, Plasmodium falciparum has also been responsible for severe cases, mainly when diagnosis and treatment have been delayed.Currently, early malaria diagnosis and treatment are the cornerstones of the Brazilian strategy for the disease control [1].Microscopic detection of Plasmodium spp.using the stained blood smears is a simple, highly sensitive and low-cost method that is the gold standard for malaria diagnosis.However, this method is time-consuming and its interpretation requires considerable expertise, particularly at low levels of parasitaemia.Malaria rapid diagnostic tests (RDT) are a relatively new and evolving technology which can provide parasite-based diagnosis in remote areas where microscopy is difficult to support and there is limited control of test storage conditions and supervision of users [2].
RDTs are based on the antibody detection of antigens specific to the malaria parasite present in the blood of infected people, such as histidine rich protein-2 (HRP2), lactate dehydrogenase (pLDH), and aldolase [2,3].In general, the sensitivity and specificity of RDTs are higher than 95% when they are performed under good storage conditions and when parasitaemia levels are higher than 100 parasites/µL for P. falciparum or higher than 500 parasites/µL for P. vivax [4].A higher the level of parasitaemia is associated with better the agreement between RDT and TBS [5,6,7].
The World Health Organization (WHO) recommends that RDT should be used only when its sensitivity and specificity are above 95% in patients with parasitaemia above 100 parasites/mL (WHO, 2000).In Brazil, the National Malaria Control Programme (NMCP) prescribes the use of RDT solely for remote, difficult to access, endemic areas in the Brazilian Amazon [8].In these locations, the diagnosis of malaria through microscopic analysis is practically impossible because of harsh environmental conditions combined with limited access to electricity and refrigeration, which are necessary for optimal functioning of sensitive equipment.Moreover it is difficult to recruit trained microscopy professionals to work in these remote areas.
Most of the companies that manufacture RDT recommend that the kits be stored at temperatures ranging from 2ºC to 30ºC (35.6ºF to 86ºF).The expiration dates for these tests are usually established according to such conditions.It is well-known that the shelf life and sensitivity of RDT are reduced when the kits are stored in temperatures above those recommended [9].It has already been demonstrated that increased temperatures lower the colour intensity of the chromatographic strips as well as the sensitivity of some RDTs for malaria antigens [10].
In the Brazilian Amazon the RDT kits are inevitably transported and stored at temperatures above 30°C (86° F) for prolonged periods of time.Therefore, it is important to monitor the quality and performance of the RDT when weather conditions are not ideal.The purpose of this study was to evaluate the performance of two RDT targeting different malaria antigens that were stored at room temperature in the Brazilian Amazon Region.

Methodology
The study was performed in five endemic municipalities for malaria in the Brazilian Amazon region: Colniza (MT), Porto Velho (RO), Manaus (AM), Macapá (AP) and Tucuruí (PA).Symptomatic patients with positive TBS for malaria attending health-care centres in the five selected cities were eligible to participate in this study.Signed, informed consent was obtained from all participants, or from a parent or legal guardian for subjects younger than 18 years old.All patients were treated and monitored by the research team using drugs recommended by the Brazilian NMCP.(panLDH).The target antigens for the ICT-Now Pf/Pan test are histidine rich protein-2 (pfHRP2) to diagnose P. falciparum and pan-malarial aldolase (pan-aldolase).Both kits belonged to the same manufacturing lot and were stored under the same ambient conditions as the room in which the RDT were performed.In the field, the RDT were performed according to manufacturer's instructions by two trained and experienced professionals from the healthcare centre.
All RDT kits were used within their expiry date.The same lot number for each RDT (46110.80.01 for the OptiMal Pf/Pan test and 017181 for the ICT-Now Pf/Pan test) were used over the course of the study.The RDT kits were transported in sealed 6 cm (2.4 inch) thick polystyrene foam boxes from the research centre to each health care centre, where the kits remained in the same boxes in which they were transported and placed in a ventilated, cool place.All centers were built with bricks and clay tile roofs, and remained open all day long.The foam box was opened in the morning and afternoon daily, and only for the time necessary to remove the kits that were going to be used.The temperature inside and outside the foam box was measured by a calibrated thermohygrometer (Incoterm, Porto Alegre-RS, Brazil; range 10 o C to 50°C accuracy ± 1°C each time the box was opened.
Since the number of available kits at the beginning of the study was limited, only TBS-positive patients were included in this RDT analysis.A new TBS was performed to confirm the presence of Plasmodium infection.The microscopic examination was performed by an experienced health professional from the health-care centre's staff.Parasite density was determined by applying a counting technique in which the number of parasites counted in 200 microscopic fields was converted to parasites per microlitre of blood, considering that 200 microscopic fields of the smear were equivalent to 0.4 µL of blood [11].A different health professional who was not aware of the patient's TBS result conducted the RDT on the same blood sample.
The RDT results were compared with the TBS results to determine sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy.Results from patients showing mixed malaria infections after the microscopic review were excluded from the analysis.Since only TBSpositive patients were included in this study, samples from patients with TBS positive for P. vivax were used to determine the specificity of each RDT for the diagnosis of P. falciparum infection and samples from patients with TBS positive for P. falciparum were used to determine the specificity of each RDT for the diagnosis of non-P.falciparum infection.All slides with discordant TBS/RDT results were reviewed by an experienced malaria microscopist.
The agreement between RDT and TBS results was analysed by the Kappa index.Confidence intervals of 95% (CI 95%) were determined for all parameters analysed.Chi-square for linear trend were calculated in order to test the effect of the parasite density on the agreement between RDT and TBS.Analysis of variance was used to compare the mean temperature of the study sites, registered throughout the research period.Linear regression was used to test the relation between RDT accuracy and the amount of time the RDTs were stored under field conditions.Significance was set at p < 0.05.Analyses were performed using the Epidata Analysis package (version 2.2.1) (Epidata Association, Odense, Denmark).
During the 15 months of the study, the mean (SD) of the maximum room temperature at the five locations where the RDT were stored was 30.3°C (2.1°C), ranging from 20.2°C to 34.4°C.No significant difference in temperature was observed among the study sites.Inside the foam boxes where the kits were stored, the mean (SD) of the maximum temperature was 29.4°C (1.6°C), ranging from 22.7°C to 33°C.
The OptiMal Pf/Pan test was performed in 1,602 blood samples; negative results for P. falciparum were found in 1,098 (68.5%) patients, and positive results for P. falciparum were found in 504 (31.5%) patients.The ICT-Now Pf/Pan test was performed on 1,627 samples, and negative results for P. falciparum were found in 1066 (65.5%) patients, while positive results for P. falciparum were found in 561 (34.5%) patients (Table 1).
The agreement between RDT and TBS results was negatively associated with the amount of time and room temperature in which the RDT was stored (Figure 1).The geometric mean of parasitaemia counts in patients with concordant results between the OptiMal Pf/Pan test and the TBS was 845.6 parasites/µL; in patients with discordant results the mean was 148.4 parasites/µL (p < 0.0001).For the ICT-Now Pf/Pan, the geometric mean of parasitaemia counts was 804.3 parasites/µL for concordant results and 403.4 parasites/µL for the discordant results (p < 0.0001).
To analyze the agreement between the results according to the antigens detected by the RDT, parasitaemia levels were stratified into less than 100, 100 to 500, 501 to 1,000, 1,001 to 5,000, and more than 5,000 parasites/µL.Agreement was positively correlated with the level of parasitaemia for both antigens detected for non-P.falciparum, and was statistically significant for aldolase (p < 0.001) and for LDH (p < 0.001).However, a negative correlation was observed between the agreement of the RDT targeting the HRP2 antigen for the diagnosis of P. falciparum and parasitaemia levels higher than 500 parasites/µL.A greater reduction in agreement occurred when the parasitaemia level was higher than 5,000 parasites/µL (p < 0.001).No significant negative correlation was observed for the RDT targeting the pfLDH antigen, even when the parasitaemia level was higher than 5,000 parasites/µL (Table 2).

Discussion
In general, studies evaluating the performance of RDT are conducted under adequate conditions in which tests are transported and stored at temperatures recommended by the manufacturer.Several studies have shown high sensitivity and specificity of the RDT, regardless of the antigens targeted by the RDT [6,12,13,14,15,16].In this study, RDT were conducted exclusively under field conditions and were exposed to environmental conditions typical of the Brazilian Amazon region for a prolonged period of time.The sensitivities and accuracies of OptiMal Pf/Pan and ICT-Now Pf/Pan tests were lower than those reported in studies performed in Africa, Asia, South America and at some travel medicine clinics [6,12,13,14,15,16,17,18,19].Low agreement between RDT and TBS results was found for both the diagnosis of P. vivax and P. falciparum, regardless of whether panLDH, pfLDH, aldolase, or HRP2 was targeted.Similar findings were also obtained by Ratsimbasoa et al., who evaluated three RDT targeting P. falciparum pLDH and pan-malaria pLDH under storage conditions similar to those used in this study [20].It is probable that the low agreement between test results found in this study was caused by RDT being stored at higher temperatures than those recommended.For the diagnosis of P. falciparum using pfLDH-based RDT, The sensitivity of pfLDH-based RDT in the diagnosis of P. falciparum was also found to be lower than that found in previous studies [6,12], including those conducted under field conditions [20].Exposure to high temperatures (with or without high humidity) has been suggested as a possible explanation for the poor performance of RDT in the tropics.RDT components can be affected by heat in several ways: heat-induced denaturation of antibodies in the test membrane can prevent their binding to the target antigen; damage to the nitrocellulose membrane forming the strip can change its flow characteristics or cause the antibody to detach from the membrane; and vulnerability to heat can vary with different RDT cassette designs [10].
The sensitivity of the ICT-Now Pf/Pan test for non-P.falciparum diagnosis was lower than that reported in other studies [14,21]  sensitivity for this species has been previously described for the Plasmodium aldolase-based RDT, even when these tests were conducted under laboratory conditions [18,22].This observation suggests that aldolase is an antigen with low sensitivity for the diagnosis of P. vivax, regardless of the environmental conditions under which this test is performed.Another explanation for the low performance of the aldolase-detecting RDT would be the low concentration of the parasite´s aldolase due to the presence of a polymorphism of the gene responsible for the production of aldolase.This finding was already been reported for some Plasmodium sp isolates of various origins, including some from Brazil [23].
On the other hand, the ICT-Now Pf/Pan test was observed to be a highly sensitive test for the diagnosis of P. falciparum, probably because of the presence of anti-HRP2 in the RDT.A meta-analysis of the performance of several RDTs showed that the HRP2detecting RDT for the diagnosis of P. falciparum demonstrate a higher degree of agreement with the Figure 1.Agreement between the results of two RDT and microscopy for the malaria diagnosis according to the time of RDT storage at room temperature in the Amazon Region TBS method [24].However, the agreement observed in this study was lower than that found in similar studies performed in malaria endemic areas of Africa [13,16,17], Asia [15], South-America [5], in travel medicine clinics in Europe [18] and the United States [14], and even in a group of patients with high levels of parasitaemia.Although exposure to high temperatures can explain the low sensitivity of the ICT-Now Pf/Pan test for the detection of P. falciparum [10], other factors must also be considered.For example, the absence of the target antigen in the parasite due to polymorphism or deletion of the gene that codes for HRP2 found in strains isolated in several malaria endemic regions is a possibility [23,25,26].However, the occurrence of these polymorphisms was described in just one strain of P. falciparum isolated in Brazil [27].
Previous studies have demonstrated that parasitaemia is a decisive factor in the performance of different RDTs, regardless of the antigen targeted by the test [19,24,28,29].Curiously, parasitaemia levels of the patients evaluated in this study were not positively associated with the RDT performance for the diagnosis of P. falciparum.On the contrary, a higher frequency of false negative results in the ICT-Now Pf/Pan test occurred in patients with parasitaemia levels higher than 5,000 parasites/µL.These findings have been previously described for tests targeting HRP2 performed in blood with parasitaemia levels higher than those described here, and have been attributed to the prozone effect, in which falsenegative or false-low results in immunological tests are caused by an excess of either antigens or antibodies in the blood [30,31].
Of the numerous studies published on the performance of RDT, only two of them state that the tests were stored at room temperature [20,32].In most of the studies performed in endemic regions the tests were kept under ideal storage conditions [5,15,17,25].In this study, the intentional exposure of the RDT to a wider temperature range (20.2°C to 34.4°C) for a prolonged period of time can be considered the main factor responsible for the low sensitivity observed.In fact, other investigators have observed a significant reduction in the levels of agreement between the results of the RDT and the TBS under simulated high temperatures ranging from 35°C to 60°C [10,33].In these cases, a lower level of agreement occurred in tests based on panLDH or aldolase enzymes, even when associated with HRP2.
In Brazil, the use of RDT is recommended in locations that do not have the infrastructure for the installation of cooling equipment.The temperature in such places is usually high during most of the year, and this can affect the performance of the RDT.Also noteworthy is the observation that RDT showed low sensitivity for the diagnosis of P. vivax, the most common species in the Brazilian Amazon.The WHO recommends that RDT should be applied in patients with parasitaemia levels above 100 parasites/ µL only when its sensitivity and specificity are above 95% [3]; thus the results of this study demonstrate the need for not only meticulous selection of the RDT to be used in programmes to control malaria, but also strict adjustment of its distribution and storage routine.
Some limitations of this study should be mentioned.It is possible that cross-reactions between falciparum and non-falciparum species may have underestimated the specificity of the RDTs since the RDT were performed only in Plasmodium positive samples.However, it is known that the rate of cross reactivity among HRP-2, pLDH and aldolase is very low [2].Moreover, the specificities detected in this study for both RDTs were not different from those reported in studies which performed the tests under ideal conditions [4,34].Another limitation of this study involves the quality of microscopic species differentiation; TBS were not read twice on an independent basis, which may have underestimated both the specificity and sensitivity of the tests.Unfortunately, the microscopy was not corroborated by PCR in this study.However, as all slides with discordant TBS/RDT results were reviewed and verified by an experienced malaria microscopist, the likelihood of bias was low.In addition, it is possible that the accuracy of both RDTs may have been overestimated as the reader of RDT was not blinded in this study and only TBS positive patients were included.This could have overestimated the accuracy of both RDT.
Other factors could explain the lower performance of the RDT found in this study, in comparison with the results seen in similar studies, including level of parasitaemias and field constraints (e.g., light for the RDT result reading, expertise of the technicians, field studies facilities in comparison to reference, etc.).In fact, the average parasitaemia patients included in this study was lower than that reported in other studies [5,6] and should have led to false negative results of RDT, as already demonstrated [19].

Conclusions
The sensitivities of the two RDTs studied (the OptiMal Pf/Pan test and the ICT-Now Pf/Pan test) stored at room temperature over a 15-month period and performed in field conditions was lower than those previously reported.The OptiMal Pf/Pan test showed better sensitivity and agreement with TBS than the ICT-Now Pf/Pan test for the diagnosis of P. vivax.In addition, the RDT based on the detection of HRP2 showed better sensitivity and agreement with the TBS than that based on the detection of pfLDH for the diagnosis of P. falciparum.For the diagnosis of P. vivax, higher levels of parasitaemia corresponded with better the agreement between RDT and TBS results.However, parasitaemia levels higher than 1 000 parasites/µL were negatively associated with the agreement between the RDT and TBS results for the diagnosis of P. falciparum, regardless of the RDT evaluated.
This study was approved by the Research Ethics Committee of the Federal University of Mato Grosso, as well as by the National Research Ethics Committee.Patients participated in the study during a 15month period from May 2005 to July 2006.To ensure availability of RDT kits throughout the period of the study, a similar number of tests was performed in each month.Approximately two tests per work day were performed in Colniza and in Porto Velho, and one test per work day was performed in Manaus, Tucuruí, and Macapá.

Table 1 .
Performance characteristics of two RDT compared to microscopy in field conditions in the Amazon Region, Brazil . In fact, low

Table 2 .
Agreement between the results of two RDT and microscopy, according to the level of parasitaemia and the RDT targeting malaria antigens