SOJ Microbiology & Infectious Diseases

Brucellosis is a bacterial disease of high veterinary importance, besides being a significant zoonosis worldwide. It is characterised by variable, and at times long and latent, incubation periods, which makes its control and eradication a challenging effort. No single test is able to identify all stages of the disease with high sensitivity. The brucellin skin test, which has been proved to identify some acute and chronic latent stages of brucellosis, has not been validated in South Africa. The objective of this study was to evaluate the brucellin skin test in brucellosis free, as well as confirmed infected cattle herds, under South African commercial farming conditions. The results indicate that the brucellin skin test has a high specificity (99.2% (95% confidence interval (CI): 98.1% – 99.7%)), and that despite a relatively low sensitivity (42.9 % (95% CI: 30.5% – 56.0%)), it was able to identify all the infected herds and detect more infected heifers in infected herds when compared to the routinely used Rose Bengal Test and Complement Fixation Test. It was concluded that the brucellin skin test could be used in parallel with serological tests to improve the sensitivity of the current diagnostic strategy in chronically infected herds where eradication of the disease has proven problematic. Removal of more infected heifers would decrease infection pressure and streamline vaccination towards heifers not exposed to Brucella abortus.

Keywords: Bovine brucellosis; brucellin skin test; validation; specificity; sensitivity; vaccination

Brucellosis, due to infection by gram-negative bacteria of the genus Brucella, is a disease of socio-economic and zoonotic importance worldwide [28]. In bovine it is associated with the ingestion of material contaminated with Brucella abortus (B. abortus) while in humans it is associated with the consumption of unpasteurised milk (including products there-from) of infected animals. In addition, brucellosis may be acquired from occupational exposure to infected material of animal origin as with farmers, veterinarians, abattoir, and laboratory workers [28]. The presence of brucellosis in South Africa was first suspected in the late nineteenth century, although the first reliable veterinary report was by Gray in 1906 [16]. It is still present in the country today, with an estimated prevalence of 8 - 10% and reported annual losses upwards of R300 million [8,17]. Although the global incidence is over half a million infections annually, human brucellosis is largely neglected in South Africa [29,6]. Owing to the direct correlation between human incidence and animal prevalence, control of brucellosis is largely a veterinary function [25,26]. Indeed, a mass vaccination programme in small ruminants resulted in a significant decline in the incidence of the zoonotic disease in the human population in Greece, while in Korea, a reduction of human brucellosis cases was nearly immediately achieved by reducing bovine brucellosis [19,21]. In South Africa, though largely unsuccessful, concerted effort to control bovine brucellosis started in 1968 with the introduction of the Bovine Brucellosis scheme [5].

However, veterinary control is compromised by the chronic nature and the variable incubation period of the disease. It is estimated that up to 15% of cattle in infected herds abort before sero-conversion (Godfroid 2013, pers. comm.). Latency, which involves up to 10% of calves born to infected dams, is another complicating factor as such infected animals tend to test negative to routine serological tests, only to seroconvert in the peri-parturient period. This characteristic of brucellosis offers opportunity for disease spread within and between herds before diagnosis is made. Brucellosis vaccination is a valuable tool to reduce the rate of abortion and limit disease spread within an infected herd. The legislation in South Africa provides for the mandatory brucellosis vaccination of all breeding heifers between the ages of four and eight months, and serological testing from 18 months onwards (Animal Diseases Act, 1984 (Act 35 of 1984)). However, vaccination is not curative when administered to an already infected (including the latently infected) individual, which is likely to abort at first pregnancy regardless of the vaccination status [2]. It follows that failure to identify and remove all infected animals from a herd, even where routine vaccination is practiced, compromises eradication efforts as infection is maintained by this vaccinated but infected group of animals. In addition, the currently used serological tests may be unable to distinguish brucellosis from cross-reacting antibodies from other infections or brucellosis vaccines [10].

The Brucellin Skin Test (BST) is a cell-mediated immune assay which is based on the use of purified and standardised protein allergen, almost completely devoid of lipopolysaccharide. The BST relies on the delayed-type hypersensitivity reaction whose results are interpreted 72 hours post brucellin injection. Therefore an animal is restrained twice: to administer brucellin, and to read the result. While serological tests depend on circulating antibodies where latently infected animals may elude detection, the BST is independent of humoral immunity. Its high specificity in non-vaccinated animals may be exploited to complement serology by assessing both humoral and cellular immunity [28]. The BST is an alternative immunological test recommended by the OIE. The BST has proved a valuable additional test in diagnosing early and latent infections as well as in differentiating brucellosis from cross-reacting organisms in unvaccinated cattle in Europe [4, 9, 30, 32].

The objective of this study was to evaluate the performance of the BST under South African conditions, and investigate whether it can improve the overall sensitivity of the currently used serological testing regimen in identifying infected heifers.

The study was conducted in fourteen herds and involved 1451 head of cattle in the districts of Lekwa, Dipaleseng, Msukaligwa, Emalahleni, Govan Mbeki and Steve Tshwete in Mpumalanga Province (South Africa). In order to minimise the effect of maternal as well as vaccinal antibodies on the tests, only calves between the ages of three and nine months (prior to vaccination for brucellosis), were selected. The herds were selected to represent, as far as possible, the farming systems of South Africa in terms of herd size, production system (beef or dairy) as well as the different breeds and management systems employed therein. The brucellin skin testing, as well as blood collection, was conducted on the farms between November, 2010, and August, 2011, while the serological tests were conducted between December, 2011, and March, 2012, at the Agricultural Research Council’s Onderstepoort Veterinary Institute (ARC-OVI) for the RBT (Rose Bengal test) and CFT (Complemement Fixation Test), and at the Department of Veterinary Tropical Diseases, University of Pretoria, for the iELISA(indirect enzyme linked immunosorbent assay).

There were 608 heifers selected from five herds on four brucellosis negative farms. They consisted of 155 mixed breed, 430 Drakensberger, and 23 Holstein-Friesland calves. The farms were certified brucellosis-free in accordance with the official Bovine Brucellosis Scheme (Animal Diseases Act, 1984, Bovine Brucellosis Scheme- Section 10) with monthly testing by the MRT (milk ring test), or annual RBT and CFT. The herds had traceable records as kept by the Department of Agriculture, Forestry and Fisheries in Mpumalanga Province. The minimum certified period of freedom was three years.

A total of 843 heifers, from ten known infected herds, were tested. They consisted of 410 of mixed breed and 433 Drakensberger. The herd infection status was based on routine serological testing, supported by, at least one bacteriological, isolation of the B. abortus organism.

Approximately 10 ml of blood was collected from each test animal by venipuncture of either the jugular or the median caudal vein into Vacutainer™ tubes without anti-coagulant. The blood was allowed to clot and the serum separated and stored at -20°C until the time of testing.

The procedure was conducted as described [28, 32]. An area of approximately ten square centimetres of healthy skin on the side of the neck was clipped with a pair of scissors and the measurement of normal skin thickness taken with a springmeter (Hauptner). With the aid of a disposable tuberculin syringe coupled to a 4mm, 25 gauge needle 100 μl of brucellin (batch 10 0001, MEGACOR diagnostic, Austria) was injected intradermally to leave a visible pea-sized nodule at the injection site. The reaction was assessed 72 hours post-injection primarily by sight and palpation followed by measurement with a spring meter. The occurrence of a skin reaction was verified qualitatively as either a firm well circumscribed induration or as a soft oedematous induration. However, the qualitative nature of the reaction was not factored into the diagnostic assessment. The same operator took both pre and post injection measurements to minimise operator variation.

The test procedure was performed as described [28]. A volume of 25 μl of antigen (Onderstepoort Biological Products (OBP), South Africa, batch 146) was added to 25 μl of test sera at room temperature (22 ± 4°C) and mixed in a WHO haemoagglutination plate before incubation for four minutes on a rotary agglutinator (Heidolph polymax 2040, Heidolph, Germany). Any agglutination, as read immediately after incubation with the aid of an X-ray viewing box, was interpreted as a positive result.

The CFT was performed as described [28]. The test was carried out in 96-well u-bottomed microtitre plates (NUNC, Thermo Scientific). The positive control serum was supplied by Onderstepoort Biological Products (lot 5), while complement (batch 303 284) and amboceptor (batch 302 183) were supplied by Siemens (Germany). When the procedure was completed, the plates were read over a magnifying mirror by comparing the haemolysis to standards corresponding to 0-4 (where 0= 100% lysis, and 4= 0% lysis), and scored according to the International CFT Units/ml table. Only results above 20 ICFTU (international complement fixation test units) were classified positive.

The iELISA assay was done according to the manufacturer’s instructions using the Chekit™ (batch BAT 1132 220 – X101, Idexx Laboratories, Switzerland). Briefly, test sera were added to Brucella antigen pre-coated microtitre plates in the supplied dilution buffer and incubated in a humid chamber at 37 ± 2°C for one hour. After washing to remove unbound antibody, Chekit anti-ruminant Ig-PO conjugate was added and the plate incubated as above for a further one hour before washing to remove unbound conjugate. Chekit TMB substrate was added, followed by 15 minutes incubation at room temperature (18-25°C) before the reaction was stopped (Chekit-Stop solution). The plate was read at 450 nm in a microtitre plate reader (Powerwave X52, BioTek, USA) with the aid of Gen 51.11 software (BioTek). The results were analysed using the formula: Sample OD% = (OD sample – OD negative control)*100/ (OD positive control – OD negative control). Test sera with optical density values equal to or greater than 80% were classified positive.

In order to re-enforce the freedom status of the brucellosis free control population, all the animals were subjected to the RBT, CFT, and iELISA.

Similarly, in order to identify reference infected animals to constitute the infected control population, all brucellosis exposed animals from the infected herds were subjected to the RBT, CFT and iELISA. In such cases as brucellosis, where no single suitable reference standard exists, it may be prudent to employ a Composite Reference Standard (CRS), where two or more assays with acceptable sensitivity are combined and the individual’s status determined by test agreement [15,18,35]. The CRS in this study employed the RBT, iELISA and the CFT, and the individual status was determined by test agreement of at least two of the three assays.

To determine its diagnostic performance, the mean of the individual increases in skin fold thickness at the brucellin injection site (in mm) was compared between the brucellosis free and the exposed reference populations. The cut-off value was established to achieve a minimum of 99% specificity [14]. Diagnostic accuracy was further evaluated with the aid of the receiver operating characteristic (ROC) curve analysis in StatsDirect version 2.7.8 software.

Having established its diagnostic cut-off value, the performance of the BST was compared to that of the RBT, CFT and iELISA, respectively in 843 brucellosis exposed heifers from ten naturally infected herds. The comparison was based on the brucellosis reactor rate of each test. In addition, result agreement between the BST, and the RBT, CFT and iELISA was assessed by means of the kappa statistic [37].

Kappa, K = (po – pe) / (1 – pe)
(Where, po = observed agreement, and pe = expected agreement).

Following the characterisation of the BST performance, the merit of widening the brucellosis detection window through the parallel use of the BST and serology was investigated by comparing the net positive reactor rate of each combination of BST with one serological test (BST-RBT, BST-CFT, and BST-iELISA) in 843 brucellosis exposed animals.

All 608 brucellosis free animals tested negative to CFT and iELISA, with one animal testing positive to RBT. All 608 subsequently constituted the brucellosis free reference group to evaluating the BST.

Among the 843 animals originating from infected herds, 53 (6.3%) reacted positively to the RBT, 58 (6.9%) were positive to CFT, while 126 (14.9%) were positive to iELISA. Seven hundred and thirteen animals did not react to any of the three tests (Figure1 and Table 1).

Sixty three heifers were identified as reference infected (i.e., classified positive by at least two tests), and subsequently constituted the infected reference population for the purpose of validating the BST (Figure 1).

In B. abortus-free reference animals, the average skin reaction to the BST was 0.1 mm (95% confidence interval (CI) 0.1 mm to 0.2 mm). The skin reactions in 603 out of 608 calves (99.2%) < 1.1 mm. Furthermore, less than 1% (5/608) of B. abortus-free reference animals showed a skin reaction above 1.5 mm (Table 2). The cut-off value of 1.5 mm was selected and resulted in an optimum specificity of 99.2% (95% CI: 98.1% – 99.7%).

Twenty-seven of the 63 reference infected animals were classified as positive by the BST, which resulted in a relative sensitivity of 42.9% (95% CI: 30.5% – 56.0%) (Table 2).

In order to further assess the diagnostic accuracy of the BST relative the CRS, a ROC analysis was performed, yielding a resultant area under the curve of 0.7 (95% CI: 0.6 – 0.8).

In the B. abortus-infected herds (which are not the same as the reference infected animals), 78 animals (9.3%) were classified positive by the BST

(9.3%), CFT (6.9%) and RBT (6.3%). Sixty three animals were classified positive by at least two serological tests (7.5%) compared to 78 animals classified positive by the BST (9.3%) (Table 3). Among the 78 BST positive animals, eight (10.3%) tested negative to all three sero-assays, and none was positive to RBT alone.

The kappa coefficients of agreement were: BST-iELISA (0.45, 95% CI: 0.35 – 0.55), BST-CFT (0.28, 95% CI: 0.14 – 0.42), and BST-RBT (0.27, 95% CI: 0.12 – 0.40); indicating a moderate to fair agreement (Table 4).

Parallel combination of BST with serology yielded net brucellosis reactor rates of: BST-iELISA (18.0%), BST-CFT (13.4%), and BST-RBT (13.1%), respectively (Table 4).

In an ideal situation a new diagnostic test should be evaluated against a perfect gold standard test which assigns infection status with certainty [1]. In field studies involving bovine brucellosis a perfect gold standard is not available and a less than perfect standard must be employed [18, 20]. As a standard, bacterial culture combines high specificity with a relatively low sensitivity. Consequently, studies which employ the bacterial culture reference standard tend to overestimate the index test’s sensitivity owing to the positive correlation between culture positive animals with immunologically more advanced cases, which are not truly representative of the infected target population [18, 20, 35]. Serological tests are less cumbersome, safer and more economical than bacterial culture; however, no single test can accurately detect all stages of brucellosis [27]. Therefore the dilemma of the reference standard influences the legitimacy of comparison. In situations where the index test is sensitive enough to correctly identify positive cases beyond the identification spectrum of the reference standard, the new assay is erroneously judged as showing a high false positive reactor rate when they are indeed true positives [1]. With a CRS composed of the RBT, CFT and iELISA, our study set out to investigate the diagnostic performance of the BST in South African herds and its potential to contribute to the improvement of the diagnostic sensitivity of the currently used brucellosis serological methods.

The relative BST sensitivity of 42.9% observed in this study (Table 2) fell within the range reported for other studies [3,7,22,24,30]. The apparent variation in the observed relative BST sensitivity between studies may in part be attributed to the different reference standard definition of an infected animal. The study by Bercovich and ter Laak (1990) used bacterial isolation to define a reference infected individual to calculate sensitivity. It follows, the relative BST sensitivity values derived from their study are likely to incorporate some spectrum bias towards animals with advanced immunological response, and thus may possibly be an overestimation of the true sensitivity in the target population, which may include early and latent infections not easily detected by a less sensitive test, or culture[18].Other research found BST sensitivity of between 93% and 78% in experimentally infected animals at 27 and 187 days post infection, respectively [32]. They found that the BST sensitivity decreased with increasing post-infection period; which could be the reason for the lower relative sensitivity value observed in our study. De Massis and others (2005) reported BST sensitivity closely similar to that observed in this study at 414 days post RB51 vaccine inoculation.

The BST specificity of 99.2% observed in our study was similar to that reported in other studies [22,30,32]. The high BST specificity observed in this study implies a similarly high positive predictive value for this assay [23]. In these circumstances, it is highly likely that the 51(78 less 27 in agreement between CRS and BST) animals positive to the BST, but negative to the CRS, is a reflection of what our gold standard missed, and hence represented truly infected individuals [1]. This observation may be explained by a gold standard which was biased towards animals that had mounted a strong humoral, but not necessarily a strong CMI response to infection. In other words, the CRS employed in our study did not completely eliminate spectrum bias. Among the 78 BST positive animals, eight (10.3%) tested negative to all three sero-assays. The true status the individual animals which tested positive on BST but negative on serology could not be determined within the limitations of this study as it would have required long term follow up serological monitoring. However, the high confidence that these animals were infected but latent to conventional serology come from the high specificity exhibited by the BST in brucellosis free herds under similar environmental conditions.

The BST area under the ROC curve of 0.7 observed in this study is a low to moderate accuracy score [38]. The apparently low BST ROC ratings, despite the high specificity and high reactor rate among infected herds, could be attributed to a CRS which was biased towards strong humoral but not necessarily strong CMI response, and thus not representative of a typical field situation [31].

It has been documented elsewhere that iELISA’s have superior sensitivity to the RBT and CFT [12, 33]. As a consequence, in known infected herds, animals classified positive by iELISA and negative by RBT and/or CFT, should be considered as infected [11]. The results in our study are consistent with this observation. Indeed, BST (9.3%) and iELISA (14.9%) showed higher detection rates in the infected herds, compared to RBT (6.3%) and CFT (6.9%), (Table 4). Agreement between diagnostic tests, as measured by the kappa coefficient, was only fair between BST on one hand, and RBT and CFT (both at 0.3) on the other. The kappa statistic between BST and iELISA was 0.4, indicating a moderate agreement and perhaps reflecting the ability of the iELISA to detect early brucellosis infections [33,36]

Perhaps the most notable finding of this study was the overall improvement in the brucellosis detection rate when the BST was employed in parallel with serology. Following this testing strategy, the BST identified 57, 55, and 26 infected heifers which are in addition to those identified by the RBT, CFT and iELISA, respectively, among the 843 brucellosis exposed animals in the study (Table 4). In the study, parallel pairing of BST-RBT, BST-CFT and BST-iELISA identified combined totals of 110, 113, and 152 infected heifers respectively (Table 4). Other studies [7,36] reported that when applied in parallel, the BST and CFT with specific RB51 antigen detected all animals that had previously been inoculated with RB51 vaccine, more so when serology was applied 10 – 20 days post BST to exploit the anamnestic immune response [34]. Thus to employ the BST-iELISA parallel regimen prior to vaccination and breeding would maximise the detection of B-abortus infected heifers, which otherwise would maintain infection despite vaccination [2, 13, 31]. The authors would like to reiterate that there is no ‘curative’ vaccine registered for animal brucellosis, and assessment of protection in animals infected prior to vaccination is not documented in international scientific literature. Thus, the pre-vaccination withdrawal from breeding of most of the infected animals simultaneously maximises vaccine herd immunity while reducing herd exposure to infection leading to the rapid elimination of brucellosis from an infected herd.

To date, efforts to control brucellosis in South African have largely been unsuccessful. We are of the opinion that apart from an ineffective test and slaughter programme, the limitations of routine serology in the identification of latently infected heifers prior to breeding has contributed immensely to the current brucellosis state in South Africa. The authors believe three possible approaches ought to be considered within each infected herd. The first scenario, which largely mimics the present situation, allows for the vaccination of all pre-breeding heifers without prior testing. This approach will most likely lead to the persistence of the herd infection status owing to the maintenance of a reservoir of infection through the vaccination of infected heifers. The second scenario involves the incorporation of pre-vaccination RBT screening of all pre-breeding heifers, as per the current testing regime. The data in Table 4 show that such an approach may lead to failure because about 50% of infected animals are likely to be misclassified false negative, vaccinated and incorporated into the breeding pool. Consequently, the brucellosis infection status is likely to be maintained. The third scenario involves pre-vaccination screening of all pre-breeding heifers with the BST-iELISA parallel regime. The authors believe the third approach offers the best opportunity to identify and exclude all infected heifers from the breeding pool prior to vaccination. The approach allows only healthy heifers to enter the breeding programme. In turn, high herd immunity coupled with low brucellosis exposure offer the protection levels necessary for the elimination of brucellosis from infected herds.

The above proves the value of the BST to the overall improvement of brucellosis diagnostic sensitivity, thus permitting better effective use of the available vaccines to facilitate the eventual eradication of bovine brucellosis from South African herds.

We acknowledge financial support from the Department of Veterinary Tropical Diseases (University of Pretoria, Faculty of Veterinary Science) and the Institute of Tropical Medicine, Antwerp. We are indebted to the Department of Agriculture, Forestry and Fisheries (Mpumalanga Province, South Africa), Ekron M.D. (retired) and Storm A., as well as the ARC-OVI (Bacterial Serology), Potts A.D, for providing training support during the laboratory analyses.

1. Alonzo TA, Pepe MS. Using a combination of reference tests to assess the accuracy of a new diagnostic test. Stat Med. 1999;18(22):2987-3003.
2. Banai M. Control of small ruminant brucellosis by use of Brucella melitensis Rev.1 vaccine: laboratory aspects and field applications. Vet Microbiol. 2002;90(1-4):497-519.
3.    Bercovich Z, ter Laak EA. An evaluation of the delayed-type hypersensitivity test for diagnosis of brucellosis in individual cattle: a field study. Vet Microbiol. 1990;22(2-3):241-248.
4.   Bercovich Z, ter Laak EA, Lipzig JHH. Detection of brucellosis in dairy herds after an outbreak of the disease using a delayed-type hypersensitivity test. Preventive Veterinary Medicine. 1992;13(4):277–285.
5.     Bosman PP. Scheme for the control and eventual eradication of bovine brucellosis. J S Afr Vet Assoc. 1980;51(2):75-79.
6.     Dean AS, Crump L, Greter H, Schelling E, Zinsstag J. Global burden of human brucellosis: a systematic review of disease frequency. PLoS Negl Trop Dis. 2012;6(10):e1865. doi: 10.1371/journal.pntd.0001865
7.     De Massis F, Giovannini A, Di Emidio B, Ronchi GF, Tittarelli M, Di Ventura M, et al. Use of the complement fixation and brucellin skin tests to identify cattle vaccinated with Brucella abortus strain RB51. Vet Ital. 2005;41(4):291-299.
8.    Ducrotoy M, Bertu WJ, Matope G, Cadmus S, Conde-Álvarez R, Gusi AM, et al. Brucellosis in Sub-Saharan Africa: current challenges for management, diagnosis and control. Acta Trop. 2017;165:179-193. doi: 10.1016/j.actatropica.2015.10.023
9.   Fensterbank R. Allergic diagnosis of bovine brucellosis. 2. Use of the allergic test in infected herds. Ann Rech Vet. 1977;8(2):195-201.
10. Godfroid J, Saegerman C, Wellemans V, Walravens K, Letesson JJ, Tibor A, et al. How to substantiate eradication of bovine brucellosis when aspecific serological reactions occur in the course of brucellosis testing. Vet Microbiol. 2002;90(1-4):461-477.
11. Godfroid J, Nielsen K, Saegerman C. Diagnosis of brucellosis in livestock and wildlife. Croat Med J. 2010;51(4):296-305.
12. Godfroid J, Scholz HC, Barbier T, Nicolas C, Wattiau P, Fretin D, et al. Brucellosis at the animal/ecosystem/human interface at the beginning of the 21st century. Prev Vet Med. 2011;102(2):118-131. doi: 10.1016/j.prevetmed.2011.04.007
13. Godfroid J, Dahouk S, Pappas G, Roth F, Matope G, Muma J, et al. A “One Health” surveillance and control of brucellosis in developing countries: moving away from improvisation. Comp Immunol Microbiol Infect Dis. 2013;36(3):241-248. doi: 10.1016/j.cimid.2012.09.001
14. Greiner M, Sohr D, Göbel P. A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests. J Immunol Methods. 1995;185(1):123-132.
15. Greiner M, Gardner IA. Epidemiologic issues in the validation of veterinary diagnostic tests. Prev Vet Med. 2000;45(1-2):3-22.
16.   Henning MW. Animal diseases in South Africa. 3rd ed. Pretoria: Central News agency; 1956.
17.   Hesterberg UW, Bagnall R, Perrett K, Bosch B, Horner R, Gummow B. A serological prevalence survey of Brucella abortus in cattle of rural communities in the province of KwaZulu-natal, South Africa. J S Afr Vet Assoc. 2008;79(1):15-18.
18. Jacobson RH. Validation of serological assays for the diagnosis of infectious diseases. Rev Sci Tech. 1998 Aug;17(2):469-526.
19.   Jelastopulu E, Bikas C, Petropoulos C, Leotsinidis M. Incidence of human brucellosis in a rural area in Western Greece after the implementation of a vaccination programme against animal brucellosis. BMC Public Health. 2008; 8:241.doi: 10.1186/1471-2458-8-241
20. Knottnerus JA, van Weel C, Muris JW. Evaluation of diagnostic procedures. BMJ. 2002;324(7335):477-480.
21. Lee HS, Her M, Levine M, Moore GE. Time series analysis of human and bovine brucellosis in South Korea from 2005 to 2010. Prev Vet Med. 2013;110(2):190-197. doi: 10.1016/j.prevetmed.2012.12.003
22. MacDiarmid SC, Hellstrom JS. An intradermal test for the diagnosis of brucellosis in extensively managed cattle herds. Preventive Veterinary Medicine.  1987;4:361-369.
23. McKenna SLB, Dohoo IR. Using and interpreting diagnostic tests. Veterinary Clinics: Food Animal Practice. 2006;22(1):195-205.
24. Marterenchar A, Njanpop BM, Yaya A, Njoya A, Tulasne JJ. Problems associated with tuberculosis and brucellosis skin-test methods in northern Cameroon. Preventive Veterinary Medicine. 1993;15:221-229.
25. Nicoletti P. The epidemiology of bovine brucellosis. Adv Vet Sci Comp Med. 1980;24:69-98.
26.   Nicoletti P. The control of brucellosis- a veterinary responsibility. Saudi Medical Journal. 1992;13:10-13.
27. Nielsen K. Diagnosis of brucellosis by serology. Vet Microbiol. 2002;90(1-4):447-459.
28.   Bovine brucellosis. In: Terrestrial Manual: Chapter 2.4.3.  World Organisation for Animal Health. Paris:OIE; 2015.
29.   Pappas G, Papadimitriou P, Akritidis N, Christou L, Tsianos EV. The new global map of human brucellosis. Lancet Infect Dis. 2006;6(2):91-99.
30.   Pouillot R, Garin-Bastuji B, Gerbier G, Coche Y, Cau C, Dufour B, et al. The brucellin skin test as a tool to discriminate false positive serological reactions in bovine brucellosis. Vet Res. 1997;28(4):365-374.
31. Rojas APC. Brucella abortus RB51 vaccine: testing its spectrum of protective and curative characteristics. Blacksburg, Virginia. 2004.
32.   Saegerman C, Vo T-KO, De Waele L, Gilson D, Bastin A, Dubray G, et al. Diagnosis of bovine brucellosis by skin test: conditions for the test and evaluation of its performance.  Vet Rec. 1999;145(8):214-218.
33. Saegerman C, De Waele L, Gilson D, Godfroid J, Thiange P, Michel P, et al. Evaluation of three serum i-ELISAs using monoclonal antibodies and protein G as peroxidise conjugate for the diagnosis of bovine brucellosis. Vet Microbiol. 2004;100(1-2):91-105.
34. Sutherland SS. The anamnestic response to Brucella abortus-infected and vaccinated cattle. Vet Microbiol. 1983;8(4):405-409.
35. TDR diagnostic evaluation expert panel. Evaluation of diagnostic tests for infectious diseases: general principles. Nat Rev Microbiol. 2010;8(12 Suppl):S17-29.
36. Tittarelli M, Atzeni M, Calistri P, Di Giannatale E, Ferri N, Marchi E, et al. A diagnostic protocol to identify water buffaloes (Bubalus bubalis) vaccinated with Brucella abortus strain RB51 vaccine. Vet Ital. 2015;51(2):99-105. doi: 10.12834/VetIt.472.2296.3
37.   Viera AJ, Garret JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360-363.
38.      Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240(4857):1285-1293.