Martina Scharmach’s research while affiliated with Freie Universität Berlin and other places

The total detection reliability of a mine searching system is analogous to NDE-systems governed by three elements; � intrinsic capability- which describes the basic physical-technical capability of the method � application factors- including those due to environment � human factor- the effect of human operators on the detection reliability. Some of these can be determined in simple laboratory measurements in which the effect on detection capability of individual parameters is measured. However, the human factor and some aspects of the effects of environmental conditions on the system need to be treated statistically. By far the most common "mine searching system " in use today is the metal detector. The test and evaluation procedures for metal detectors described in CEN CWA 14747: 2003 include the above ideas. This is why, in addition to parameter tests, they include detection reliability or blind field tests under local conditions with local personnel. A series of three field trials was performed in the ITEP-project 2.1.1.2 in 2003 “Reliability

The performance of mine detecting instruments is embedded in the behavior of a complex system. The total reliability is always composed of the intrinsic physical detection capability of the sensor, application/environmental influences and human factors. The intrinsic capability and some application factors can be investigated in laboratory measurements. Human factors, other application factors and the overall reliability, can only be evaluated in blind field trials in which the probability of detection (PoD) and false alarm rate (FAR) are measured statistically. Both of these approaches are included in CEN Workshop Agreement CWA 14747:2003, which standardizes detector testing in Humanitarian Demining. We report here the results of a study to investigate how to optimize such testing. For efficient and statistically valid field trials, the number, types and burial depths of targets, and the number of test lanes, soil types, repetitions and operators need to be carefully chosen. Laboratory results should be used to help construct field trial protocols and also to help distinguish the different contributions to the PoD and FAR, to determine where to improve insufficient performance. In this study, four models of metal detector were tested in three field trials and in the laboratory. The repeatability of the field trials is assessed, taking into account operator training and experience. Results of the laboratory tests are compared with results of the field trials and used to construct a "modular model" of the system, as used in nondestructive testing. The conclusions are, in principle, applicable to trials of other types of sensor.

The total reliability of a mine searching system is driven by the triple of intrinsic capability, which describes the physical-technical basis capability, the application and environmental factors and the human factors. Some of them can be determined in laboratory measurements but the human factor and a part of environment conditions and their interaction with the device need to be treated statistically. That is why the test & evaluation procedure described in CEN CWA 14747:2003, includes in addition to parameter tests of metal detectors also the reliability or blind field tests under local conditions with local personnel. A series of three big field trials had been accomplished in the ITEP-project 2.1.1.2 "Reliability Model for Test & Evaluation of Metal Detectors" to specify the optimum conditions for reliable trial results with affordable effort. For each set of specific working conditions, characterized in terms of a combination of one mine type in one soil with one detector handled by local personnel, the searching system will show up a working performance in mine detection rates as a function of mine depth and show up a certain overall false call rate. During the ITEP-trials in Benkovac and Oberjettenberg the authors learnt to determine this function separately for each mine type in each soil, which is especially important for low metal mines in uncooperative soil which will be illustrated for the mine PMA2 in different types of soil. The question of representativeness for field conditions on the one hand and necessary statistical set up for the possibility to distinguish between individual detector performances is still under discussion. Introduction and Background

The detection of defects in aircraft components or parts of a power plant or mines in the ground is connected with the task to distinguish between signals caused be the item searched for and noisy signals from other sources. The reliability of the system is the better the better the system is in detecting the wanted signal and avoiding false alarms caused by noise. It is usual to measure the reliability of diagnostic systems in terms of ROC-curves (Re-ceiver Operating Characteristics) and POD-curves (Probability of Detec-tion). The ROC curves describe the POD versus Probability of False Alarm with growing system sensitivity along the curve. POD curves describe the Probability of Detection for a fixed sensitivity for a variation of item sizes or depths. Reliability of the whole system is always a composition of the physics and technique, application influences and the human factors. For the scien-tific understanding and possibilities of improvement, decomposition accord-ing to a modular model is helpful. The authors present their experiences with the reliability evaluation of metal detectors for humanitarian demining and an NDT system of mechanized ultrasonic phased array testing of copper welds. The special concern in demining is the influence of soil types, detec-tors and the experiences of human beings.

Because the demining process is always connected with a danger for human beings, it is necessary to make sure the proper functioning of mine searching equipment in detecting mines. Detecting a hidden target by penetrating physical interaction of rays or waves is similar to non-destructive testing. The non-destructive testing profession developed procedures to check the reliability of testing. These procedures, like the performance demonstration where the successful detections are statistically evaluated against false call rates and their implementation in an industrial standard (ASME section XI appendix VIII) are used as a template. A first adoption to demining was accomplished in the prescription for blind trials in the CEN workshop agreement CWA 14747: 2003 for test and evaluation of metal detectors. Within an ITEP project a number of blind trials were carried out to learn about the necessary statistical basis of those trials to achieve true, reproducible and repeatable results to give guidance for the selection and improvement of metal detectors. The special focus in this first investigation was on the influence of the human factor due to the different degree of experience of the operators and the influence of the soil.

For safe and reliable demining it is necessary to determine the actual true performance of mine searching equipment in detecting mines. The subject to detect a hidden subject by penetrating physical interaction with the target is similar to that of non-destructive testing where it is looked for hidden cracks etc. in material via waves and rays. The non destructive testing profession is now about 100 years old and developed some procedures to check reliability of testing. Those principles like the performance demonstration where the successful detections are statistically evaluated against false calls rates and their implementation in an industrial standard (ASME section XI appendix VIII) are analysed. A first adoption to demining was accomplished in the prescription for blind trials in the CEN workshop agreement CEN BT 126 CWA07 for test and evaluation of metal detectors. A number of blind trials were accomplished within an ITEP project to learn about the necessary statistical layout of those trials to achieve true, reproducible and repeatable results to give guidance to selection and improvement of metal detectors. The special focus in these investigations was on the influence of the human factor due to the degree of experience of the operators and the infulence of uncooperative soil. The correlation of the results of the physical parameter measurement and the statistical results is analysed in a first attempt.