Combating fraud with fiction

To combat fraud, authorities and insurance companies in the UK invested public funds in voice-based lie-detectors to discourage the public from entering false claims[i]. The deterrent effect was significant and there were substantial savings as the number of accepted claims dropped dramatically[ii]. Yet a pertinent ethical problem is that these “lie-detectors” are based on Nemesysco’s[iii] LVA-technology and there is no scientific assessment of the technology that suggests it works. The technology does not extract relevant information from the speech signal but even if it would perform correct phonetic analysis the relevance of such measurements to correctly assess the speaker’s emotional state still is at issue. Indeed the LVA-technology claims to assess the speaker’s state of mind exploring the minute traces that it leaves in speech waveform but an analysis of the LVA patent[iv] indicates that the technology falls miserably short of its suggestion of high precision analysis. All it does is counting local maxima and minima (“thorns”) within a three-samples’ running window, and a simple statistic over “plateaus” in the waveform, performed on a crudely digitized (11.025 kHz, 8-bit/sample) speech signal. In fact, the crudeness of the amplitude coding is even worse than 8-bit/sample because the signal is further “filtered” in yet another quantization step that ends up representing the amplitudes in only 85 levels (as compared to the already poor 256 levels of the 8-bit representations). There is no rationale for why counting just such thorns and plateaus, where the amplitude and time information is lost, would be meaningful and no logic principle is provided for why the subsequent operations and the thresholds involved in them would possibly lead to any valid estimate of the speaker’s mental state. The system’s estimate of the speaker’s mental state is difficult to predict because it is based on the unstable thorns and plateaus in an acoustic wave which are influenced by room acoustics, noise or anything that changes the number of thorns and plateaus. Under optimal circumstances these measures would indeed describe gross average characteristics of the speech wave but they are so crude that for any given count of thorns and plateaus there is a vast family of curves that would be interpreted by the LVA-technology as being exactly the same, although many of them would not even resemble a speech signal. This is an immediate consequence of the low information content of the analysis. It simply cannot distinguish the signals from each other. This is probably also the reason why the LVA-systems are perceived as being robust. Since they rely on highly noisy and crude measures, it is difficult to distinguish anything at all, so changes in the background noise or other spurious acoustic accidents go simply unnoticed. It is like trying to seeing the world through greasy glasses. For both the speaker and the tester this erratic behavior may easily give the impression that the system picked up something “deep” that not even the speaker knows about. Finally, “certified” personal issues the final interpretation of these “complex” instrumental results. That does not make things better. Unless the certified personal has some independent basis for the judgment, analyzing a non-valid output is simply irrelevant.

Why were not relevant questions asked from the beginning?

An intriguing aspect of all this is how come such a “technology” could be adopted by some of the British authorities. Why were not the highly competent speech scientists in the UK asked to look at this amazing technology? It would have been enough to ask one of my first grade students of Phonetics, I believe. Did not the responsible authorities suspect that Nemesysco’s promises were “too good to be true”? Is it a calculated risk of using the technology’s deterrent effect, as long as it’s lack of basis is not denounced to the public? I find it hard to believe that authorities would engage in undermining the public’s respect by engaging in such practices and it simply doesn’t make sense for me. Isn’t it predictable that someone would eventually point out the hoax? Did anyone believe that it would be possible to choke scientists’ freedom of speech or were there a hope that no one would even bother to address the issue?

I have no answer to these questions but one thing is for sure: Nemesysco’s sellers must be extremely convincing and well organized to have succeeded in this way. But of course there must also be a wide range of people willing to listen and being convinced by their arguments. From the buyer’s short-term perspective it may be easy to think that, as long as we fool people that are naïve enough to believe in this hoax, no harm is done, but isn’t it obvious that this will backfire and has the potential of eventually affect even those who are not fooled into the false belief?

Ungrounded scientific claims cannot be left unchallenged

Unfortunately scientists may have contributed to the “success” of the LVA-technology by understandably refusing to study its arbitrary principles, thereby leaving the public scene to be taken by Nemesysco’s propaganda. A pedagogic scientific effort may be necessary to explain for the public why the LVA-technology cannot work. Now Nemesysco’s resources have grown so large that they could even force the withdrawal of a peer-reviewed paper questioning its technology, rather than engaging in the scientific debate[v]. Nemesysco’s official excuse was that the paper is defamatory because we use the word “charlatanry” but I believe this is wrong. The word is used in a general sense and to be a charlatan the person is supposed to know that the product actually does not work, but we do not imply that the inventor actually knows that. On the contrary, we rely on a published interview with the inventor where he says he has no formal academic competence in speech processing and we draw the conclusion that he may indeed not have been aware of the lack of scientific basis of the method he proposes. To be unaware of LVA-technology’s fundamental problems is okay (perhaps naïve) before the publication of our paper. Perhaps a more plausible reason for Nemesysco’s action was that our paper was damaging their business, as also stated in their lawyer’s first letter. Indeed, rather than discussing percentages of correct responses (which any random system obviously generates) our paper addressed the validity of the method, not its reliability, and the only way of discussing validity is to argue convincingly in support of the technology’s fundamental principles. I believe there are none, so the next “best” option was to shoot the messenger so that the news does not spread…

The validity of LVA-technology remains unproven

So now it is known, unless someone proves that we are wrong, that LVA-technology does not live up to the claims of detecting a speaker’s emotional state using samples of her/his speech. The issue is of no scientific interest and not even in my main field of research but I happened to be curious about the principles of the LVA-technology and I do have the necessary background to address
the question, from both the speech-processing and the phonetic perspectives. As researcher, being paid by public funds, I also have the responsibility of denouncing that the emperor is naked. However, having said that, I have no illusions that my shouting will last long enough to prevent similar cases in the future. Even if for the moment such LVA-based devices may be removed from official uses, it is likely that the human fascination for “fantastic machines” along with the company’s effective propaganda and possible short-term benefits will soon override my efforts to inform the public. That is worrying but just a part of reality that I have to live with. These devices are not cheap and it will take much courage for the people who invested in them to recognize that they just wasted their money.

Recovering from a mistake

Meanwhile I would say that it is urgent that authorities, who publicly have praised these devices, take prompt and courageous action to admit that the investment was a mistake. When it becomes known that the LVA-technology does not produce relevant results the public’s confidence on the authorities will be deeply damaged. Professionally conducted structured interviews of (randomly) selected customers will do a far better and responsible job than Nemesysco “lie-detectors”. LVA-technology may be acceptable for entertainment, but not for serious applications influencing people’s lives.

Related links and references:
Science magazine: http://sciencenow.sciencemag.org/cgi/content/full/2009/210/1
Debate article (Swedish) :
http://www.newsmill.se/artikel/2009/02/03/professor-avsloja-bluffen-om-logndetektorerna
‘Ministry of Truth’ – www.ministryoftruth.me.uk