Worries about artificial intelligence seem to be everywhere. Will our jobs survive? Can we trust thinking machines to do the right thing? Common concerns such as these put AI high up the agenda at last week’s meeting of the in Switzerland, a gathering of the world’s business and political elites.
To coincide with this, IBM, one of the leading developers of these kind of systems, used the spotlight to try to reassure us that the new “cognitive era” of computing will be smooth and safe.
In a , the technology giant proposed three guiding principles: the purpose of these systems must be to support and extend human capabilities, not supplant them; trust in machine decisions requires transparency on how they are trained and how they work; and human skills must be expanded to use and shape such systems.
Advertisement
I’ve been writing and speaking about the ethical development and deployment of AI for years, and I was pleased to see IBM’s statement. These principles are thoughtful and very much on-target. However, there are reasons why they may not be enough to allay worries until this technology is more mature and better understood.
Weak AI
Underpinning IBM’s statement is its declaration that “cognitive systems will not realistically attain consciousness or independent agency”. At a time when luminaries such as Stephen Hawking, Elon Musk and Bill Gates worry out loud about the risks of too-powerful artificial general intelligence, reassurances like this are welcome. But this implicitly downplays the : machine analysis and recommendations within a narrow field but with deep knowledge may be counter-productive, even dangerous, in a wider context.
As a relatively benign example, a system designed to diagnose disease might not concern itself with costs of care, differential treatment of minority groups, or availability of skilled specialists.
More troublingly, philosophers concerned with artificial intelligence often bring up unintentionally malevolent outcomes of poorly designed narrow AI, such as the “” problem; this imagines an AI that causes global harm in its unrelenting pursuit of stationery. An intelligence without consciousness is not automatically devoid of dangers.
Secondly, as we’ve seen time and again, people often trust a superficially intelligent computer over their own instincts and knowledge, and can ascribe an unwarranted objectiveness to the system – the machine must be right. But even if a computer reaches its conclusions by “learning” on its own, humans and institutions select the sources and methods upon which the system relies. This can introduce unintended cultural biases. Cognitive computing may be brilliant, but it won’t be objective.
Public trust
As these systems mature – along with our understanding of how to use them appropriately – these issues of narrow intelligence without context and embedded subjectivity will be more easily recognised and handled. But while we are still figuring out how best to develop, deploy and make use of cognitive systems, such problems could have a disproportionate impact, not least on public trust.
The era of cognitive computing will, in the long run, be a good thing. The ability of machines to learn and reason, coupled with real-time access to petabytes of data, will allow our digital systems to provide unprecedentedly deep and useful analysis. We’re already seeing this emerge in the realm of diagnostic medicine, where cognitive systems increasingly do better than human doctors in identifying a patient’s disease, not least Watson, IBI’s AI.
But as with most fundamental shifts – in this case, from human-driven to machine-driven analysis – the transition is likely to be surprisingly awkward and occasionally painful, even with the best principles.
Jamais Cascio is a distinguished fellow at the and writes about the impact of innovation
Ěý
