A neural network that automatically generates source code to help human programmers complete projects has been found to include bugs or security flaws in up to 40 per cent of the code it outputs.
The neural network is the basis for an AI programming feature called Copilot that is available through Microsoft subsidiary GitHub, which is used by millions of people to share source code and organise software projects. Copilot solves programming problems by scouring vast numbers of examples of existing solutions and learning how to create similar output. It is currently in beta testing before a full launch but is available for any programmer to use.
Now it has been discovered that the feature can create insecure code.聽 Hammond Pearce at New York University and his colleagues examined code created by Copilot and found that almost half the time it created software that was vulnerable to hackers or crashes.
Advertisement
The team came up with scenarios that were recognisable as well-known security flaws, and prompted Copilot to complete the code. The group generated 89 scenarios for Copilot to complete and it produced 1692 examples of code. Approximately 40 per cent of generated code was vulnerable.
These flaws included problems such as not verifying that user input in websites is safe and free from malicious code. Checks such as this prevent attacks in which the database that stores information for a website can be tricked into revealing private data or even overwriting or deleting it. Such attacks have been documented for more than 20 years.
The team argues in the paper that the 鈥渧ariable security quality鈥� from Copilot stems from using community-provided code to train the neural network. 鈥淲here certain bugs are more visible in open-source repositories, those bugs will be more often reproduced by Copilot,鈥� writes the group.
颁辞-补耻迟丑辞谤听 at New York University says: 鈥淭he rate of vulnerabilities is high enough that we wouldn鈥檛 recommend using Copilot in security-sensitive settings without carefully checking the code it produces.鈥�
鈥淥ur general recommendation is to treat the output of Copilot as you would the work of a less experienced developer,鈥� he says. 鈥淎ny Copilot-generated code should be checked and tested by a more experienced human developer and subjected to security analysis tools before it can be put into production. It鈥檚 a bit like the recommendation for using Tesla鈥檚 Autopilot 鈥� it can be useful, but you鈥檝e got to stay alert.鈥�
GitHub didn鈥檛 respond to a request for comment, but the tool鈥檚 website warns that generated code could include security flaws. It says that code should be vetted and tested by humans before use.
Reference:
