- Facebook develops an AI tool that performs automated end-to-end testing and repair in a large-scale codebase.
- It lets programmer spend less time on debugging code and more on focusing what’s next.
- The tool is still in development phase and intended for open source release.
Programmers like to write code, not debug it. Sadly, it’s an inseparable part of coding and could be a tedious and time-consuming process. Although there are several debugging techniques available, it’s quite hard to map out an exact method for debugging as different scenarios throw different sets of challenges.
Now engineers at Facebook have designed a new artificial intelligence (AI) hybrid tool that can decrease the amount of time programmers spend on debugging, while diverting their focus on rolling out new software. They have named this tool SapFix.
SapFix is capable of generating fixes for certain types of bugs, and before deploying the changes to production, it presents them to developers for approval. Facebook has already used this tool to boost the speed of shipping stable, robust code updates to millions of users using their Android app.
According to the Facebook, this is the first time someone has used an AI-based debugging tool in production at such large scale. They plan to make SapFix an open source tool to increase the production as well as the stability of new software for various research organizations and companies.
How SapFix Works?
Although SapFix fixes bugs detected by Sapienz (an intelligent automated software testing tool developed by Facebook to efficiently design many of the test cases), it can work as an independent tool.
The process of testing and debugging begins with Facebook’s static analysis tool and Sapienz, which pinpoints the particular portion(s) of the code that needs fixing. Then SapFix automatically selects a debugging approach from numerous methods to create a patch of that particular portion of the code.
To handle high volumes of bugs, it generates patches that partially or completely revert the code submission. In case of complicated crashes, SapFix creates patches by picking from its ‘template-fix’ collection. These templates contain an array of past fixes that were previously developed and applied by human engineers.
When these templates do not work, the system performs a mutation-based fix by applying minor changes to the abstract syntax tree of the statement that is causing crash. It continuously modifies the patch until it detects a liable solution.
SapFix creates multiple fixes for one bug and checks their quality against the following 3 factors:
- Is there any compilation error?
- Does the crash persist?
- Does modification create new crashes?
SapFix handles the last two queries by executing human-written tests on the patched builds and Sapienz-generated tests. The large codebase remains isolated in the overall validation process.
SapFix isn’t developed to apply fixes to production software on its own, but to replicate the debugging task that programmers currently do. Thus, once the patches are completely tested, they are sent for human approval. In short, SapFix generates fixes, selects the best out of multiple options and presents recommendations to human developers.
While Facebook is currently working on how Sapix can autonomously head off crashes before they occur in different kinds of bugs, the long-term application could include creating faster, accurate and more responsive version of this tool that can operate at both large and small scales.