Researchers at Bangladesh University of Engineering and Technology (BUET) and the University of California- Los Angeles (UCLA) recently developed a framework that can create Android applications from text descriptions. This new app was presented in a paper pre-published on arXiv. Masum Hasan, one of the researchers who carried out the study commented;
“Our research group at the Department of CSE, BUET works on bridging the gap between human language and programming language to make computer programming more accessible. We have previously worked on natural language-based program repair, language-based code search, code document generation, etc. In continuation, we wondered if fully functioning software could be built from natural language specification.”
Most existing models for creating software based on text descriptions are based on end-to-end neural machine translation (NMT) models, similar to the one underpinning Google Translate. These models utilize NMT frameworks to translate human language into source code.
While some NMT-based techniques for text to app translation have achieved promising results, most of them are unable to generate large programs with hundreds of lines of code. To overcome this limitation, Hasan and his colleagues created a new formal language that captures the complexity of app source code, representing it in a highly compact form. Hasan also said:
“We designed an intermediate formal language that can represent a user-specified app in a fraction of the tokens—as much as 20 times less. We then developed a compiler that can convert the compact representation into actual source code and trained a neural machine translation model to translate the natural language to this intermediate format, which can then be compiled into an app.”
Essentially, the researchers trained a translation model to generate representations in the intermediate language they created, rather than creating raw source code, as most existing text to app techniques would. By transforming text descriptions into this formal language, their framework allowed them to create fully-fledged mobile applications, which proved unfeasible using previously proposed methods. Hasan also added:
“Generating long sequences is a known limitation of current sequence generation models. Source code is usually much longer than natural language and the current best models still struggle to achieve 20% accuracy in generating short code snippets. This makes it impractical to create large software, such as mobile applications consisting of hundreds of lines of codes, with current sequence generation networks.”
The framework created by Hasan and his colleagues, dubbed Text2App, can create Android applications using instructions in English. To use the system, users simply need to describe a mobile application with a specific scope and the framework automatically creates it for them. Hasan said:
“Imagine you want to memorize your grandmother’s special recipe for pasta. Instead of remembering the entire recipe step by step, you can only learn the steps that are unique, and look up the rest of the steps from a cookbook. This will significantly reduce the amount of effort you would have to spend. Similarly, our intermediate app representation only captures the specific requirements provided by the user through the natural language instruction and the compiler fills in the rest.”
The recent work by Hasan and his colleagues could allow researchers to re-think program and app generation, by introducing a new way through which programs can be summarized or represented that is easier for AI technology to understand. Notably, the AI models trained by the researchers exhibited a far higher capacity than other techniques to convert natural language into apps. Hasan also noted:
“If we design programming languages or frameworks with AI in mind, our current AI technologies can perform remarkably better in code generation from human language. Additionally, programming languages are made to be understood by compilers, and not entirely by AIs. We demonstrate that relying on AI models alone for complex tasks like code generation is not adequate, and additional support from components like compilers can help a great deal.”
The most notable achievement of the study carried out by this team of researchers in the development of the intermediate formal language that enables the conversion of text into an app, as well as a compiler that ultimately creates the app. So far, Hasan and his colleagues explored a mere fraction of its capabilities, but they are now also asking the open-source community to contribute to its further evaluation and development.
