Introducing a new approach known as natural language embedded programs (NLEPs) has been proven to enhance the numerical and symbolic reasoning abilities of large language models (LLMs). This technique prompts LLMs to create and execute Python programs to solve user queries, providing solutions in natural language.
Despite the impressive performance of LLMs like ChatGPT across various tasks, they often struggle with problems that require numerical or symbolic reasoning.
NLEPs follow a four-step problem-solving template: calling necessary packages, importing natural language representations of essential knowledge, implementing a solution-calculating function, and presenting results in natural language with optional data visualization.
Several advantages come with this approach, including improved accuracy, transparency, and efficiency. Users can examine generated programs and correct errors directly, eliminating the need to rerun entire models for troubleshooting. Furthermore, a single NLEP can be reused for multiple tasks by replacing specific variables.
Research has shown that NLEPs enabled GPT-4 to achieve over 90% accuracy in various symbolic reasoning tasks, surpassing task-specific prompting methods by 30%.
Besides accuracy enhancements, NLEPs have the potential to enhance data privacy by running programs locally, removing the necessity to send sensitive user data to external companies for processing. This technique could also enhance the performance of smaller language models without requiring expensive retraining.
However, the effectiveness of NLEPs depends on a model’s program generation capability and may not be as effective with smaller models trained on limited datasets. Future research will explore ways to make smaller LLMs generate more efficient NLEPs and examine the impact of prompt variations on reasoning robustness.
This research, partially supported by the Center for Perceptual and Interactive Intelligence of Hong Kong, will be presented at the Annual Conference of the North American Chapter of the Association for Computational Linguistics later this month.
(Photo by Alex Azabache)
See also: Apple is reportedly getting free ChatGPT access
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo happening in Amsterdam, California, and London. This comprehensive event is co-located with other leading events like Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
