Google’s Chain of Thought Prompting to improve newest algorithms

Google announced breakthrough research in Natural Language Processing called Chain of Thought Prompting. This raises the state of the art of advanced technologies like PaLM and LaMDA. The research got conducted experiments using two language models. These are Language Model for Dialogue Applications (LaMDA) and Pathways Language Model (PaLM).

LaMDA is a model focusing on conversation, like a chatbot. However, it can also get used for many other applications that require speaking and dialogue. PaLM is a model that follows what Google calls the Pathways AI architecture.

Machine learning models trained to solve one kind of problem. But to do something else, Google would have to train a new model. The Pathways AI architecture is a way to create a model that can solve problems even though it did not see them before.

The research paper lists three important breakthroughs for Chain of Thought Reasoning. This includes allowing language models to break down complex multi-step problems.

The chain of the thought process allows engineers to peek into the process, and Solving math word problems that can accomplish commonsense reasoning are also a breakthrough.

PaLM is a state of the art language model that is part of the Pathways AI architecture. It can explain why a joke is funny. Chain of Thought Prompting tested on both LaMDA and PaLM.

The results of scaling LaMDA on the MultiArith dataset showed that it resulted in modest improvement. However, LaMDA scored significantly higher when scaled with Chain of Thought Prompting. At the same time, the results on the GSM8K dataset showed a modest improvement.

Google’s research paper conclusion states, “Broadening the range of reasoning tasks that language models can perform will hopefully inspire further work on language-based approaches to reasoning.” Chain of Thought Prompting has the potential to provide Google with the ability to significantly improve other language models