The workings of ChatGPT, the latest natural language processing tool
GS-3 | Science and Technology
News: The latest artificial intelligence tool is creating a buzz among computer scientists and programmers due to its creative capabilities. The tool has been developed by OpenAI, a research institute founded in 2015
A tool called ChatGPT was released on the internet. It created quite a stir especially among the artificial intelligence (AI) crowd because this tool ‘knew’ every topic under the sun; it could answer questions and carry on a conversation.
v Experts in the AI community call this an epochal moment, stressing how powerful ChatGPT is.
v This tool interacts with humans in natural language and is impressive because aside from answering general queries, it has many other functions.
v ChatGPT has been developed by OpenAI, which is a research institute and company that focuses on developing artificial intelligence technology in a responsible and safe way. It was founded in 2015 by a group of entrepreneurs and researchers, including Elon Musk, Sam Altman, and Greg Brockman.
Language models
v ChatGPT is much more than a chat bot. For example, you can ask it to write a program or even a simple software application.
v It can also do creative tasks such as writing a story. It can explain scientific concepts and answer any question that needs factual answers.
v ChatGPT is what is called a Language Model, rather than a chat bot. A language model is a software that prints out a sequence of words as output that are related to some words given as input with appropriate semantic relation; in practical terms, it means that it can perform tasks like answering questions and carrying on a conversation with humans.
v It is often used in natural language processing (NLP) applications, such as speech recognition, automatic translation, and text generation.
v It is also a neural network. A neural network can be thought of as a large network of computers that can fine tune its output of words based on the feedback given to it during stages of training: this training process and the technology together are called Reinforcement Learning. The input data is typically huge corpus of text. All these technologies are part of the artificial intelligence (also called Machine Learning) that has been witnessing tremendous advancements.
The development of ChatGPT
v ChatGPT follows a generation of language models that were released by OpenAI from 2018. In 2018, OpenAI released the Generative Pre-Training (GPT) language model.
v Generative means that it is a type of neural network that can create new content based on input content, called Training Data. This technology makes it suitable for creative tasks like writing a new story.
v In early 2022, GPT3.5 was released and ChatGPT is successor to GPT3.5. Each successive generation is more advanced than its predecessor. For example, GPT-3 was trained with 175 billion parameters.
v ChatGPT is fine-tuned to provide conversational responses, as against essay-type content, because the neural network behind it has been additionally trained on conversational transcripts with human feedback.
There are a few other language models that are popular in the AI community, like BERT (Bidirectional Encoder Representations from Transformers) from Google. However, ChatGPT seems to be the most powerful for conversational purposes.
The conversational AI
v The accuracy of ChatGPT or any language model can be measured using standard techniques. One such technique is “Recall-Oriented Understudy for Gisting Evaluation” or the ROUGE metric which compares ChatGPT’s output of content against a standard expected content and measures the overlap as success percentage.
v For language models like GPT that are also used in translation, another metric called the BLEU metric (Bilingual Evaluation Under Study) is employed; this metric compares overlap in translated content with a standard translation.
v ChatGPT can become a powerful pedagogy tool on any topic to anyone, because we can instruct it to “explain it to me like I am a six year old”. It can explain in simple terms anything from philosophy to cooking recipes, including new recipes of its own. If you are in mood for some fun, you can ask ChatGPT to narrate a new story to you!
Is ChatGPT the most powerful NLP tool?
v For conversational purposes, the answer is yes. However, it may not be equally powerful in specialised contexts.
v For example, if a doctor needs an automatic conversational assistant for medical queries, the neural networks behind ChatGPT need to have been trained on specialised data. Considering that ChatGPT can write programs, it should be possible to make it knowledgeable on any specialised topic eventually. For general purposes, ChatGPT can be considered the most powerful for now.
Source: TH
Team Manvendra IAS