CHAT GPT

1. Context

2. Key Points 

• Experts in the AI community call this an epochal moment, as it could answer any question & carry on a conversation.
• Chat GPT was developed by Open AI, which is a research institute & company that focuses on developing AI technology in a responsible & safe way.
• It was founded in 2015 by a group of entrepreneurs & researchers, including Elon Musk, Sam Altman & Greg Brockman.

3. Language Models

• ChatGPT is much more than a chatbot.
• It can write a software program, a creative task like writing a story etc.
• It can explain scientific concepts & answer any question that needs factual answers.
• ChatGPT is called a Language Model, rather than a chatbot.

4. Neural network

• A neural network can be thought of as a large network of computers that can fine-tune its output of words based on the feedback given to it during the training stages which is known as Reinforcement Learning.
• ChatGPT is considered a neural network. The input data is typically a huge corpus of text.
• All the technologies are part of the artificial intelligence known as Machine Learning, which has been witnessing tremendous advancements.
• While one tries to understand how a language model works, word embedding needs to be focussed which represents words as a matrix of numbers that can be manipulated inside computers.
• When a neural network processes these numbers, it can differentiate words according to different contexts.
• For instance, when 'Shoot' appears with 'gun', the neural network knows that the words that will follow may mostly be 'bullets/victims', whereas when 'shoot' appears with 'camera', the neural network knows that the following words may be 'picture/ pixel'.
• With a refining technique called 'Transformer', a neural network can accurately 'understand' the context of a sentence or paragraph.

5. Evolution of ChatGPT

• ChatGPT follows a generation of languages that was released by OpenAI in 2018.
• OpenAI released the Generative Pre-Training (GPT) language, model.
• Generative means that it is a type of neural network that can create new content based on input content called Training Data.
• This technology makes it suitable for creative tasks like writing a new story.
• With the transformer technique, GPT was improved & 'Generative Pre-trained Transformer 2'(GPT-2) was released.
• GPT-3 with more sophisticated neural networks was launched in 2020.
• GPT-3.5 was released & ChatGPT is successor to GPT3.5. Each successive generation is more advanced than its predecessor.
• GPT-3 was trained with 175 billion parameters. These language models have looked at almost all text available on the internet & many text documents, thereby making them highly informed.
• ChatGPT is fine-tuned to provide conversational responses, against essay-type content, because the neural network behind it has been additionally trained on conversational transcripts with human feedback.
• There are a few other language models popular in the AI community like BERT-Bidirectional Encoder Representations from Transformers, from google.

6. Conversational AI

• The accuracy of ChatGPT or any language model can be measured using standard techniques.
• One such technique is "Recall-Oriented Understudy for Gisting Evaluation" or the ROUGE metric which compares ChatGPT's output of content against a standard expected content.
• For language models like GPT that are also used in translation, another metric called the BLEU metric(Bilingual Evaluation Under Study) is employed.
• This metric compares overlap in translated content with a standard translation.
• ChatGPT can become a powerful pedagogy tool on any topic because it can be instructed to "explain it to me like I am a 6-year-old."
• It may not be equally powerful in specialised contexts like if a doctor needs an automatic conversational assistant for medical queries, the neural networks behind ChatGPT need to be trained on specialised data.
• Considering ChatGPT can write programs, it should be possible to make it knowledgeable on any specialised topic eventually.

For Prelims & Mains