Introduction
The race to develop advanced AI tech, especially in Chatbots, is attracting a lot of players. One such instance is OpenAI’s ChatGPT. It was the talk of the town when it hit the chatbot scene in 2023. Driven by Generative AI tech, it is one of the most used tools for various tasks, ranging from composing emails to writing code.
Another highly performing chatbot came out in early 2023 – Claude, courtesy of Google and Anthropic AI, an Amazon-backed start-up. With over 350,000 waitlist sign-ups for Claude 2, the second version, it is one of the most booming chatbots to come out.
Fast-forward to 2024, Anthropic AI announced Claude 3 on March 3rd as its latest member of AI models expected to set a new industry benchmark on a wide range of cognitive problems.
In this read, we’ll discuss all about Claude 3, how it holds up against ChatGPT-4 and some limitations.
But before we proceed, let us familiarize ourselves with Anthropic AI and the different versions of Claude.
What is Anthropic AI?
Founded in 2021, Anthropic AI is an artificial intelligence safety and research startup based in San Francisco, California. The brand’s primary focus is working on and building LLMs, or large language models, and Chatbots that leverage them. One such Chatbot is Claude, a competitor to OpenAI’s ChatGPT. Brands such as Slack, Quora, and Notion have understood the potential of Claude 2 and have integrated it into their systems.
Anthropic takes a Constitutional AI approach to developing technologies, leveraging a framework that ensures the model reacts in a safer, honest, and helpful manner.
A Brief History of Anthropic AI
Anthropic AI is the brainchild of Dario and Daniela Amodei, former Vice President of Research and Vice President of Safety and Policy at OpenAI, respectively.
Initially, Anthropic AI took to funding in 2021, raising $124 million with investors like Jaan Tallin (co-founder of Skype).In May 2023, Anthropic AI received $450million from tech giant Google. Later that year, in September, Amazon pledged a whopping $4 billion with the intent that Amazon employees and cloud customers would gain early access from Anthropic AI, where the latter would rely solely on the former’s cloud services like training future models.
What is Claude AI?
Claude is an AI chatbot developed by Anthropic. It is designed to generate text-based content and engage in conversations much like we humans do. The first version was powered by LLM Claude 1.3 and was released in March 2023. Claude expanded its content width from 9,000 tokens to 100,000 tokens in May.
Shortly after, in July, a second version was introduced called Claude 2, which was a larger and more powerful LLM than its predecessor. Claude 2 is able to connect to vast amounts of data, in turn being able to forecast trends, compare or contrast documents, or even perform Q&A. Users can upload technical documentation, for instance, codebases or long literary works. Claude 2 can handle about 75,000 words. Then we’ve Claude 3, which sets a new industry standard of being able to process 200,000 words and reduce hallucinations much more than its previous versions.
Claude 2.0 vs Claude 2.1 Performance
What is Claude 3?
On March 4th, 2024, Anthropic AI introduced its newest member to the AI model family—Claude 3, which “sets new industry benchmarks across a wide range of cognitive tasks.”
Claude 3 is trained on vast amounts of text data through public web pages, including Wikipedia articles and books. Anthropic has leveraged reinforcement learning along with human feedback to better predict the next most likely word in its trail of responses.
Claude 3 is a family of three AI models, each having varying performance capabilities, allowing users to have the right balance of cost, speed, and intelligence. All models have impressive capabilities in the realm of content creation, code generation and even carry conversations in multiple languages.
Claude Haiku
Described as “light and fast”, Haiku is the most compact and fastest of the Claude family. It is very suitable for tasks that require speed but are also cost-effective. As expected, it is the least performing among the three available Claude 3 models.
Claude Sonnet
Described as “hard-working,” Sonnet is the middle-of-the-road model among the three. It offers strong performance in cognitive problems and better processing time compared to Opus. It aims to strike a balance between speed and performance. It is more inclined towards serving enterprise tasks like data processing, quality control, and product recommendations.
Claude Opus
Described as “powerful”, Opus is the most intelligent model of the three, outperforming Sonnet and Haiku on many evaluation benchmarks for AI systems. It outperforms competitor models like basic mathematics or graduate-level expert reasoning.
All three models are tested and proved to be faster and more intelligent than their predecessors. According to Anthropic, “Opus and Sonnet are now available to use in claude.ai and the Claude API, which is now generally available in 159 countries. Haiku will be available soon.”
Where does Claude 3 stand with its peers?
Claude Opus model outperforms other AI systems on most common evaluation benchmarks like mathematics, reasoning, and testing expert knowledge. It also showcases human-like comprehension and fluency in complex problems. One key feature that sets Claude apart is its adoption of Constitutional AI, unlike its peers that look solely at performance. The 3 Claude models are designed to monitor risks and analyze ethical considerations. When compared with other models like GPT-3.5, GPT-4, Gemini Ultra, or Gemini Pro, Claude 3 models possess impressive capabilities across various tasks and problems like code generation, content creation, or even proficiency in non-English languages like French, Spanish, and Japanese.
The Claude models can not only be multilingual but also have vision capabilities. Since they are trained on a diverse range of visual data, the models can effectively analyze and infer information from various visual contents. For instance, it can interpret visual data as showcased in tasks like AI2D science diagrams and achieve high accuracy rates for zero-shot and few-shot settings.
A Claude 3 benchmark chart of multimodal vision capabilities provided by Anthropic
Now, let us get to the big picture!
Claude 3 vs ChatGPT-4
Claude 3 or Claude Opus, to be precise, and GPT-4 were run on 10 AI benchmarks like HumanEval (coding), MMLU (undergraduate level knowledge) and GSM8K (grade school mathematics). As seen in the table, Claude 3 narrowly beats its competitors, including GPT-4. For instance, Opus beats GPT-4 in GSM8K by 3% and HumanEval by 17.9%. The numbers speak for themselves as Claude beats GPT-4 across the table on all 10 benchmarks.
10 AI Benchmarks
Another aspect to note is the word count. As discussed earlier, Claude 3 can process as many as 200,000 words as opposed to GPT-4’s 64,000-word limit. This means Claude has a larger “context window” wherein it can remember a lot more of the conversations and can generate longer texts. This is a huge plus for enterprise tasks that deal with huge datasets and documents.
Another issue that looms with chatbots like GPT-4 is privacy concerns. It is known to most GPT users that the conversations users have with the bot are saved to train the models to make more accurate outcomes. But this can be of concern, especially for sensitive information. Claude on the contrary, does not use conversations to train models and deletes prompts and outputs on its backend within 90 days.
Limitations of Claude 3
According to early access users, Claude 3 performs well for instructional tasks or answering factual questions. But it sometimes struggles with mathematical and complex reasoning problems. There are also instances of biased responses.
Claude 3 is also currently the most expensive model on the market, but there are discussions of having more affordable plans in the near future.
Conclusion
Researchers at Anthropic AI aim to make Generative AI more stable and helpful. With the power of Constitutional AI, they not only intend to improve the responses but also make it easier to assimilate for the users. Much like humans who can comprehend the right responses, Claude 3 is expected to mimic human responses.
Go check it out for yourselves! Head on over to www.claude.ai and begin your conversations for Free! The prompts per day will be limited, so if you wish to use many more prompts, you can take the Claude Pro plan, which is offered at $20 per month.
Stay tuned for more awesome reads. See you guys in the next one!
