Hugging Face's New LLM Leaderboard Sets Higher Bar for Language Models

Happy Friday, Team!

June 28 is known for the signing of the Treaty of Versailles in 1919, ending World War I. As we reflect on historical milestones, today's focus shifts to the future of technology. Hugging Face has launched a new LLM leaderboard, setting a higher standard for evaluating AI performance. Let's explore the latest breakthroughs in AI shaping tomorrow's world.

TRACK HIGHLIGHTS
Article Takeaways

• Hugging Face's New LLM
• Google Translate's AI-Powered Leap: 110 New Languages Added
• Goldman Sachs: AI Bubble Warnings
• AI Michaels for Personalized Olympic Recaps
• Character.AI Launches AI Avatar Calls

MARKET PITSTOP
A Quick Look at the Market

Stock Market: S&P 500 closes little changed as investors brace for Fed’s preferred inflation gauge. Read More

Crypto: Bitcoin (BTC) options worth $6.68 billion and Ethereum (ETH) options worth $3.5 billion are slated to expire on Deribit, the leading crypto derivatives exchange, this Friday. Read More

LAP 1 Featured Story
Hugging Face's New LLM Leaderboard Sets Higher Bar for Language Models

• Hugging Face launches a new LLM leaderboard.
• Alibaba's Qwen models secure top positions.
• Tests cover knowledge, reasoning, math, and instruction.
• Closed-source models excluded for reproducibility.

Hugging Face has released its second LLM leaderboard, aiming to establish a more challenging standard for evaluating large language models across diverse tasks. Alibaba's Qwen models dominate the inaugural rankings, securing three of the top ten spots. The new leaderboard tests models on knowledge, reasoning, complex math, and instruction following, using benchmarks such as solving murder mysteries and explaining PhD-level concepts in simple terms.

Notably, closed-source models like ChatGPT are excluded to ensure reproducibility, and the tests are conducted on Hugging Face's own computers powered by 300 Nvidia H100 GPUs. This initiative seeks to address the issue of over-training on previous benchmarks, which has led to a decline in real-world performance for some models. Read More

LAP 2
Google Translate's AI-Powered Leap: 110 New Languages Added

• Google Translate adds 110 new languages using AI
• PaLM 2 enhances efficiency by learning related languages
• 614 million speakers now have support in Translate

Google has expanded Google Translate to include 110 new languages, reaching 614 million speakers worldwide. This expansion is facilitated by their large language model, PaLM 2, which efficiently learns languages that are closely related to each other. Notably, around a quarter of the newly added languages come from Africa. This addition follows the Zero-Shot Machine Translation update from May 2022 and aligns with Google's 1,000 Languages Initiative. Google aims to continually enhance language support through advanced technology and collaborations with linguists and native speakers. Read More

BRAIN TEASER
In 1990, a person is 15 years old. In 1995, that same person is 10 years old. How can this be?

WALLSTREET PITSTOP
Goldman Sachs: AI Bubble Warnings

• Goldman Sachs warns of AI bubble, but supports Nvidia and AI infrastructure investments
• Analyst Jim Covello highlights skepticism about AI's ability to solve critical business problems
• Nvidia stock surges, driven by AI demand from cloud computing giants and internet companies

Goldman Sachs' top analyst Jim Covello cautions about a potential AI bubble but sees value in investing in Nvidia and AI infrastructure companies. Despite concerns about AI's ability to solve business problems and the high investment costs, Covello believes AI infrastructure providers remain a good investment. Nvidia, a key player in AI, has seen substantial stock growth driven by demand from cloud computing and internet companies. However, Covello warns that investor sentiment could shift if significant AI applications don't emerge soon. Read More

Lap 3
Al Michaels for Personalized Olympic Recaps

• NBC to use AI to replicate Al Michaels' voice for Olympic recaps
• Personalized recaps on Peacock will feature "high-quality" AI narration
• Nearly 7 million personalized recap variants available during the Olympics

NBC will use artificial intelligence to replicate Al Michaels' voice for the 2024 Paris Olympics, providing personalized daily recaps on Peacock. This innovative feature will offer nearly seven million possible recap variants, tailored to users' interests and narrated by a "high-quality" AI version of Michaels' voice. Initially skeptical, Michaels was impressed by the AI's accuracy, describing it as almost perfect. This new feature will be available on the Peacock app starting July 27, one day after the opening ceremonies. Read More

FUN FACT STOP
Dead skin cells are a main ingredient in household dust

LAP 4
Character.AI Launches AI Avatar Calls

• Character.AI introduces AI avatar calls in multiple languages
• Users can make calls for language practice, mock interviews, and role-playing games
• Seamless switching between calling and texting with AI characters, featuring reduced latency and interruption option

Character.AI has unveiled a new feature allowing users to engage in calls with AI avatars, supporting languages such as English, Spanish, Portuguese, Russian, Korean, Japanese, and Chinese. This capability, tested with over 3 million users and 20 million calls prior to its public launch, facilitates language practice, mock interviews, and integrates with role-playing games. Users can effortlessly transition between calling and texting, benefiting from reduced latency and the ability to interrupt AI responses with a simple tap. Read More

FINAL LAP - FINANCE & THE STOCK MARKET
China Caps Finance Pay, Amazon Embraces AI, U.S. Stocks Dominate, Top City for Finance Workers Revealed

• China’s Finance Elite Face $400,000 Pay Cap, Bonus Clawbacks

• Amazon Leans Into Generative AI to Manage Its Finances

• American stocks are consuming global markets

• The Best U.S. City for Finance Workers to Live In

BRAIN TEASER ANSWER

The person was born in 2005 BC.

THE RACE CONCLUDES
Thank You for Joining us on the AI Race! 🏁 

We want to THANK YOU for joining us on this exciting journey into the world of AI! Our mission is simple - to keep those who feel left behind in the AI curve up to date.

Your subscription marks the beginning of a collaborative exploration into the boundless possibilities of Artificial Intelligence.

Warm regards,

AI Pitstop Team