AI Roundup 06/06 -> 06/13/2024

Apple's long-awaited AI announcements, new text-to-video models to rival OpenAI's Sora, Meta's open agent solving complex tasks, and new benchmarks to reliably measure generative models' performance

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

---

Overview

• ✨ Special feature: Apple WWDC

• ✨ Highlights (5 entries)

• Language Models (13 entries)

• Autonomous Agents (3 entries)

• Vision (6 entries)

• Audio (2 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Special feature: Apple Worldwide Developers Conference (WWDC)

• Apple Intelligence - a suite of new AI features for iPhone, Mac, and other devices. This includes a more conversational Siri, custom AI-generated "Genmoji," and integration with OpenAI's GPT-4o. Despite being efficient for their size, Apple's on-device AI models perform below state-of-the-art systems like GPT-4. However, they excel in specific tasks using an adapter strategy, with on-device models outperforming larger models in summarizing and composing text.

• Enhanced Siri capabilities - Siri will gain new abilities such as managing notifications, writing and summarizing text, and carrying out actions across multiple apps. Users can interact with Siri through voice or typing, with many features functioning on-device for enhanced privacy.

• Genmoji and Image Playground - Apple is launching Genmoji to create emoji-like reactions on demand and Image Playground for AI-generated images. These features will be integrated into various apps, including Photos, which will have improved search and editing capabilities similar to Google's Magic Eraser.

• OpenAI integration - Siri will leverage ChatGPT, powered by GPT-4o, for complex requests, ensuring user permission before sending data. ChatGPT will be available across iOS, macOS, and iPadOS, supporting AI writing and image generation tools.

—> Apple WWDC 2024 keynote in 18 minutes

✨ Highlights

1. A Chinese company launches a Sora competitor called Kling - an AI model that generates realistic video clips from text prompts using advanced 3D AI techniques (VentureBeat)

   

2. Luma AI debuts Dream Machine - an OpenAI Sora-like tool enabling users to create realistic videos from text prompts in just two minutes (Company blog)

   

3. Stability AI releases the weights for Stable Diffusion 3 Medium - its latest and most advanced text-to-image model (Stability AI)

   

4. Alibaba releases Qwen2 - a commercially permissive suite of language models with up to 72B parameters and a 128 context window, excelling in multilingual understanding and logic, outperforming Llama 3 across benchmarks (Project website)

   

5. Researchers establish a comprehensive taxonomy and vocabulary for prompting techniques in generative AI systems, categorizing 58 text-only and 40 other modality techniques (Paper)

Language Models

1. Researchers introduce JailbreakHub - a framework for analyzing jailbreak prompts, revealing inadequacies in LLM safeguards and highlighting five highly effective prompts with a 95% attack success rate on ChatGPT

   

2. Meta Researchers develop the Comprehensive RAG Benchmark (CRAG) to address real-world QA deficiencies, revealing significant challenges in current RAG and LLM capabilities and guiding future research

   

3. OpenAI showcases a new technique to understand GPT-4, revealing millions of features and offering deeper insights into its internal representations

4. Anthropic shares more details on how it trained Claude 3 “personality” and instilled nuanced traits like curiosity and thoughtfulness

5. Stanford introduces TextGrad - a flexible and powerful framework that uses LLMs' textual feedback for automatic optimization, significantly enhancing performance across various AI-driven tasks

6. Researchers develop WildBench - an automated framework to benchmark LLMs using real-world queries, showing a strong correlation with human evaluations and offering reliable, interpretable performance assessments

   

7. Researchers introduce LiveBench - a benchmark with 18 diverse tasks to limit test set contamination and provide automatic, objective scoring

   

8. Together AI develops a Mixture-of-Agents (MoA) approach, leveraging multiple LLMs to surpass GPT-4 Omni in benchmarks like AlpacaEval 2.0, achieving state-of-the-art performance by combining outputs from multiple agents

   

9. IBM proposes a novel framework that uses engineered features and logistic regression to estimate confidence in LLM responses, significantly outperforming existing methods

   

10. Google develops Proofread - a Gboard feature using a server-side LLM for seamless sentence and paragraph corrections, achieving high-quality results through innovative tuning

    

11. Google presents a novel LLM designed to analyze personal health data and provide personalized recommendations by leveraging advanced multimodal reasoning

    

12. DeepMind presents Tx-LLM - an LLM that integrates biochemical knowledge, achieving competitive and state-of-the-art results across 66 drug discovery tasks

13. Harvard and MIT present findings that junior professionals may not effectively train senior colleagues on generative AI, recommending targeted training for all employees

Autonomous Agents

1. The University of Washington and Meta release Husky - a comprehensive open-source language agent that solves complex, multi-step reasoning tasks

   

2. DeepMind develops NATURAL PLAN - a challenging benchmark that uses real-world tool outputs to assess LLM planning capabilities, uncovering significant performance gaps and the limited effectiveness of current techniques

   

3. DeepMind introduces AndroidControl - a diverse dataset of 15k everyday Android app tasks, showing that fine-tuned models outperform baselines in-domain but struggle with out-of-domain high-level tasks

Vision

1. Researchers introduce the ShareGPT4Video series, enhancing video understanding and generation through dense and precise captions, featuring 40K annotated videos and a superior captioning model

   

2. Snap releases a groundbreaking weight quantization technique, reducing the size of the Stable Diffusion v1.5 model by 7.9x while enhancing image quality, demonstrating superior performance across benchmarks and human evaluations

   

3. Snap proposes a novel adversarial training approach to fine-tune pre-trained video diffusion models, enabling single-step video generation with significantly reduced computational costs and improved quality

   

4. Alibaba releases VideoLLaMA 2 - a set of video LLMs that enhance spatial-temporal and audio understanding, achieving competitive results in various video and audio tasks

   

5. ByteDance and TUM unveil TiTok - a transformer-based 1-Dimensional Tokenizer, which significantly enhances image tokenization efficiency and effectiveness, reducing image tokens dramatically while achieving competitive performance

   

6. Researchers develop MimicBrush - a generative model for image editing, allowing users to effortlessly incorporate references

Audio

1. Microsoft unveils VALL-E 2 - a novel neural codec language model that achieves human parity in zero-shot text-to-speech synthesis with enhanced stability and speed

2. Udio releases new features allowing users to upload audio files as song starters and create tracks up to 15 minutes long with advanced control options

Last week’s AI Tidbits roundup

AI Roundup 05/30 -> 06/06/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.