AI Roundup 07/18 -> 07/25/2024

Meta’s open Llama 3.1 models rivaling GPT-4o and Claude Sonnet, OpenAI's cost-effective GPT-4o mini, Mistral's new powerful models, Kling's text-to-video open to all, and automated prompt optimization

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

📩 Published a new breakthrough paper? Just released an open-source package? Submit it here to ensure we don’t miss it and that it gets featured in next week’s post.

---

Overview

• ✨ Highlights (8 entries)

• Language Models (10 entries)

• Multimodal (2 entries)

• Autonomous Agents (2 entries)

• Vision (8 entries)

• AI Tools (2 entries)

• Open-source Packages (3 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. Meta unveils Llama 3.1 - an open suite of three language models (7B, 80B, 405B), with the largest one outperforming GPT-4o and Claude Sonnet on multiple benchmarks, along with hosted inference on Azure, AWS, and IBM (Meta)

   

2. OpenAI releases GPT-4o mini - a cost-effective multimodal model, offering superior performance at 60% less cost than GPT-3.5 Turbo (OpenAI)

   

3. Mistral releases Mistral Large 2, boasting a 128k context window and superior performance in multiple languages and coding tasks (Mistral)

   

4. Mistral AI releases Mistral NeMo - a powerful 12B open language model developed with Nvidia, featuring a 128k context window and outperforming competitors in reasoning and coding tasks (Mistral)

   

5. Researchers publish OpenDevin - a platform for developing AI agents that interact like human developers by writing code, using the command line, and browsing the web (Paper)

6. Stability AI releases the technical report of its recent Stable Audio model - a text-to-audio model that generates high-quality stereo audio at 44.1kHz from text prompts (Stability AI)

7. Kling, the Chinese text-to-video Sora competitor, is now generally available

   

8. Eleven Labs presents Turbo 2.5, which now supports Vietnamese, Hungarian, and Norwegian and speeds up English processing by 25% (Eleven Labs)

   

Language Models

1. Nvidia develops ChatQA 2 - a Llama3-based model with a 128K context window and improved RAG capabilities, matching GPT-4-Turbo's long-context performance and exceeding it in RAG tasks

   

2. Apple introduces LazyLLM - an innovative method that dynamically prunes tokens to accelerate language model inference, achieving a 2.34x speedup in the prefilling stage while preserving model accuracy

   

3. Peking and Baichuan present PAS - an LLM-based automatic prompt engineering system that achieves state-of-the-art results with an average improvement of 6%

   

4. Researchers demonstrate that optimizing vocabulary size in LLMs leads to significant performance improvements, proposing methods that increased ARC-Challenge scores by 10% with the same compute budget

   

5. The University of Washington and AI2 Researchers introduce MassiveDS - a 1.4 trillion-token datastore that enhances retrieval-based language model performance, demonstrating that larger datastores significantly improve task performance

   

6. Researchers develop a taxonomy and evaluation suite for broader noncompliance in language models, finding high inappropriate compliance rates and proposing effective training methods to balance noncompliance and performance

   

7. UNC Chapel Hill introduces the System-1.x Planner - a hybrid planning framework that combines quick intuition and detailed search for better LLMs performance on complex tasks

   

8. Salesforce publishes a comprehensive survey categorizing and explaining methods to align LLMs with human expectations (RLHF, DPO, etc.)

9. Eleuther AI introduces GoldFinch - a hybrid Linear Attention/Transformer model that achieves up to 2550x smaller cache sizes and significantly improved performance over Finch and Llama models

10. Google develops Conditioned Language Policy (CLP) - a versatile framework that finetunes language models that balance conflicting goals such as creativity and safety, outperforming state-of-the-art approaches

Multimodal

1. Tsinghua University introduces MultiTrust - a comprehensive benchmark assessing multimodal LLM trustworthiness across truthfulness, safety, robustness, fairness, and privacy, revealing significant challenges and risks in 21 modern models

   

2. Researchers introduce AUTOHALLUSION - an automatic benchmark generation method that effectively identifies hallucinations in large vision-language models by creating diverse, contextually challenging examples

Autonomous Agents

1. Researchers propose AgentPoison - a novel backdoor attack on LLM agents that poisons their memory or knowledge base, ensuring malicious data retrieval with high success and minimal impact on normal tasks

   

2. Tel Aviv University and UPenn release AssistantBench to evaluate the ability of web agents to automatically solve realistic and time-consuming tasks

Vision

1. Stability AI releases Stable Video 4D - its latest model for 3D dynamic video synthesis, generating multiple perspectives from a single video input

   

2. Apple proposes SlowFast-LLaVA (SF-LLaVA) - a training-free video LLM that excels in capturing spatial and temporal details, outperforming existing methods and rivaling fine-tuned state-of-the-art Video LLMs

   

3. Alibaba publishes OutfitAnyone - a two-stream conditional diffusion model that significantly improves the fidelity and consistency of virtual try-on images, adeptly handling garment deformation

   

4. Researchers develop MovieDreamer - a framework merging autoregressive models and diffusion rendering, setting new standards in long-duration video generation with intricate narratives and high visual fidelity

   

5. Meta and Oxford develop VGGSfM - a deep learning-based method for 3D scene reconstruction from photos, achieving record performance and simplifying the process for better accuracy and efficiency

   

6. Berkeley and Google unveil a method to create detailed 3D animations of moving scenes from regular videos, achieving top performance using advanced motion analysis and data consolidation

   

7. Stanford and Google present Streetscapes - a method for generating long sequences of city views based on language input and map layouts, achieving high visual quality over extended camera trajectories

   

8. Stanford introduces GET-Zero - a model architecture using Graph Embodiment Transformer (GET) that enables robots to adapt to new hardware configurations without retraining

   

AI Tools

1. PhotoMaker V2 - customize your photos to whatever style and instructions

2. MimicMotion - high-quality human motion video generation with pose-guided control

Turn yourself into an Olympic athlete with PhotoMaker

Open-source Packages

1. orch - Rust framework for LLM orchestration

2. DOM to Semantic Markdown - Convert HTML DOM to semantic Markdown for LLMs processing

3. MimicMotion - generate high-quality human motion videos

Plus >70 more open-source packages for AI engineers

MimicMotion turns static images into high-quality human animations

Last week’s AI Tidbits roundup

AI Roundup 07/11 -> 07/18/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.