AI Roundup 06/27 -> 07/04/2024

Google's open Gemma 2 models outshine Llama 3, UIUC's cost-effective autonomous agents, Resemble's DETECT-2B detecting deep fake audio, and OpenAI's CriticGPT enhances code reliability

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

📩 Published a new breakthrough paper? Just released an open-source package? Submit it here to ensure we don’t miss it and that it gets featured in next week’s post.

---

Overview

• ✨ Highlights (5 entries)

• Language Models (13 entries)

• Autonomous Agents (5 entries)

• Multimodal (3 entries)

• Vision (5 entries)

• Open-source Packages (4 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. Google openly releases Gemma 2 - two new models (9B, 27B) which substantially outperform similar models like Llama 3, along with providing general availability to its 2M context window Gemini Pro (Google Blog)

   

2. French-based Kyutai unveils Moshi - the first publicly available real-time voice AI, featuring a realistic and emotionally nuanced voice capable of 70 expressions (YouTube)

3. UIUC proposes Agentless - a simplified approach that outperforms complex autonomous software agents, achieving 27.3% performance and the lowest cost ($0.34) on the SWE-bench Lite benchmark (Paper)

   

4. Resemble AI releases DETECT-2B - a deepfake audio detection tool achieving 94% accuracy and near real-time results across over 30 languages (Company blog)

   

5. Figma releases new AI features to enhance design workflows, offering tools like visual search, realistic mockup generation, and quick-click prototyping (Company blog)

   

Language Models

1. Salesforce showcases xLAM-7B and xLAM-1B, excelling in function calling with state-of-the-art performance, outperforming GPT-3.5-Turbo and Claude-3 Haiku and surpassing multiple GPT-4 models despite their smaller size

   

2. Meta releases the LLM Compiler - advanced models that enhance code optimization and compiler functions, outperforming existing solutions like CodeLlama and GPT-4 Turbo

   

3. OpenAI showcases CriticGPT - a model trained to catch coding errors, outperforming human reviewers and enhancing the reliability of AI-generated code

   

4. Researchers develop WildGuard - a lightweight moderation model improving LLM safety by identifying malicious prompts and reducing jailbreak success rates from 79% to 2%

   

5. Berkeley develops RouteLLM - efficient router models that dynamically select LLMs, achieving over 2 times cost reduction and maintaining high response quality with strong transfer learning capabilities

6. Microsoft Research releases GraphRAG on GitHub, which constructs a knowledge graph from text data to connect related information and improve understanding within large datasets

7. Researchers develop PubMedVision, refining PubMed image-text pairs to enhance Multimodal LLM performance, resulting in two open-weight superior medical multimodal models HuatuoGPT-Vision models (7B and 34B)

   

8. Salesforce proposes the "Summary of a Haystack" (SummHay) benchmark to evaluate LLM and RAG system performance on long-context tasks, showing that even the best systems fall over 10 points short of human performance

   

9. Princeton develops CharXiv - an evaluation suite exposing weaknesses in Multimodal LLM chart understanding, showing GPT-4o at 47% accuracy, Sonnet 3.5 at 60$, and open-source models at 32%, compared to human performance of 80%

   

10. Researchers develop WE-MATH - a benchmark with 6.5K visual math problems and a four-dimensional metric, advancing the understanding of LMMs' reasoning processes and highlighting GPT-4o's progress in knowledge generalization

    

11. Researchers propose Adam-mini - an optimization method that partitions model parameters into blocks with optimal learning rates, outperforming AdamW by achieving 49% higher throughput during Llama2-7B pre-training

    

12. Tencent introduces Persona Hub - a collection of 1B diverse personas, enabling scalable and versatile synthetic data creation for various scenarios

    

13. Researchers showcase how demographic features in prompts significantly influence LLM responses on the Political Compass Test, revealing biases and disparities in generated stances

    

Autonomous Agents

1. Nexa AI and Stanford openly release Octo-planner - an efficient Planner-Action framework with a Phi-3 Mini LLM for planning and Octopus model for execution, optimized for edge devices with a 97% success rate in task completion

   

2. Researchers propose OmniJARVIS - a Vision-Language-Action model that unifies tokenization of multimodal interaction data for enhanced reasoning and decision-making in Minecraft

   

3. Tsinghua University showcases SimClass - an AI system that simulates realistic classroom interactions with virtual teacher and student agents, boosting engagement of human students

   

4. CMU showcases a best-first tree search algorithm for LM agents, achieving significant improvements in web task success rates, including a 39% increase on VisualWebArena

   

5. Princeton proposes a new benchmarking framework for AI agents that jointly optimizes cost and accuracy, aiming to develop more practical and robust agents

Multimodal

1. CMU and Google propose In-Context Abstraction Learning (ICAL), enabling LLMs and Visual LMs to generate high-quality prompt examples from sub-optimal demonstrations, significantly enhancing decision-making

   

2. Peking University proposes MMEvalPro - a robust benchmark for evaluating Large Multimodal Models that mitigates biases with a trilogy evaluation pipeline, showing a 31.7% gap between the best LMM and human performance

   

3. Researchers introduce an intuitive robot programming framework that uses natural language prompts and Robot Operating System (ROS) integration with LLMs, allowing non-experts to easily specify tasks and demonstrating strong performance across multiple applications

Vision

1. Researchers open source InternLM-XComposer-2.5 - a versatile vision-language model achieving GPT-4V level capabilities with a 7B LLM backend, excelling in various text-image comprehension and composition tasks

   

2. Researchers propose OMG-LLaVA - a framework combining pixel-level vision understanding with reasoning abilities for flexible user interaction through visual and text prompts

   

3. Google presents Magic Insert - a method that enables style-aware drag-and-drop of subjects into different styled images, significantly outperforming traditional inpainting techniques

   

4. Researchers propose FoleyCrafter, a novel framework for generating high-quality, synchronized sound effects for videos, leveraging a pre-trained text-to-audio model for realistic and well-aligned audio

5. Tencent introduces RealTalk - a novel framework that enhances audio-driven face generation by achieving precise lip synchronization and real-time high-quality facial renderings

Open-source Packages

1. Vanna - chat with your SQL database

2. Tabby - a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot

3. phidata - build AI agents with memory, knowledge, and tools

4. llama-agents - async-first framework for building, iterating, and productionizing multi-agent systems

Plus >70 more open-source packages for AI engineers

Tabby is an open-source alternative to GitHub Copilot

Last week’s AI Tidbits roundup

AI Roundup 06/20 -> 06/27/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.