AI Roundup 08/08 -> 08/15/2024

Musk's xAI reveals Grok-2 to rival GPT-4, Anthropic's prompt caching reduces costs by 90%, Google's Gemini 1.5 slashes API prices, Cosine's Genie tops AI engineering benchmarks, and new open TTS tools

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

📩 Published a new breakthrough paper? Just released an open-source package? Submit it here to ensure we don’t miss it and that it gets featured in next week’s post.

---

Overview

• ✨ Highlights (10 entries)

• Language Models (14 entries)

• Multimodal (3 entries)

• Vision (7 entries)

• Audio (2 entries)

• Open-source Packages (3 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. Elon Musk's xAI unveils Grok-2 and Grok-2 mini, showcasing advanced chat, coding, and reasoning capabilities that outperform top models like Claude 3.5 Sonnet and GPT-4-Turbo (xAI)

   

2. Anthropic introduces prompt caching on its API, reducing costs and latency for large prompts by up to 90% and 85% respectively, now in public beta for Claude 3.5 Sonnet and Claude 3 Haiku

   

3. Cosine releases Genie - an autonomous AI software engineer that outperforms industry leaders like Devin's and Factory's software developer agents by capturing the cognitive processes of human developers, achieving the highest score on SWE-Bench (30%) (Company blog)

   

4. Salesforce develops DEI - a meta-framework that leverages diverse LLM-based software engineering agents, achieving a 25% improvement over individual agents and securing the highest performance on SWE-Bench Lite with a 55% resolve rate

   

5. MultiOn and Stanford showcase Agent Q - a fine-tuned LLaMa 70B agent that can self-correct and autonomously improve by self-play on real tasks on the internet, scaling its zero-shot success rate from 18.6% to 81.7%, exceeding GPT-4's performance in complex real-world tasks (Company blog)

   

6. Sakana AI introduces The AI Scientist - a novel system that automates the entire scientific research process using LLMs, capable of independently generating, executing, and documenting new research ideas at a low cost (Company blog)

   

7. Georgia Tech and IBM release Transformer Explainer - an interactive tool that helps non-experts understand Transformers by visualizing the inner workings of GPT-2 in real-time (Project website)

   

8. Google releases Gemini Live - a voice-interactive AI chatbot with enhanced emotional expression and real-time adaptive dialogue, similar to ChatGPT's new Advanced Voice Mode capability, offering hands-free and long-context conversational capabilities (Google Blog)

   

9. DeepMind releases Imagen 3 - a latent diffusion model that outperforms state-of-the-art models in generating high-quality images from text prompts, with built-in measures to enhance safety and representation (Paper)

   

10. Mistral releases its custom GPTs-like feature on its platform, allowing users to launch agents with custom instructions and built-in examples of input-output pairs (in-context learning) (Mistral)

    

Language Models

1. Researchers open source OpenResearcher - an AI-driven platform that integrates LLMs with domain-specific knowledge through Retrieval-Augmented Generation, enabling researchers to efficiently navigate and generate insights from scientific literature

   

2. Google reduces Gemini 1.5 Flash API costs by up to 78%, expands language support to over 100 languages, and enables free fine-tuning for developers

   

3. Alibaba Cloud presents Qwen2-Math - a new series of math-focused LLMs that outperform much larger models like GPT-4o in mathematical reasoning

   

4. Researchers release Med42-v2 - an open suite of clinical LLMs (7B, 70B) built on Llama3, fine-tuned with specialized data to excel in clinical settings, surpassing Llama3 and GPT-4 in medical benchmarks

5. Nvidia releases a pruned Llama 3.1 model via a unique weight pruning and distillation method, reducing its size from 8B to 4B, achieving 2.7x speed up while maintaining similar performance

6. TII introduces Falcon Mamba 7B - an open-access model that overcomes sequence scaling limitations of Transformers, processing long sequences efficiently on a single GPU

   

7. Nvidia and BlackRock develop HybridRAG - a novel approach that combines vector database and knowledge graph-based techniques to enhance Q&A systems, significantly improving information extraction from complex financial documents

   

8. Tsinghua University develops AgentWrite and the LongWriter-6k dataset, enabling LLMs to generate outputs exceeding 20,000 words, with their model setting new standards in ultra-long generation tasks

   

9. Microsoft develops rStar - a self-play mutual reasoning method that significantly enhances small language models' reasoning capabilities, achieving remarkable accuracy improvements across multiple benchmarks without requiring fine-tuning

   

10. OpenAI collaborates to improve SWE-bench by releasing a human-validated subset of SWE-bench that more reliably evaluates AI models’ autonomous software engineering capabilities

    

11. DeepMind publishes Gemma Scope - an open suite of JumpReLU SAEs trained on Gemma 2 models, offering interpretable feature extraction to support safety and interpretability research by lowering training cost barriers

    

12. Researchers develop OpenLogParser - an unsupervised log parsing approach that uses open-source LLMs to overcome privacy and cost challenges, achieving 25% higher accuracy and 2.7 times faster processing compared to state-of-the-art methods

13. Apple introduces ToolSandbox - a comprehensive evaluation framework for tool-use capabilities in LLMs, revealing significant challenges even for state-of-the-art models

14. A survey shows 87% of Harvard students regularly use generative AI, with ChatGPT being the most popular tool, impacting study habits and academic support

Multimodal

1. Tencent unveils VITA - the first open-source multimodal LLM capable of simultaneous processing of video, image, text, and audio, with advanced interactive capabilities

   

2. Alibaba openly releases mPLUG-Owl3 - a versatile multimodal LLM that excels at understanding long image sequences using novel hyperattention blocks and achieves state-of-the-art performance on diverse benchmarks

   

3. Researchers present GMAI-MMBench - a comprehensive and customizable benchmark for evaluating large vision language models in medical AI, revealing significant performance gaps and the need for advancements in model development

Vision

1. Researchers release VideoGameBunny - an AI model that outperforms larger models in video game understanding, with impressive accuracy in recognizing game-specific details

   

2. Zhipu and Tsinghua University publish CogVideoX - a diffusion transformer model that uses a 3D Variational Autoencoder (VAE) and expert transformers for generating coherent, high-quality videos from text prompts, achieving state-of-the-art performance in both machine and human evaluations

3. Alibaba develops UniPortrait - a framework unifying single- and multi-ID image customization with high fidelity and editability, using adaptive ID embedding and routing modules for superior performance

   

4. CUHK introduces ControlNeXt - an innovative and efficient architecture that dramatically reduces learnable parameters and enhances controllable image and video generation with robust performance and minimal additional cost

   

5. Researchers introduce KEEP - a Kalman-inspired framework for video face super-resolution that maintains temporal consistency and effectively captures facial details across frames

6. Tencent proposes Sketch2Scene - a novel approach for generating interactive 3D game scenes from hand-drawn sketches, using a 2D diffusion model and procedural content generation to create playable scenes

   

7. Meta open sources VFusion3D - a scalable 3D generative model that uses pre-trained video diffusion models to generate 3D assets from single images, outperforming state-of-the-art methods with over 90% user preference

   

Audio

1. HuggingFace releases Parler TTS v1 - a commercially permissive open-source text-to-speech model offering up to 4x faster generation, enhanced speaker options, and fine-tuning capabilities

   

2. The Qwen team releases Qwen2-Audio, a multimodal LLM that enables voice interaction and audio analysis in over eight languages, now openly available on Hugging Face

Open-source Packages

1. speech-to-speech - a pipeline for speech-based conversational AI applications

2. DistillKit - an open-source toolkit for LLM distillation

3. LiteLLM - call 100+ LLM APIs using the same format - OpenAI, Cohere, Anthropic, HuggingFace, and more

Plus >70 more open-source packages for AI engineers

speech-to-speech employs multiple models to power conversational AI apps

Last week’s AI Tidbits roundup

AI Roundup 08/01 -> 08/08/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.