AI Roundup 08/15 -> 08/22/2024

Microsoft’s new Phi 3.5 models, OpenAI’s fine-tuning for GPT-4o, AI21’s Jamba with 256k context, Dream Machine’s next-level video generation, Ideogram 2.0, and running Whisper in one file

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

📩 Published a new breakthrough paper? Just released an open-source package? Submit it here to ensure we don’t miss it and that it gets featured in next week’s post.

---

Overview

• ✨ Highlights (6 entries)

• Language Models (7 entries)

• Multimodal (3 entries)

• Vision (5 entries)

• Audio (3 entries)

• Open-source Packages (2 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. Microsoft releases three new open-source AI models in the Phi 3.5 series: Phi 3.5 mini-instruct, MoE-instruct, and vision-instruct models, offering scalable reasoning capabilities for commercial and scientific use across languages (Hugging Face)

   

2. AI21 releases Jamba Large and Jamba Mini - two new language models in its family of Mamba-Transformer models, featuring the longest context window for open models (256k) and rivaling state-of-the-art models like Llama 3.1 and Mistral Large (AI21)

   

3. OpenAI introduces fine-tuning for GPT-4o, allowing developers to tailor model responses and improve performance on domain-specific tasks like software engineering and text-to-SQL (OpenAI)

4. Luma Labs releases Dream Machine 1.5 - offering superior text-to-video generation, smarter prompts, and custom text rendering for elevated video creation (X)

   

5. Ideogram introduces Ideogram 2.0 - a new version of its text-to-image model, outperforming DALL-E, Midjourney, and FLUX Pro with improved text accuracy and an API for developers (X)

   

6. Midjourney launches a new web editor with public access along with free trials (X)

Language Models

1. Nous Research presents Hermes 3 - a new generation of Llama 3.1-based language models (8B, 70B, 405B) featuring significant improvements in agentic tasks and multi-turn conversations, with open-source weights available on Hugging Face

   

2. Researchers introduce TableLLM - a family of models that achieves results on par with GPT-3.5, and TableBench - a benchmark for evaluating LLM performance on complex real-world tabular data

   

3. Cohere demonstrates that integrating code into LLM pre-training delivers broad performance enhancements across various domains, including up to an 8.2% increase in natural language reasoning and a 12x improvement in coding tasks

4. Zed introduces Zed AI, leveraging Anthropic’s Claude 3.5 Sonnet to enhance coding with AI-powered inline transformations and an efficient assistant panel

   

5. Researchers develop the concept of Automated Design of Agentic Systems (ADAS) and demonstrate Meta Agent Search, an algorithm that automatically invents versatile agents, significantly surpassing state-of-the-art designs across various domains

   

6. YouTube Music proposes a context-aware ranking system using Transformer models to dynamically refine music recommendations based on user preferences and actions

   

7. Rutgers University shows that GPT-4 can accurately predict molecular structures of amino acids and proteins, revealing unexpected potential for structural biology applications

Multimodal

1. Salesforce releases xGen-MM (BLIP-3) - a framework for developing large multimodal models featuring strong in-context learning and competitive performance alongside safety-tuned models to reduce harmful behaviors

   

2. Meta and Waymo develop Transfusion - a multimodal training strategy that scales better than traditional methods by combining language modeling and diffusion, capable of generating high-quality text and images at scale

   

3. Hugging Face releases an in-depth tutorial on building and training a neural network-powered robotic arm, making advanced robotics accessible to all

   

Vision

1. Researchers introduce LongVILA - a solution for long-context vision-language models that extends vision-language models' frame capacity by 128x and improves long video captioning by 1.6x, achieving near-perfect accuracy in ultra-long context tasks

   

2. NUS and Oxford introduce SurgSAM-2 - an advanced surgical video segmentation model that enhances efficiency and accuracy with an Efficient Frame Pruning mechanism, achieving 3x the FPS of SAM2

   

3. Researchers develop MM-SAM - an advanced multimodal segmentation model that outperforms SAM, enabling robust processing across diverse sensors like LiDAR, depth, and thermal data

   

4. University of Washington and Meta introduce JPEG-LM - a model that generates images by directly outputting compressed JPEG file bytes, outperforming traditional pixel-based and vector quantization methods with a 31% FID reduction

   

5. Researchers introduce MeshFormer - a sparse-view 3D reconstruction model that efficiently generates high-quality textured meshes by leveraging explicit 3D structures and SDF supervision

   

Audio

1. Mozilla openly releases Whisperfile - a high-performance, local tool for audio transcription and translation using OpenAI's Whisper model

2. Dolby develops a self-supervised approach and pipeline for creating smooth "audio match cuts" in videos, introducing a new dataset and evaluating methods to optimize seamless audio transitions

3. Google launches HeAR - an open and powerful AI model trained on 300 million audio samples, enabling advanced health diagnostics from sounds like coughs and breaths

   

Open-source Packages

1. Deep-Live-Cam - real-time face swap for videos using a single image

2. Phoenix - an AI observability platform designed for experimentation, evaluation, and troubleshooting of LLM-powered apps

Plus >70 more open-source packages for AI engineers

Last week’s AI Tidbits roundup

AI Roundup 08/08 -> 08/15/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.