AI Roundup 06/13 -> 06/20/2024

A new AI engineer beats Devin AI, OpenAI's former founder starts AI safety company, SOTA open coding model in 300+ languages, and tiny open vision models outperforming 100x larger ones

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

---

Overview

• ✨ Highlights (5 entries)

• Language Models (6 entries)

• Vision (10 entries)

• Multimodal (4 entries)

• Audio (3 entries)

• Open-source Packages (2 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. Factory emerges out of stealth to automate software engineering by modeling the cognitive processes of developers, achieving top performance on SWE-bench (19.27% compared to Devin's 13.86%) (Company blog)

   

2. Ilya Sutskever, OpenAI's former chief scientist, starts Safe Superintelligence Inc. - a new company to build safe superintelligence (X)

3. Open Interpreter introduces Local III - a suite of tools for running powerful language models locally to control your personal computer, enhancing control and privacy (Company blog)

   

4. Meta releases four new AI models, including the multimodal Chameleon (7B, 34B) mode, JASCO, a text-to-music generation model, and AudioSeal, a model for detecting AI-generated speech (Meta AI)

   

5. Runway releases Gen-3 Alpha - an AI model generating high-quality, hyperrealistic videos with expressive human characters and smooth transitions (Company blog)

Language Models

1. DeepSeek-AI releases DeepSeek-Coder-V2 - an open-source language model supporting 338 programming languages, beating top commercial models like GPT-4 Turbo in code generation and mathematics

   

2. Nvidia releases Nemotron-4 340B - an open-source toolkit generating and evaluating synthetic data for training robust LLMs

   

3. Apple releases 20 Core ML models and four datasets on Hugging Face, enabling developers to build AI applications that run on Apple devices offline

   

4. Fireworks AI releases Firefunction-v2 - an open weights model that outperforms Llama 3 and rivals GPT-4o in function calling capabilities, offering superior cost-efficiency and latency

   

5. Researchers release PowerInfer-2 - an optimized inference framework for smartphones, supporting Mixtral 47B models at 11 tokens per second, 22x faster than current frameworks

   

6. Anthropic investigates how AI models exploit training objectives, highlighting the risks of specification gaming and misaligned incentives

   

Vision

1. Microsoft releases Florence-2 - a commercially permissive state-of-the-art small vision model family (200M, 800M params) that outperforms larger specialized models in tasks like image description and object recognition

   

2. Stanford and Berkeley present OpenVLA - a 7B-parameter SOTA open-source model for vision-language-action tasks, achieving superior performance in robot manipulation and efficient fine-tuning on consumer GPUs

   

3. HKU and TikTok unveil Depth Anything V2 - a highly efficient and accurate monocular depth estimation model that leverages synthetic images to significantly outperform previous models in speed and accuracy

   

4. Wayve releases PRISM-1 - a scene reconstruction model creating 4D simulations of real-world environments to train autonomous driving systems

   

5. University of Amsterdam and Meta introduce PiT - a transformer model that processes images at the pixel level, demonstrating superior performance over conventional approaches like Vision Transformers

   

6. Bar-Ilan University and NVIDIA develop CountGen, significantly improving object count accuracy in text-to-image generation by using internal diffusion model features for precise object separation and correction

   

7. Microsoft develops FontStudio - a system for multilingual font text effects using shape-adaptive diffusion models, achieving superior user preference ratings over commercial products

   

8. Nvidia presents L4GM - a 4D model that generates animated objects from single-view videos in a second, leveraging a novel dataset and advanced temporal self-attention techniques

   

9. UCSC releases Recap-DataComp-1B - an enhanced dataset using Llama 3 to recaption 1B web images, yielding superior performance in cross-modal retrieval and text-to-image generation tasks

   

10. Researchers introduce MMDU and MMDU-45k to benchmark and enhance large vision language models' performance in multi-turn, multi-image conversations, showing significant improvements with fine-tuning
    

Multimodal

1. Stanford researchers present HumanPlus - a system that enables robots to learn complex tasks from human demonstrations through real-time shadowing and autonomous training

   

2. Researchers develop Traversal of Layers (TroL) - an innovative layer reuse approach that significantly boosts Large Language and Vision Models (LLVM) performance, outperforming larger open-source models and rivaling closed-source counterparts with fewer resources

   

3. The National University of Singapore and Meta develop the LIVE framework, enabling efficient, real-time video streaming dialogue in large multimodal models, achieving impressive performance both online and offline

   

4. Researchers present OlympicArena - a benchmark with 11k bilingual problems to evaluate AI's cognitive reasoning abilities across text and text-image modalities, showing advanced models like GPT-4o achieve only ~40% accuracy

   

Audio

1. Google DeepMind is developing video-to-audio (V2A) - a technology that generates synchronized soundtracks for silent videos using video pixels and natural language text prompts

   

2. Camb AI opens Mars5 - a voice cloning model that captures speech nuances with higher realism, available on GitHub

   

3. Researchers propose GAMA - an audio-language model with advanced audio understanding and complex reasoning abilities, significantly outperforming other models on diverse tasks by up to 84%

Open-source Packages

1. Faster Whisper - a reimplementation of OpenAI's Whisper model using CTranslate2, which is a fast inference engine for Transformer models

2. Omni-zero - a diffusers pipeline for zero-shot stylized portrait creation

Plus >70 more open-source packages for AI engineers

Last week’s AI Tidbits roundup

AI Roundup 06/06 -> 06/13/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.