AI Roundup 06/20 -> 06/27/2024

A faster and cheaper Claude 3.5 from Anthropic that outperforms GPT-4o, a chip to rival Nvidia, LLMs that train LLMs, and a set of powerful multimodal models from Apple and NYU

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

---

Overview

• ✨ Highlights (5 entries)

• Language Models (10 entries)

• Multimodal (3 entries)

• Vision (9 entries)

• Open-source Packages (3 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. Anthropic releases Claude 3.5 Sonnet - a model with a 200k token context window, outperforming GPT-4o and featuring a new dynamic Artifacts workspace (Anthropic)

   

2. Anthropic's Claude got another major update in the form of "Projects", a GPTs-like feature to share chats with custom documents (RAG) and instructions

   

3. A new startup called Etched is trying to take on Nvidia by presenting the world’s first specialized chip for Transformers, delivering over 500,000 tokens per second and claims to be >10x faster and cheaper than NVIDIA’s next-generation Blackwell (Company blog)

   

4. A new venture by ex-Meta researchers introduces ESM3 - a language model capable of generating novel proteins and reasoning over the sequence, structure, and function of proteins (Company blog)

   

5. Toys"R"Us showcases the first-ever branded video created using OpenAI Sora (Company blog)

Language Models

1. Together AI releases a personalized code assistant using Mistral 7B Instruct v0.2, achieving up to 19% better accuracy and significant cost savings with RAG fine-tuning

   

2. UCLA finds that decision boundaries in in-context learning for LLMs are often irregular and non-smooth, and propose methods to enhance their robustness and generalizability

   

3. Microsoft and Tsinghua University introduce Instruction Pre-Training - a scalable supervised multitask pre-training framework that significantly enhances language model performance, allowing smaller models like Llama3-8B to match or surpass larger models like Llama3-70B

   

4. Sakana AI showcases LLM² - an approach where LLMs autonomously generate and refine hypotheses and code, leading to the state-of-the-art DiscoPOP algorithm

   

5. Peking University and Microsoft release EAGLE-2 - an enhanced speculative sampling method for LLMs, featuring a context-aware dynamic draft tree for improved efficiency and a ~4x speedup

   

6. The University of Waterloo develops LongRAG - a new RAG framework using long-context LLMs that enhances retrieval and answer extraction, achieving remarkable performance improvements on multiple datasets

   

7. Imbue releases a comprehensive toolkit and a blog post sharing learnings from its training of a 70B model optimized for reasoning and coding

   

8. Hugging Face releases the BigCodeBench Leaderboard to evaluate LLMs with practical and challenging programming tasks

   

9. Researchers present a comprehensive study of the "LLM-as-a-judge" paradigm, finding Llama-3 70B and GPT-4 Turbo have high human alignment

10. Researchers find that GPT-4o provides superior moral advice compared to a renowned ethics expert

Multimodal

1. NYU releases Cambrian-1 - a state-of-the-art family of multimodal LLMs with enhanced visual grounding and a new benchmark, CV-Bench

   

2. EPFL and Apple publish a new training approach for multimodal models along with commercially permissive Any-to-Any vision models that handle three times more tasks without performance loss

3. Researchers introduce MathV360K - a diverse and synthesized multimodal dataset that significantly enhances the mathematical reasoning capabilities of multimodal LLMs, along with Math-LLaVA - a fine-tuned model specializing in math

Vision

1. Berkeley and UIUC develop DigiRL - an autonomous reinforcement learning approach for fine-tuning visual language models, achieving a 49% improvement in controlling real-world GUIs over traditional static training methods

   

2. Researchers develop LongVA - a groundbreaking model that uses extended context length to process extremely long video sequences, achieving top performance

   

3. Researchers introduce MotionBooth - a framework for animating customized subjects with precise control over object and camera movements

   

4. Fudan University and Baidu unveil Hallo - an end-to-end diffusion-based approach for portrait image animation driven by speech audio, enhancing precision in lip synchronization, expression, and pose motion

5. Researchers introduce ExVideo - a novel post-tuning methodology that extends the temporal duration of video synthesis models by up to five times without increasing training costs

   

6. Researchers propose Text-Animator for text-to-video visual text generation using text embedding injection, camera control, and text refinement modules to enhance accuracy and stability

   

7. Researchers introduce Video-Infinity, utilizing innovative parallel processing techniques to enable fast, long-form video generation, achieving speeds 100x faster than prior methods

   

8. Researchers introduce StyleFeatureEditor - a new method for StyleGAN inversion that enhances image reconstruction and editing, outperforming existing approaches and handling out-of-domain examples effectively

   

9. Virginia Tech and ETH Zurich develop Stylebreeder - a groundbreaking dataset and model collection that enhances artistic creation through personalized style recommendations

Open-source Packages

1. Claude Engineer - an interactive command-line interface that leverages Sonnet 3.5 to assist with software development tasks

2. Continue - connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains

3. DocsGPT - GPT-powered chat to search over your project's documentation

Plus >70 more open-source packages for AI engineers

Build your private coding assistant with Continue

Last week’s AI Tidbits roundup

AI Roundup 06/13 -> 06/20/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.