AI Roundup 05/16 -> 05/23/2024

Microsoft's new AI PC, Copilot agents to automate work, Anthropic peek into LLMs' brain, open-source Phi-3 models with 128k context window and vision, and Mistral's new 7B model with function calling

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

---

Overview

• ✨ Special feature: Microsoft Build 2024

• ✨ Highlights (3 entries)

• Language Models (6 entries)

• Multimodal (4 entries)

• Vision (6 entries)

• Open-source Packages (3 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Special feature: Microsoft Build 2024

• Introduction of Copilot+ PCs - Microsoft revealed a new category of AI-optimized Windows PCs called Copilot+ PCs. These devices feature powerful new silicon, all-day battery life, and access to advanced AI models. Capable of tasks like real-time image creation with Cocreator and language translation with Live Captions, these devices will be available from June 18, starting at $999.

• Copilot AI Agents - Microsoft unveiled the expansion of Copilot AI agents to handle menial tasks autonomously. These agents can monitor emails, automate tasks, assist with onboarding, and perform data entry. The new capabilities will be available in Copilot Studio later this year.

• Phi-3 Vision and multimodal AI - Microsoft introduced Phi-3-vision, a compact multimodal AI model capable of reading text and analyzing images on mobile devices. This model is part of the Phi-3 family, designed for efficient performance and available in preview now.

• Phi Silica local language model - Microsoft announced Phi Silica, a 3.3B transformer-based model optimized for Copilot+ PCs. It is the first small language model deployed locally on Windows, providing fast performance.

• Real-time video translation in Edge - Edge browser will soon feature AI-powered real-time video translation. This tool will enable users to translate videos from platforms like YouTube and LinkedIn in multiple languages, enhancing accessibility and global communication.

✨ Highlights

1. Anthropic releases a detailed study peeking into LLMs' brains, showcasing how millions of concepts such as gender, conversational styles, and political views are represented, revealing the first-ever detailed look inside a modern LLM (Anthropic)

   

2. Microsoft open sources new Phi-3 models, including a 7B, 14b, and a new multimodal variant with vision capabilities featuring a 128k context window (Microsoft Blog)

   

3. Mistral releases 7B v0.3 - a new version of its small and powerful open language model, with function calling support (Hugging Face)

Language Models

1. Nvidia develops ChatQA - a suite of models leveraging innovative tuning and retrieval techniques to outperform GPT-4 in RAG and QA tasks, open-sourcing the underlying models and benchmarks to support further research

   

2. Researchers introduce TransAgents - a multi-agent framework for literary translation using LLMs, which improves translation quality and is preferred by both humans and advanced LLMs

   

3. Databricks compares LoRA and full finetuning for language models, finding LoRA underperforms but offers strong regularization, preserving base model performance on non-target tasks

   

4. ByteDance introduces OpenRLHF - an open-source framework that efficiently scales reinforcement learning from human feedback (RLHF) for LLMs, optimizing resource use and simplifying the training process

5. Researchers introduce MoRA - a method that enhances LLMs' learning and memory capabilities by using high-rank updating with the same number of trainable parameters as LoRA

6. Microsoft and Mila AI introduce a library of adapters and new methods for task generalization, improving large language models' performance on new tasks without retraining

Multimodal

1. Meta presents Chameleon - a family of multimodal models that excels at understanding and generating images and text, achieving state-of-the-art performance in several tasks

   

2. Researchers introduce Imp - a family of lightweight multimodal models (2B, 3B, and 4B) that outperform existing models of similar and larger sizes and can be efficiently deployed on mobile devices

   

3. Stanford shows that many-shot in-context learning (ICL) with up to 2,000 examples significantly improves the performance of multimodal models like GPT-4o and Gemini 1.5 Pro

   

4. Berkeley and Stanford introduce Octo - a new AI model that can quickly learn and adapt to different types of robots and tasks, making it easier to train robots for various jobs

   

Vision

1. Google unveils CAT3D - a method that quickly creates high-quality 3D scenes from any number of input images using a multi-view diffusion model

   

2. Researchers introduce Glyph-SDXL - a model that improves text rendering accuracy in design images from less than 20% to nearly 90% by integrating a customized text encoder

   

3. The University of Michigan develops a method to create spectrograms that look like natural images and sound like natural audio using pre-trained diffusion models in a shared latent space

   

4. Microsoft introduces BiomedParse - an AI model that simplifies biomedical image analysis by accurately performing segmentation, detection, and recognition across various imaging types

   

5. Researchers introduce FIFO-Diffusion - a novel technique for generating infinitely long videos from text prompts using a pretrained diffusion model without additional training

   

6. Researchers introduce DIAMOND - a reinforcement learning agent using diffusion models for world modeling, achieving a record score on the Atari 100k benchmark by retaining better visual details

   

Open-source Packages

1. Indexify - extract and retrieve information from unstructured data such as documents, images, videos, and audio files

2. SWE-agent - takes GitHub issues and automatically fixes them using GPT-4 or other LMs, achieving 12.29% on SWE-bench

3. DIY-Astra - a Flask application that utilizes computer vision and natural language processing to create an interactive AI assistant (like the Google I/O demo)

Plus >70 more open-source packages for AI engineers

SWE-agent resolves GitHub bugs and issues with minimal human intervention

Last week’s AI Tidbits roundup

AI Roundup 05/09 -> 05/16/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.