AI Roundup 08/01 -> 08/08/2024

A 50% cheaper GPT-4o, a new open model beating DALL-E 3 and Midjourney, OpenBMB's MiniCPM excelling in real-time video understanding, Meta's AI self-training, and Stability's instant 3D models

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools every week so you won’t have to.

📩 Published a new breakthrough paper? Just released an open-source package? Submit it here to ensure we don’t miss it and that it gets featured in next week’s post.

---

Overview

• ✨ Highlights (5 entries)

• Language Models (7 entries)

• Multimodal (7 entries)

• Vision (7 entries)

• Audio (1 entry)

• Open-source Packages (2 entries)

Recent Deep Dives

Top 8 leaderboards to choose the right AI model for your task

12 techniques to reduce your LLM API bill and launch blazingly fast products

Harnessing research-backed prompting techniques for enhanced LLM performance

✨ Highlights

1. OpenAI releases a new GPT-4o version that is slightly better and 50% cheaper than the previous GPT-4o model

   

2. Black Forest Labs emerges from stealth and open sources FLUX.1 - a commercially permissive image generation models suite that outperforms leading models like DALL-E 3 and Midjourney v6, teasing the release of a similar open model for text-to-video generation

   

3. OpenBMB introduces MiniCPM-V 2.6 - an open multimodal model that outperforms GPT-4V on single image, multi-image, and video understanding, supporting real-time on-device video understanding

   

4. GitHub releases GitHub Models in limited public beta, enabling users to test leading AI models directly within the GitHub platform

   

5. OpenAI faces leadership upheaval as co-founder John Schulman departs for rival Anthropic and co-founder and president Greg Brockman takes an extended leave

Language Models

1. OpenAI releases Structured Outputs, ensuring AI-generated data conforms precisely to developer-supplied JSON schemas for enhanced reliability, achieving perfect accuracy with the new gpt-4o-2024-08-06 model

   

2. Meta develops a Self-Taught Evaluator that uses synthetic data to iteratively train LLM-as-a-Judge, significantly boosting LLM performance on RewardBench and matching top reward models trained with human annotations

   

3. Researchers propose RAGate - a gating model that predicts when conversational systems need Retrieval Augmented Generation (RAG) for enhanced responses, ensuring high quality and confidence

   

4. Berkeley and DeepMind unveil a compute-optimal scaling strategy for LLMs, enhancing test-time compute efficiency by 4x and enabling smaller models to outperform significantly larger models on specific tasks

5. Researchers develop the Hybrid Multimodal Memory module, empowering their new Optimus-1 agent to excel in long-horizon tasks with near human-level performance and outperform the GPT-4V baseline through enhanced planning and reflection

6. Tsinghua University develops contrastive fine-tuning to boost text embeddings in smaller language models, leading to a 56% performance gain in MiniCPM and improvements across Phi-2 and Gemma

7. Microsoft showcases how generative AI tools like Copilot enhance workplace productivity, with 29% of power users saving over 30 minutes daily, reducing email reading by 11% and increasing document creation by 10%

Multimodal

1. Researchers release the technical report for the MiniCPM-V model - efficient and open multimodal LLMs outperforming GPT-4V-1106 and deployable on mobile devices, offering robust performance, low hallucination rates, and multilingual support

   

2. ByteDance and NTU present LLaVA-OneVision - an advanced multimodal model family that excels in single-image, multi-image, and video scenarios, demonstrating strong transfer learning across modalities

   

3. Researchers develop Lumina-mGPT - a versatile family of multimodal models that excels in text-to-image generation and various vision-language tasks through innovative pretraining and finetuning techniques

   

4. Researchers release the Multimodal Multi-image Understanding (MMIU) benchmark - the most comprehensive suite for evaluating Large Vision Language Models (VLMs), highlighting significant challenges in multi-image comprehension with top models achieving just 55.7% accuracy

   

5. National University of Singapore and Microsoft develop MM-Vet v2, enhancing multimodal model evaluation with "image-text sequence understanding" and expanded evaluation set, showcasing Claude 3.5 Sonnet as the top performer

   

6. Salesforce open source a 1 trillion-token multimodal dataset to further support the training of larger and more capable multimodal models

   

7. Researchers develop MedTrinity-25M, a large-scale medical dataset with enriched annotations for multimodal AI tasks, achieving state-of-the-art results in medical image understanding and report generation

Vision

1. Oxford researchers present MedSAM-2 - an AI model for medical image segmentation that outperforms previous models by treating images as video sequences

   

2. Zhipu AI and Tsinghua University present CogVideoX 2B - an open diffusion transformer model that excels in generating coherent, long-duration videos from text prompts

   

3. Stability AI releases Stable Fast 3D - a technique generating high-quality 3D models from single images or text descriptions in under a second

   

4. Google presents the first learned robot agent that reaches amateur human-level performance in table tennis, winning 45% of matches against varied human opponents through hierarchical and modular policy architecture

   

5. Researchers develop Object Images - a method that transforms 3D shapes into 2D formats for use with image generation models, achieving high-quality 3D models

   

6. Researchers present MeshAnything V2 - an autoregressive transformer that excels in high-quality, controllable 3D mesh generation using the novel Adjacent Mesh Tokenization method

   

7. Researchers develop an analysis-by-synthesis method using generative models for visual data mining, achieving scalable and versatile analysis across diverse datasets by defining a typicality measure for visual elements

Audio

1. ByteDance presents listening-while-speaking language model (LSLM) - a model that achieves full duplex communication in speech language models, enhancing real-time interaction

Open-source Packages

1. Llama Coder - an open source Claude Artifacts alternative, generating small apps with one prompt

2. llama-agentic-system - run Llama 3.1 as a system capable of performing agentic tasks like breaking a task down, performing multi-step reasoning, and using tools

Plus >70 more open-source packages for AI engineers

Llama Coder generates apps from natural language prompts

Last week’s AI Tidbits roundup

AI Roundup 08/01 -> 08/08/2024

Reach AI builders, researchers, and entrepreneurs by partnering with AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.