AI Roundup 11/09 -> 11/16/2023

An OpenAI-powered wearable device, an easy way to remove RLHF protections from GPT-4, a model for navigating smartphones, and Microsoft's first-ever AI chips

Welcome to the weekly edition of AI Tidbits, where I curate the firehose of AI research papers and tools so you won’t have to.

---

Overview

• ✨ Highlights (8 entries)

• Research (20 entries)

• Announcements (2 entries)

• Cool tools (2 entries)

• Open-source (4 entries)

Highlights

1. Humane introduces the AI Pin - a $699 OpenAI-powered wearable device for everyday use (The Verge)

   

2. Researchers introduce JARVIS-1, an open-world agent that leverages multimodal memory and pre-trained language models to achieve breakthrough performances in Minecraft (Project website)

   

3. Researchers introduce LLaVA-Plus - a general-purpose multimodal assistant that excels in tool usage and interactive scenarios (Project website)

   

4. Adept announces Adept Experiments - a new program for users to explore Adept's models, along with ACT-2 - Adept's new multimodal model for UI understanding and action-taking (Adept)

   

5. UIUC and Stanford reveal that fine-tuning can effectively remove RLHF protections from LLMs like GPT-4 with a high success rate, highlighting a critical vulnerability in current AI safety measures (Paper)

   

   The Success rate of generating harmful content from GPT-3.5, GPT-4, and the fine-tuned version of GPT-4

6. Notion releases Q&A - allowing users to quickly retrieve information and receive AI-generated answers with citation links to relevant Notion pages (Company blog)

   

7. Microsoft announces Azure Maia 100 and Cobalt 100 - its first custom AI chips (The Verge)

   

8. DeepMind releases Graphcast - the world's most accurate global weather forecasting model delivering 10-day weather predications (DeepMind)

   

Research

1. Stanford and UNC reduce Llama-2's hallucinations by up to 58% without any human annotation

   

2. Google introduces Mirasol3B - a compact multimodal model that efficiently processes long audio, video, and text sequences achieving SOTA performance for long video question-answering

   

3. Researchers release OtterHD-8B - a Fuyu-inspired multimodal model capable of processing high-resolution inputs (1024x1024), showcasing exceptional detail recognition capabilities

   

4. UCSD and Microsoft release MM-Navigator - a large multimodal model to navigate iOS and Android phone

   

5. Researchers introduce ChatAnything, combining LLMs, text-to-speech, and image generation techniques to animate characters

   

6. USC and Microsoft introduce PE2 - a novel method for improving LLMs' prompt engineering abilities, leading to significant improvements in complex reasoning and task performance

7. Researchers introduce Everything of Thoughts (XoT) - a novel prompting approach that combines reinforcement learning and external domain knowledge, outperforming existing methods

8. Google and Yale release IFEval to evaluate LLMs' instruction-following capabilities

9. Helsinki's Silo AI unveils Poro 34B - an open-source, multilingual LLM, part of Europe's response to the English-dominant AI models like GPT-4

10. Microsoft releases Phi-2, a 2.7B parameter model with improved reasoning capabilities achieving comparable performance to Llama 2 7B, under a research license

    

11. Alibaba releases Qwen-Audio - a model that handles over 30 audio tasks (speech recognition and translation, audio-based Q&A, and more), achieving remarkable performance across diverse benchmarks without task-specific fine-tuning

    

12. CMU and Adobe introduce Music ControlNet, offering enhanced control over melody, dynamics, and rhythm in music generation, achieving higher fidelity to inputs compared to leading models with significantly fewer resources

13. UIUC, CMU, and Meta unveil an advanced robotic navigation system capable of locating and navigating to any object in unknown settings, achieving an 83% success rate in varied home environments

    

14. Stanford presents NOIR - an intelligent brain-robot interface that lets humans command robots for everyday tasks using brain signal

    

15. SenseTime proposes a comprehensive solution for generating human motion from text descriptions, integrating LLMs, motion semantics, and trajectory control

    

16. Researchers introduce One-2-3-45++, which fine tunes 2D diffusion models for quick and high-fidelity conversion of images to 3D meshes

    

17. Researchers introduce Instant3D - a novel text-to-3D generation method that produces high-quality 3D assets in 20 seconds

    

18. Researchers provide a comprehensive survey of advancements in coding LLMs

19. Microsoft evaluates GPT-4's capabilities in scientific discovery, revealing its proficiency in areas like drug discovery, computational chemistry, and PDE

20. Researchers explore the transformative impact of LLMs on mental health care, highlighting opportunities and risks across care levels

Announcements

1. Microsoft adds Mistral, Code Llama, Whisper, BLIP, and more to Azure AI

2. Together AI releases Together Inference Engine - the world's fastest, achieving unparalleled speeds of up to 171 tokens per second

   

Cool tools

1. DesignerGPT - create and host beautiful websites

2. ProPainter - edit videos using text

Open-source

1. draw-a-ui - turn mock-ups into a functioning html using GPT-4 Vision

2. LLaMA Factory - easy-to-use LLM fine-tuning framework (LLaMA, BLOOM, Mistral, etc.)

3. Fleet Context - a CLI tool and API chatbot over the top 1221 Python libraries

4. EmotiVoice 😊 - a text-to-speech engine with >2000 different voices, allowing emotional synthesis including happy, excited, and sad tonations

Last week’s AI Tidbits

AI Roundup 11/02-> 11/09/2023

Reach AI builders, researchers, and entrepreneurs by advertising in AI Tidbits

If you find AI Tidbits valuable, share it with a friend and consider showing your support.