Hugging Face

Open-source AI strikes back with AuraFlow v0.1🤩 - Largest open-sourced flow-based T2I model with Apache 2.0 license - 6.8B parameters - DiT Encoder blocks - Better instruction following - GenEval score 0.703 with prompt enhancement 🥳 Congratulations to Simo, Batuhan & the whole fal ai team on this epic release!👏 Explore AuraFlow with the community Gradio demo: https://lnkd.in/gY27BETy Find the open-source AuraFlow model on Hugging Face Hub: https://lnkd.in/g4ttnJTw Read the fal ai official release blog: https://lnkd.in/gNiK9uYN

Lovely time speaking at EuroPython'24. Some really, really amazing talks and libraries I am taking back home. EuroPython, thanks for the opportunity 🇨🇿 Slides: https://lnkd.in/g8ENyECd

Today, I am super excited to introduce Lynx, the leading hallucination detection model! 🚀 Hallucinations make it really hard to ship AI products. To address this problem, developers have begun to use general-purpose LLMs like GPT-4 to evaluate other LLMs (‘LLM-as-a-judge’). However, general-purpose LLMs weren’t designed to be great at evaluation. Prior research shows that current LLM-as-a-judge approaches are unreliable and inconsistent. Enter Lynx. Lynx beats GPT-4o and all state-of-the-art LLMs on RAG hallucination tasks. And we’ve open sourced it ✨ You can use quantized Lynx-8B locally, use Lynx-70B with GPUs, or just reach out to Patronus AI for easy API access 😃 We are thrilled to launch Lynx with our Day 1 Integration Partners: NVIDIA, MongoDB, and Nomic AI. ⚡ At Patronus AI, our mission is to make high quality LLM evaluation accessible to everyone. The best is yet to come. Download quantized Lynx-8B on Hugging Face: https://lnkd.in/eaTpM3u6 Download Lynx-70B on Hugging Face: https://lnkd.in/eauKirMc Read the arXiv paper: https://lnkd.in/e--_R8Cg  Read our blog: https://lnkd.in/eJ4AgX8T Use Lynx with NVIDIA's Nemo Guardrails: https://lnkd.in/ec2yJFrZ Read about our Nomic AI Atlas integration: https://lnkd.in/eKxeq65D

Real Time Object Tracking with RT-DETR🔥 Simply upload a video, set up a confidence threshold, or try the examples given in the demo 👇 Demo: https://lnkd.in/gEXMka-2 Do read Merve Noyan's amazing post on LinkedIn with more details on RT-DETR model over here: https://lnkd.in/gH6Mqc4j

🚀Introducing LLaVA-NeXT Interleave: Now AI can understand and reason with multiple images at once! ⬇️Learn everything about Llava-Next-Interleave that you would ever need - Opens up multi-image scenarios like multi-frame videos, multi-view 3D, and multiple inter-leaved images. - An all round LMM that can understand videos, images, and 3D - Interleave data format unifies different tasks. - New datasets on 🤗Hub: 1️⃣M4-Instruct, high-quality dataset, 1.1M samples from domains: multi-image, video, 3D & single-image 2️⃣LLaVA-Interleave Bench - Set of tasks to evaluate multi-image capabilities LLaVA-NeXT-Interleave💪 - Attached video show how Llava-Interleave can explain jokes and understand content spread in multiple images and videos🤯 - Model has SoTA Performance, both, in multiple and single images - Matches in performance with LLaVA-NeXT for single image - Improved performance in video tasks Gradio Multimodal Demo for LLaVA-NeXT-Interleave😍 : https://lnkd.in/gAaPrQtT Models and Datasets are on 🤗@huggingface Hub: https://lnkd.in/g2JEj2Uv

It's possible to finetune a GPT-2 model😲 to directly predict the result of 15-digit multiplications without using any intermediate reasoning steps. [New Paper & Gradio implementation] Super cool demo from the authors of the paper "From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step"🔥 Research Paper: https://lnkd.in/ghCX94TJ Official Gradio demo: https://lnkd.in/gNiyewQZ

Google presents PaliGemma A versatile 3B VLM for transfer paper page: https://lnkd.in/eQevVDJh PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model. It is trained to be a versatile and broadly knowledgeable base model that is effective to transfer. It achieves strong performance on a wide variety of open-world tasks. We evaluate PaliGemma on almost 40 diverse tasks including standard VLM benchmarks, but also more specialized tasks such as remote-sensing and segmentation.

Tencent presents MiraData A Large-Scale Video Dataset with Long Durations and Structured Captions paper page: https://lnkd.in/eSSQ5XzQ Sora's high-motion intensity and long consistent videos have significantly impacted the field of video generation, attracting unprecedented attention. However, existing publicly available datasets are inadequate for generating Sora-like videos, as they mainly contain short videos with low motion intensity and brief captions. To address these issues, we propose MiraData, a high-quality video dataset that surpasses previous ones in video duration, caption detail, motion strength, and visual quality. We curate MiraData from diverse, manually selected sources and meticulously process the data to obtain semantically consistent clips. GPT-4V is employed to annotate structured captions, providing detailed descriptions from four different perspectives along with a summarized dense caption. To better assess temporal consistency and motion intensity in video generation, we introduce MiraBench, which enhances existing benchmarks by adding 3D consistency and tracking-based motion strength metrics. MiraBench includes 150 evaluation prompts and 17 metrics covering temporal consistency, motion strength, 3D consistency, visual quality, text-video alignment, and distribution similarity. To demonstrate the utility and effectiveness of MiraData, we conduct experiments using our DiT-based video generation model, MiraDiT. The experimental results on MiraBench demonstrate the superiority of MiraData, especially in motion strength.

The bleeding-edge alignment technique DPO for vision language models is now available in Hugging Face TRL along with LoRA/QLoRA ⚡️ Links and more in comments 🔖 DPO is a popular cutting-edge alignment technique for language models. TLDR; a (preference) model is trained using a dataset of inputs and chosen and rejected outputs, and this model generates scores for each input. the main model is fine-tuned using the scores. Essentially DPO in vision language models is pretty similar, since vision language models are models that take in images projected to text embedding space, it's just input tokens output tokens.  Quentin Gallouédec implemented support for Idefics2, Llava 1.5, and PaliGemma in TRL. 👏 as of now, VLM processors are quite non-standard, only difference is due to processor and chat templates themselves, you can implement it very easily (see his PR in links) Thanks to TRL's support for PEFT and bitsandbytes you can also try QLoRA and LoRA fine-tuning (which comes in blog post) 😏 Please try the scripts, share your models and let us know how it goes!

🌟Introducing Transcription Delight!🌟 Effortlessly Generate Transcripts from any YouTube video (or any uploaded video/audio)! This App is super cool 😎 & incredibly handy! 🛠 It also refines your transcript with an LLM, transforming it into a polished markdown output for your downstream needs😍 🤔For example, You can use the markdown transcript as input in Claude/GPT-4, and get ready to throw questions at it or summarize with ease!🚀📝 🔥Transcription Delight is an app created by Abubakar Abid🙌 -- Dive into this Gradio app and up your game! Try on Hugging Face Spaces: https://lnkd.in/gHFyHFbr OR Build the app locally in three lines of Code, by doing: 👉 𝚐𝚒𝚝 𝚌𝚕𝚘𝚗𝚎 𝚑𝚝𝚝𝚙𝚜://𝚑𝚞𝚐𝚐𝚒𝚗𝚐𝚏𝚊𝚌𝚎.𝚌𝚘/𝚜𝚙𝚊𝚌𝚎𝚜/𝚊𝚋𝚒𝚍𝚕𝚊𝚋𝚜/𝚝𝚛𝚊𝚗𝚜𝚌𝚛𝚒𝚙𝚝𝚒𝚘𝚗-𝚍𝚎𝚕𝚒𝚐𝚑𝚝 👉 𝚌𝚍 𝚝𝚛𝚊𝚗𝚜𝚌𝚛𝚒𝚙𝚝𝚒𝚘𝚗-𝚍𝚎𝚕𝚒𝚐𝚑𝚝 👉 𝚙𝚢𝚝𝚑𝚘𝚗 𝚊𝚙𝚙.𝚙𝚢