Post
4492
who want grug 35b?
- 5 replies
π In a Training Loop
Jorge Munoz Laredo
jorgemunozl
AI & ML interests
I like Vision Language Action Models, Machine Learning Inter atomic Potentials AI4Science, Diffusion based architectures and I love physics.
Recent Activity
new activity 1 day ago
context-course/unit_3_quiz:Cert. liked a model 2 days ago
lerobot/smolvla_base liked a Space 2 days ago
victor/boeing-747-kimi-k3Organizations
reacted to ProCreations's post with π 6 days ago
reacted to Quazim0t0's post with π 6 days ago
Post
4250
I never really posted about my DaisyChain project because it's still work in progress. I decided to post a small bit about it and the demo.
DaisyChain Genomics: four small DNA/RNA specialists chained behind a learned router that behave like one big genomics model, at ~7Γ less active compute.
I built a modular genomics model chasing a 500M-parameter foundation model, then caught myself measuring it wrong. Here's the honest version.
DaisyChain is a different bet: instead of one monolithic DNA model, it's four ~74M specialists (eukaryote, prokaryote, mRNA, splice) chained behind a learned router, each distilled per-domain from HuggingFaceBio's Carbon-500M. Every specialist reports how surprised it is (bits/base) and the router hands each sequence to the link most at home with it. In lineage it's a cluster Branch-Train-Merge mixture of experts, so you can chain on a new domain without retraining the others.
The pitch: ~295M total params (under Carbon-500M), but only one ~74M specialist runs per query, so ~7Γ cheaper per token, routing at 100% held-out.
The mistake: Carbon works in 6-mers, and I'd been scoring likelihood as 6-mer cross-entropy. By that number I was +0.043 bits/base behind, splice even "beating" Carbon. But Carbon scores at the base-pair level, which is harder and more honest. Re-run their way:
Real gap: 1.862 vs 1.787 bits/base, +0.089 behind, not +0.043
No domain actually beats Carbon; the "splice win" was an artifact
Seq recovery: euk 31.5% vs 38.9%, bacteria 40.9% vs 54.1%
DaisyChain is still behind Carbon-500M (itself a draft model, not built to top benchmarks), but by a number I can defend, and the gap closes with every per-domain pass. πΌ
DaisyChainAI
DaisyChainAI/Daisychain-Genomics-Demo
DaisyChainAI/daisychain-genomics
DaisyChain Genomics: four small DNA/RNA specialists chained behind a learned router that behave like one big genomics model, at ~7Γ less active compute.
I built a modular genomics model chasing a 500M-parameter foundation model, then caught myself measuring it wrong. Here's the honest version.
DaisyChain is a different bet: instead of one monolithic DNA model, it's four ~74M specialists (eukaryote, prokaryote, mRNA, splice) chained behind a learned router, each distilled per-domain from HuggingFaceBio's Carbon-500M. Every specialist reports how surprised it is (bits/base) and the router hands each sequence to the link most at home with it. In lineage it's a cluster Branch-Train-Merge mixture of experts, so you can chain on a new domain without retraining the others.
The pitch: ~295M total params (under Carbon-500M), but only one ~74M specialist runs per query, so ~7Γ cheaper per token, routing at 100% held-out.
The mistake: Carbon works in 6-mers, and I'd been scoring likelihood as 6-mer cross-entropy. By that number I was +0.043 bits/base behind, splice even "beating" Carbon. But Carbon scores at the base-pair level, which is harder and more honest. Re-run their way:
Real gap: 1.862 vs 1.787 bits/base, +0.089 behind, not +0.043
No domain actually beats Carbon; the "splice win" was an artifact
Seq recovery: euk 31.5% vs 38.9%, bacteria 40.9% vs 54.1%
DaisyChain is still behind Carbon-500M (itself a draft model, not built to top benchmarks), but by a number I can defend, and the gap closes with every per-domain pass. πΌ
DaisyChainAI/Daisychain-Genomics-Demo
DaisyChainAI/daisychain-genomics
reacted to Anran-MLLM's post with π₯ about 1 month ago
Post
3578
π Introducing PerceptionDLM β the first multimodal diffusion LLM for parallel region perception!
Most MLLMs are autoregressive, so captioning N regions costs N sequential passes. PerceptionDLM instead describes ALL masked regions in a single denoising process. π§©
β¨ Highlights
β’ β‘ Up to 3.4Γ faster on dense multi-region captioning, with stable per-image latency
β’ π PerceptionDLM-Base beats LLaDA-V on 15/16 multimodal benchmarks (new SOTA among open diffusion VLMs)
β’ π New benchmark: ParaDLC-Bench β jointly evaluates caption quality AND inference efficiency
β’ π Code, models & benchmark all open-sourced
π€ Models
MSALab/PerceptionDLM-Base
MSALab/PerceptionDLM
π Benchmark
MSALab/ParaDLC-Bench
π Paper: PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models (2606.19534)
π» Code: https://github.com/MSALab-PKU/PerceptionDLM
Diffusion LLMs aren't just for text β they unlock efficient, parallel visual perception. ποΈβ¨
#multimodal #diffusion #VLM #perception
Most MLLMs are autoregressive, so captioning N regions costs N sequential passes. PerceptionDLM instead describes ALL masked regions in a single denoising process. π§©
β¨ Highlights
β’ β‘ Up to 3.4Γ faster on dense multi-region captioning, with stable per-image latency
β’ π PerceptionDLM-Base beats LLaDA-V on 15/16 multimodal benchmarks (new SOTA among open diffusion VLMs)
β’ π New benchmark: ParaDLC-Bench β jointly evaluates caption quality AND inference efficiency
β’ π Code, models & benchmark all open-sourced
π€ Models
MSALab/PerceptionDLM-Base
MSALab/PerceptionDLM
π Benchmark
MSALab/ParaDLC-Bench
π Paper: PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models (2606.19534)
π» Code: https://github.com/MSALab-PKU/PerceptionDLM
Diffusion LLMs aren't just for text β they unlock efficient, parallel visual perception. ποΈβ¨
#multimodal #diffusion #VLM #perception
awesome, tough I ask a double pendulum and it failed, but pretty awesome to be honest
reacted to ovi054's post with β€οΈ about 1 month ago
Post
3618
Qwen3-14B Manim Expert LoRA
For "Build Small Hackathon", I built a Gradio app that turns any concept into a Manim explainer video.
This is powered by Qwen3-14B + Manim LoRA I trained on a synthetic 10k dataset I generated.
π Try it now: build-small-hackathon/anim-vid-ai
For "Build Small Hackathon", I built a Gradio app that turns any concept into a Manim explainer video.
This is powered by Qwen3-14B + Manim LoRA I trained on a synthetic 10k dataset I generated.
π Try it now: build-small-hackathon/anim-vid-ai
- 2 replies
reacted to dealermatt72's post with π 3 months ago
Post
6728
Hey Hugging Face community π
My name is M. I'm a solo founder and self-taught developer based in Houston, TX. I build AI-powered apps β I have an iOS app called DeFilter currently in App Store review, a security scanning platform called Sentinel, and a job marketplace called HireHuman.fyi for connecting humans with companies that prefer non-AI workers.
I'm also a poker dealer by night, which means I think a lot about reading situations in real time β and that's exactly what sparked this idea.
I'm not the most technical person in the room. But I have a vision, I have drive, and I believe the best projects get built when people with different skills come together around a shared idea.
That's why I'm posting here. I want to build this with the community.
β M (@dealermatt )
My name is M. I'm a solo founder and self-taught developer based in Houston, TX. I build AI-powered apps β I have an iOS app called DeFilter currently in App Store review, a security scanning platform called Sentinel, and a job marketplace called HireHuman.fyi for connecting humans with companies that prefer non-AI workers.
I'm also a poker dealer by night, which means I think a lot about reading situations in real time β and that's exactly what sparked this idea.
I'm not the most technical person in the room. But I have a vision, I have drive, and I believe the best projects get built when people with different skills come together around a shared idea.
That's why I'm posting here. I want to build this with the community.
β M (@dealermatt )
- 3 replies
reacted to philipp-zettl's post with π 3 months ago
Post
2685
I've been cooking something neat over the past weeks π¨βπ³
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around Β―\_(γ)_/Β―
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
Final models get exported in
- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around Β―\_(γ)_/Β―
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
----
Currently available: 1112735744, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6744 MB / 7805 MB
Data on GPU: 1641 MB
Grads on GPU: 3459 MB
CPU Offloaded: 18230 MB
---------------------------------
Currently available: 1079181312, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6776 MB / 7805 MB
Data on GPU: 1561 MB
Grads on GPU: 3279 MB
CPU Offloaded: 18590 MB
-----------------------------Final models get exported in
safetensors format and are compatible with PyTorch and transformers, for accessibility.- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory
- 1 reply
reacted to MikeDoes's post with π₯ 5 months ago
Post
1147
Are you sure the open-source model you just downloaded is safe?
A recent paper on "Privacy Backdoors" reports a new vulnerability where pre-trained models can be poisoned before fine-tuning them. This is a serious challenge for everyone building on open-source AI.
Instead of just pointing out problems, we believe in finding better solutions. To understand this threat, the researchers needed to test their attack on realistic data structures. They needed a dataset that could effectively simulate a high-stakes privacy attack, and we're proud that our Ai4Privacy dataset was used to provide this crucial benchmark. The paper reports that for our complex dataset, the privacy leakage on a non-poisoned model was almost zero. After the backdoor attack, that number reportedly jumped to 87%.
Ai4Privacy dataset provided a realistic benchmark for their research. Our dataset, composed of synthetic identities, helped them demonstrate how a poisoned model could dramatically amplify privacy leakage.
This is why we champion open source: it enables the community to identify these issues and develop better, safer solutions together.
Kudos to the authors Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini, University of Maryland and Google DeepMind.
π Read the research to understand this new challenge: https://arxiv.org/pdf/2404.01231
π Stay updated on the latest in privacy-preserving AIβfollow us on LinkedIn: https://www.linkedin.com/company/ai4privacy/posts/
A recent paper on "Privacy Backdoors" reports a new vulnerability where pre-trained models can be poisoned before fine-tuning them. This is a serious challenge for everyone building on open-source AI.
Instead of just pointing out problems, we believe in finding better solutions. To understand this threat, the researchers needed to test their attack on realistic data structures. They needed a dataset that could effectively simulate a high-stakes privacy attack, and we're proud that our Ai4Privacy dataset was used to provide this crucial benchmark. The paper reports that for our complex dataset, the privacy leakage on a non-poisoned model was almost zero. After the backdoor attack, that number reportedly jumped to 87%.
Ai4Privacy dataset provided a realistic benchmark for their research. Our dataset, composed of synthetic identities, helped them demonstrate how a poisoned model could dramatically amplify privacy leakage.
This is why we champion open source: it enables the community to identify these issues and develop better, safer solutions together.
Kudos to the authors Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini, University of Maryland and Google DeepMind.
π Read the research to understand this new challenge: https://arxiv.org/pdf/2404.01231
π Stay updated on the latest in privacy-preserving AIβfollow us on LinkedIn: https://www.linkedin.com/company/ai4privacy/posts/
replied to MonsterMMORPG's post 6 months ago
Yeah, it seem pretty nice !
reacted to MonsterMMORPG's post with π 6 months ago
Post
1650
LTX 2 & Z Image Base Full Tutorial + Audio to Video Lip Sync + ComfyUI + SwarmUI + Windows + Cloud
Full tutorial link > https://www.youtube.com/watch?v=SkXrYezeEDc
Info
LTX 2 is the newest state of the art (SOTA) Open Source video generation model and tutorial will show you how to use it with very best and most performant way in ComfyUI and also in SwarmUI. Moreover, Z Image Base model published and I will show how to use Z Image Base with most amazing preset and workflow as well. Furthermore, this tutorial will show you how to install, update, setup, download ComfyUI and SwarmUI and models and presets and workflows both on Windows and on RunPod, Massed Compute and SimplePod. Linux users can use Massed Compute scripts and installers directly. This is a masterpiece entire lecture level complete tutorial. This video will kickstart your AI journey 100x. Both local Windows and Cloud.
45 Second Raw Demo Video
This video made with text + image + audio = lip synched and animated video at once
See video below
Full tutorial link > https://www.youtube.com/watch?v=SkXrYezeEDc
Info
LTX 2 is the newest state of the art (SOTA) Open Source video generation model and tutorial will show you how to use it with very best and most performant way in ComfyUI and also in SwarmUI. Moreover, Z Image Base model published and I will show how to use Z Image Base with most amazing preset and workflow as well. Furthermore, this tutorial will show you how to install, update, setup, download ComfyUI and SwarmUI and models and presets and workflows both on Windows and on RunPod, Massed Compute and SimplePod. Linux users can use Massed Compute scripts and installers directly. This is a masterpiece entire lecture level complete tutorial. This video will kickstart your AI journey 100x. Both local Windows and Cloud.
45 Second Raw Demo Video
This video made with text + image + audio = lip synched and animated video at once
See video below
- 3 replies
posted an update 6 months ago
Post
482
just published a short article about something that bit me hard while porting PI05βs subtask prediction to PyTorch: left vs right alignment in transformer padding.
turns out JAX (what Physical Intelligence used) and Hugging Face use opposite padding conventions β and if you donβt catch it, your model silently produces nonsense instead of crashing. no NaN, no error, just garbled subtasks π€‘
i walk through the full tensor pipeline β images β embeddings β pad masks β attention masks β position IDs β and show exactly where the mismatch corrupts everything. also included the implementation file with the fix.
if youβve ever ported a model between frameworks or messed with custom attention patterns, i think you will enjoy it
turns out JAX (what Physical Intelligence used) and Hugging Face use opposite padding conventions β and if you donβt catch it, your model silently produces nonsense instead of crashing. no NaN, no error, just garbled subtasks π€‘
i walk through the full tensor pipeline β images β embeddings β pad masks β attention masks β position IDs β and show exactly where the mismatch corrupts everything. also included the implementation file with the fix.
if youβve ever ported a model between frameworks or messed with custom attention patterns, i think you will enjoy it
- 1 reply
reacted to hassenhamdi's post with π₯ 6 months ago
Post
2083
Google published the paper. I shipped the code. π
DeepMind just released PACEvolve (Progress-Aware Consistent Evolution), a massive overhaul of the AlphaEvolve framework. It solves the critical issues of "Context Pollution" and "Mode Collapse" that have historically crippled evolutionary coding agents.
But there was no public implementation. So I built one.
Introducing OpenPACEvolve: A fully open-source, production-grade implementation of the PACEvolve framework.
π I engineered this framework solo, but I wasn't working alone. I orchestrated a custom coding agents powered by Claude Opus 4.5 as Engineer and Gemini Pro 3 Preview ensuring fiedelity and quallty.
By leveraging these SOTA models, I was able to translate complex theoretical research into functional, modular Python architecture in record time. This is what the future of AI engineering looks like: Human architectural oversight + AI velocity.
π§ What OpenPACEvolve Solves: Unlike standard agents that get "stuck" in loops, this framework implements the paper's full recipe for long-horizon stability: β Hierarchical Context Management (HCM): Bi-level pruning to keep the agent's memory clean. β Momentum-Based Backtracking (MBB): Uses "power-law backtracking" to detect stagnation and force pivots. β Self-Adaptive Crossover: Intelligent code-sharing between parallel "islands."
π¨βπ» This project is more than a repo; it's a demonstration of rapid research-to-production cycles using next-gen AI workflows.
π Link of the paper : https://arxiv.org/abs/2601.10657
The code is live. The agents are ready. Check out the repository below. π
https://github.com/hassenhamdi/OpenPACEvolve
Star the repo π.
DeepMind just released PACEvolve (Progress-Aware Consistent Evolution), a massive overhaul of the AlphaEvolve framework. It solves the critical issues of "Context Pollution" and "Mode Collapse" that have historically crippled evolutionary coding agents.
But there was no public implementation. So I built one.
Introducing OpenPACEvolve: A fully open-source, production-grade implementation of the PACEvolve framework.
π I engineered this framework solo, but I wasn't working alone. I orchestrated a custom coding agents powered by Claude Opus 4.5 as Engineer and Gemini Pro 3 Preview ensuring fiedelity and quallty.
By leveraging these SOTA models, I was able to translate complex theoretical research into functional, modular Python architecture in record time. This is what the future of AI engineering looks like: Human architectural oversight + AI velocity.
π§ What OpenPACEvolve Solves: Unlike standard agents that get "stuck" in loops, this framework implements the paper's full recipe for long-horizon stability: β Hierarchical Context Management (HCM): Bi-level pruning to keep the agent's memory clean. β Momentum-Based Backtracking (MBB): Uses "power-law backtracking" to detect stagnation and force pivots. β Self-Adaptive Crossover: Intelligent code-sharing between parallel "islands."
π¨βπ» This project is more than a repo; it's a demonstration of rapid research-to-production cycles using next-gen AI workflows.
π Link of the paper : https://arxiv.org/abs/2601.10657
The code is live. The agents are ready. Check out the repository below. π
https://github.com/hassenhamdi/OpenPACEvolve
Star the repo π.
reacted to jzhang533's post with π₯ 8 months ago
Post
3302
Weβve officially kicked off the ERNIE AI Developer Challenge!
We want to create something interesting with you all, so we partnered with Unsloth, LLaMA-Factory, Novita AI, D-Robotics, and CAMEL-AI to empower your creativity.
Come build with us: https://baiduernieai.devpost.com/?utm_source=ERNIE-HF&utm_medium=ERNIE-HF&utm_campaign=ERNIE+AI+Developer+Challenge
We want to create something interesting with you all, so we partnered with Unsloth, LLaMA-Factory, Novita AI, D-Robotics, and CAMEL-AI to empower your creativity.
Come build with us: https://baiduernieai.devpost.com/?utm_source=ERNIE-HF&utm_medium=ERNIE-HF&utm_campaign=ERNIE+AI+Developer+Challenge
reacted to cjerzak's post with π 8 months ago
Post
2907
>>> We're writing a new book, <Planetary Causal Inference>, on how to model counterfactuals at planetary scale by combining satellite imagery + other global data with local studies and RCTs. Forthcoming in 2026+.
>>> Book info: https://planetarycausalinference.org/book-launch
>>> All datasets used in the book will be openly available on our labβs Hugging Face hub:
theaidevlab
>>> Book info: https://planetarycausalinference.org/book-launch
>>> All datasets used in the book will be openly available on our labβs Hugging Face hub:
reacted to codelion's post with π 9 months ago
Post
3625
On this day in 2019, OpenAI released the final GPT-2 model as part of their staged release. I still remember that November well - so much was happening, but GPT-2's release felt like a watershed moment for the field. It showed us what was possible with carefully trained language models.
To recreate some of that GPT-2 magic, I recently tackled an interesting challenge: can you pretrain a language model with just 1 billion tokens - roughly 1/10th of what GPT-2 used - and still get comparable performance? After 50+ systematic experiments testing different dataset mixtures, the answer is yes.
The result is codelion/gpt-2-70m, which achieves over 90% of GPT-2's benchmark performance despite being trained on 10x less data. The key was finding the optimal dataset composition: 50% high-quality textbook PDFs, 30% filtered web content, and 20% educational resources. It even beats GPT-2 on TruthfulQA (47.31% vs 40.69%).
If you're interested in the full story of how we discovered this optimal mixture and why curriculum learning catastrophically failed, check out the complete article: https://huggingface.co/blog/codelion/optimal-dataset-mixing
Sometimes less really is more - when you mix it right.
To recreate some of that GPT-2 magic, I recently tackled an interesting challenge: can you pretrain a language model with just 1 billion tokens - roughly 1/10th of what GPT-2 used - and still get comparable performance? After 50+ systematic experiments testing different dataset mixtures, the answer is yes.
The result is codelion/gpt-2-70m, which achieves over 90% of GPT-2's benchmark performance despite being trained on 10x less data. The key was finding the optimal dataset composition: 50% high-quality textbook PDFs, 30% filtered web content, and 20% educational resources. It even beats GPT-2 on TruthfulQA (47.31% vs 40.69%).
If you're interested in the full story of how we discovered this optimal mixture and why curriculum learning catastrophically failed, check out the complete article: https://huggingface.co/blog/codelion/optimal-dataset-mixing
Sometimes less really is more - when you mix it right.
- 1 reply
reacted to davidquicast's post with π 9 months ago
Post
2901
Just applied for HF Community Grant for βHugging Researchβ β a lightweight CodeAgentβbased research assistant built on Hugging Faceβs Open Deep Research project for the Hugging Face Hub (models, datasets, Spaces, users, collections, papers). It gathers links via dedicated tools and organizes them for easy review.
As this is for the community, comments and suggestions are appreciated: https://huggingface.co/spaces/daqc/hugging-research/discussions/1#68a94d9bcb035c54bc671119
As this is for the community, comments and suggestions are appreciated: https://huggingface.co/spaces/daqc/hugging-research/discussions/1#68a94d9bcb035c54bc671119
reacted to MonsterMMORPG's post with β€οΈ 11 months ago
Post
7429
I have concluded first 8 traininings of Qwen Image LoRA - we are not at the level of FLUX yet and next 8 trainings starting hopefully - 2656x2656px image generated with 8 steps Fast Qwen LoRA + myself trained LoRA :
Grid test results shared here along with App installer : https://www.patreon.com/posts/137551634
Grid test results shared here along with App installer : https://www.patreon.com/posts/137551634
reacted to mrs83's post with π 11 months ago
Post
2866
Introducing the Computer Says No Dataset: ethicalabs/computer-says-no
An LLM can do almost anything, but should it?
This dataset provides clear examples of when LLMs should decline requests, such as:
- Counting characters (e.g., "number of 'r's in 'raspberry'" β seriously, youβve got this)
- Solving basic equations (like *5.9 = x + 5.11* β please, show that calculator some love)
Inspired by Little Britain's iconic "Computer Says No" sketch, we address a critical issue in AI systems today: the waste of using a rocket launcher to swat flies (aka powerful models for trivial tasks).
Goals:
- Reduce waste by saving compute for tasks that actually need it
- Guide users to better tools
- Spark discussion about ethical AI
This isnβt a training set. Itβs a provocation: if we donβt define AI's limits, who will?
An LLM can do almost anything, but should it?
This dataset provides clear examples of when LLMs should decline requests, such as:
- Counting characters (e.g., "number of 'r's in 'raspberry'" β seriously, youβve got this)
- Solving basic equations (like *5.9 = x + 5.11* β please, show that calculator some love)
Inspired by Little Britain's iconic "Computer Says No" sketch, we address a critical issue in AI systems today: the waste of using a rocket launcher to swat flies (aka powerful models for trivial tasks).
Goals:
- Reduce waste by saving compute for tasks that actually need it
- Guide users to better tools
- Spark discussion about ethical AI
This isnβt a training set. Itβs a provocation: if we donβt define AI's limits, who will?
- 9 replies
reacted to pcuenq's post with π₯ 11 months ago
Post
10371
OpenELM in Core ML
Apple recently released a set of efficient LLMs in sizes varying between 270M and 3B parameters. Their quality, according to benchmarks, is similar to OLMo models of comparable size, but they required half the pre-training tokens because they use layer-wise scaling, where the number of attention heads increases in deeper layers.
I converted these models to Core ML, for use on Apple Silicon, using this script: https://gist.github.com/pcuenca/23cd08443460bc90854e2a6f0f575084. The converted models were uploaded to this community in the Hub for anyone that wants to integrate inside their apps: corenet-community/openelm-core-ml-6630c6b19268a5d878cfd194
The conversion was done with the following parameters:
- Precision: float32.
- Sequence length: fixed to 128.
With swift-transformers (https://github.com/huggingface/swift-transformers), I'm getting about 56 tok/s with the 270M on my M1 Max, and 6.5 with the largest 3B model. These speeds could be improved by converting to
I'm also looking at optimizing inference using an experimental kv cache in swift-transformers. It's a bit tricky because the layers have varying number of attention heads, but I'm curious to see how much this feature can accelerate performance in this model family :)
Regarding the instruct fine-tuned models, I don't know the chat template that was used. The models use the Llama 2 tokenizer, but the Llama 2 chat template, or the default Alignment Handbook one that was used to train, are not recognized. Any ideas on this welcome!
Apple recently released a set of efficient LLMs in sizes varying between 270M and 3B parameters. Their quality, according to benchmarks, is similar to OLMo models of comparable size, but they required half the pre-training tokens because they use layer-wise scaling, where the number of attention heads increases in deeper layers.
I converted these models to Core ML, for use on Apple Silicon, using this script: https://gist.github.com/pcuenca/23cd08443460bc90854e2a6f0f575084. The converted models were uploaded to this community in the Hub for anyone that wants to integrate inside their apps: corenet-community/openelm-core-ml-6630c6b19268a5d878cfd194
The conversion was done with the following parameters:
- Precision: float32.
- Sequence length: fixed to 128.
With swift-transformers (https://github.com/huggingface/swift-transformers), I'm getting about 56 tok/s with the 270M on my M1 Max, and 6.5 with the largest 3B model. These speeds could be improved by converting to
float16. However, there's some precision loss somewhere and generation doesn't work in float16 mode yet. I'm looking into this and will keep you posted! Or take a look at this issue if you'd like to help: https://github.com/huggingface/swift-transformers/issues/95I'm also looking at optimizing inference using an experimental kv cache in swift-transformers. It's a bit tricky because the layers have varying number of attention heads, but I'm curious to see how much this feature can accelerate performance in this model family :)
Regarding the instruct fine-tuned models, I don't know the chat template that was used. The models use the Llama 2 tokenizer, but the Llama 2 chat template, or the default Alignment Handbook one that was used to train, are not recognized. Any ideas on this welcome!
- 5 replies