Hacker News

2724 readers

248 users here now

Posts from the RSS Feed of HackerNews.

The feed sometimes contains ads and posts that have been removed by the mod team at HN.

founded 1 year ago

MODERATORS

patrick@lemmy.bestiver.se

rssbot@lemmy.bestiver.se

A $196 fine-tuned 7B model outperforms OpenAI o3 on document extraction (arxiv.org)

submitted 4 days ago by rssbot@lemmy.bestiver.se to c/hackernews@lemmy.bestiver.se

3 comments fedilink hide all child comments

Comments

you are viewing a single comment's thread
view the rest of the comments

[–] mindbleach@sh.itjust.works 1 points 3 days ago (2 children)

The supervised fine-tuning phase employed Low-Rank Adaptation (LoRA) to efficiently adapt the base DeepSeek- R1-Distill-Qwen-7B model for extraction tasks

So this is bolted on top of a model that cost six figures.

[–] Dionysus@leminal.space 1 points 3 days ago (1 children)

And deepseek is based on llama, more than six figures.

I'm not aware of any larger parameter LLMs not based on one which is absurdly expensive.

[–] mindbleach@sh.itjust.works 1 points 3 days ago

DeepSeek is trained from-scratch. Only some variants used other LLMs.

This is a megaphone made from string, a squirrel, and a megaphone.