Ideezy

Overview

  • Sectors Telecommunications
  • Posted Jobs 0
  • Viewed 9

Company Description

Open-R1: a Completely Open Reproduction Of DeepSeek-R1

Hey there! This article is an introduction to the project, not a claim that we’ve replicated R1 yet. We’re integrating in the open, so as soon as we have assessment numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.

True, however it looks like there’s nothing to be evaluated since right now. I presume the ultimate objective is to train a new thinking design and then utilize the exact same examination metrics as o1 and the DeepSeek-R1.

Well, there ought to be at least some sanity check and validation to ensure the model was trained correctly.

Oh yes, if you are talking about the assessment number of deepseek’s design it’s coming soon!

As discussed in the article there is no model called Open-R1 to test at all … not yet anyhow. This is a blog describing that Hugging face will take the R1 Deepseek model, exercise how it was constructed as laid out in the paper and from what they released, and then reproduce that process.

in truth this is practically how science works … A develops a plan, discovery or development and it is tested by B, C and D to see if it is reproduceable. Thats been the cornerstone of research now for a few centuries.

This blog site is not stating they have already done so … Its a blog outlining an intent to start training a model like R1 and calling it Open-R1.

Also DeepSeek-R1 was just launched last week, and even in their paper they outlined the calculate hours required. While those are low calculate hours for a SOTA design this does not suggest you can train stated model in a week. I ‘d personally enjoy to be able to train a transformer model in a week, but we might need to wait a while for that level of calculate innovation.

So there are no standards for a design that has not been constructed yet right? As outlined in the blog site, and again in reply to your concern.

However fear not, there is a GitHub Repo currently and contributors (hell I may join myself), some prelim work done, and a master plan. A good starting position.

n
@edbeeching
has examined the released models already

( src: https://x.com/edwardbeeching/status/1884273209136275742)

R1 just trained on o1 outputs, so collectively …/ s. This is what the brand-new AI czars are saying

Hi! This article is an intro to the task, not a claim that we have actually recreated R1 yet. We will absolutely share the missing out on piece when we have them, you can anticipate the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

That’s nice and important to understand this remarkable hype that does not have technical comprehension and explanation. Science has to do with recreation, and if they claim to be open, let them fullfill the open part.

Please do release the training expense.

We will!

Excalidraw Hi n
@bojan2501
thanks, we will indeed be striving to make certain this training dish can work for little language models on consumer hardware since not everybody has a cluster of H100s in the house:-RRB- The tool we used for the images was Excalidraw! https://excalidraw.com

anticipating it! WTF are your speaking about?

need to be a joke

It’s truly cool to see how the whole open source neighborhood comes together!

Ops …

5.5 M is number press reporter in the deepseekv3 tech report (simply the training, not the experiment afaik), for R1 tough to estimate tbh however much less than 5.5 M imo

Historically, they have never launched code or datasets of their LLM training, so I would not expect this time to be different. If they would launch it that would be fantastic naturally!

Yes naturally!

So basically you’re asking to change existing censorship with another flavour of censorship?

The code for the designs are inside the design repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py

Hello Team, I’m Ray Bernard, the author and developer of EQUATOR. My research group will be working on a paper concentrated on reproducing certain components of DeepSeek R1. Our aim is to reproduce the cold start and provide your group with a dataset that consists of COT and other strategies to support these efforts. We like to contribute our work to help. Please let me understand if you discover this useful. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/

Where is the examination numbers? without it you can’t call it reproduction.

8 replies

True, but it appears like there’s absolutely nothing to be evaluated as of right now. I presume the ultimate goal is to train a brand-new reasoning design and after that use the same examination metrics as o1 and the DeepSeek-R1.

That’s quite interesting, I was asking myself why the questions the author exposed here are not being asked by others? I think the work they have actually done is memorable but at the exact same time I question why they would not put these missing out on pieces on if they are expected to be fully open.
Why even without reproduction and comprehension of the development they could impact a lot the market in this method?

4 replies

Hi! This post is an intro to the project, not a claim that we’ve reproduced R1 yet. We will absolutely share the missing piece when we have them, you can anticipate the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

Interesting read, and it is great that we see more effort into this direction: more and less brute force.
Also wonder what tool did the author usage for developing step diagram.

2 replies

Excalidraw I’m so glad that initiative like this already exist, I’m gon na attempt to contribute:-RRB- 1 reply

anticipating it! So racist articel

2 replies

WTF are your talking about?

Awesome to have this open reproduction began!

For Step # 1 check out https://github.com/open-thoughts/open-thoughts!

https://x.com/ryanmart3n/status/1884284101265612856

Let’s do this thing!

1 reply

It’s truly cool to see how the whole open source neighborhood comes together!

Does anybody understand the actual training cost of r1? I can’t discover it in the paper or the announcement post. Is the 6M cost reported by media just the number taken from v3’s training expense?

2 replies

Ops …

Has anybody asked the DeepSeek team to publish their training information and code, or at least share them independently with an independent replication project like this? Have they turned down such a demand?

A loyal replication depends upon using the very same dataset and hyperparameters. Otherwise, any significant inconsistencies with the released benchmarks would be difficult to pin down-whether due to training data distinctions or the replication method itself.

1 reply

Historically, they have never released code or datasets of their LLM training, so I would not expect this time to be different. If they would release it that would be amazing naturally!

In the meantime we have to make finest guess estimates and see if we can arrive ourselves.

You offer excellent duplication procedure of Deepseek reasoning training. I will try something comparable to it.

This is really good info, can we tweak with particular usage case when code is launched?

1 reply

Yes obviously!

Please consider getting rid of biased, polluted or unaligned training data and make an effort to eliminate copyrighted works from the crawl from intake. This will make the design more usable. If you recycled anthropic curation checks, this might likewise help, eliminate obviouslybiased information will likely add a great deal of worth. We don’t desire another tainted, unaligned open source design, right? And no business would ever utilize deepseek or a model that reuses it, right?
We appreciate your work for the advantage of humanity, we hope.
Miike C from NJ

1 reply

So generally you’re asking to replace existing censorship with another flavour of censorship?

Can’t wait! Hopefully the model will be uncensored however whatever you can do is alright! Love seeing open source building itself up. I’m not clever sufficient to in fact help however I can contribute support lol

Hello guys, I am even simply looking for code for DeepSeek-V2, in order to completely comprehend multi-head hidden attention. You do not appear to have code in Hugging Face even for that. Or am I missing something? Don’t see anything in src/transformers/models. MLA is not effectively described in their paper, so it would be necessary to have code for this.