Mark Zuckerberg Open Letter: Distillation Is Innocent, the Model Is About to Learn Everything

According to DONGCHA Beating's monitoring, Mark Zuckerberg today published a lengthy post openly defending model distillation. He believes that model learning from other models is inherently part of the open-source ecosystem. If the United States wants to maintain its lead in open-source AI, it must relax restrictions on distillation and training data.

He also directly refuted the claim that "distillation is harmful": if something is observable, it should be allowed to learn from. If the U.S. ties its own hands first, it will only allow other countries' open-source models to advance more quickly.

This stands in contrast to the recent actions of OpenAI and Anthropic. Both companies are taking strong measures to prevent competitors from bypassing restrictions and wholesale scraping of model outputs; however, Zuckerberg believes that this should not lead to restrictions on learning between models.

Meta's newly released Muse Glimmer today is itself distilled using Muse Spark.