Perplexity improves AI self-distillation: not only training students, but also training teachers.

Beating AI News Flash: In large model self-distillation training, the same model plays two roles: the student solves problems independently, while the teacher can see the reference answers in advance and provides guidance token by token along the content written by the student.

The two roles share parameters, eliminating the need to run an additional larger teacher model, and can obtain more fine-grained training signals than final scoring. However, the extra information held by the teacher may also lead it to adopt reasoning paths that the student cannot complete independently, which may instead hinder learning.

In June this year, teams including UCLA proposed SDPG, which retains the final result reward and only allows solution processes that perform above the group average to receive distillation guidance, then gradually reduces the distillation weight in the later stages of training to avoid excessive imitation hindering the model's own exploration. If a group of attempts all fail, the distillation signal is turned off; the same applies when all are answered correctly.

JOLT, newly proposed by Perplexity and NVIDIA, brings the teacher into training as well. The teacher uses extra information to solve problems independently and receives rewards based on final results; at the same time, KL divergence is used to constrain the output difference between teacher and student, preventing guidance from deviating too far from the student's current ability. The student still follows its own solution process and receives teacher feedback token by token. The two roles share parameters, and problem-solving ability and teaching style are adjusted together during training.

In the MATH test with Qwen3-0.6B, JOLT achieved an accuracy of 72.14%, higher than the previous self-distillation method OPSD's 61.20%, and also exceeding reinforcement learning GRPO's 68.36%. This set of JOLT experiments did not add a separate final result reward for the student; only the teacher branch directly used the result reward.

To reach the same 63.44% accuracy, GRPO was measured to consume about 8.4 million training tokens, while JOLT is estimated based on the training curve to require only 1.3 million, a difference of about 6.5 times.

However, when switched to the AppWorld agent task, JOLT without using the student result reward scored only 51.92%, lower than GRPO's 62.20%. Only after adding the final task reward for the student as well did JOLT+ rise to 72.09%. The step-by-step guidance provided by the teacher still cannot fully replace the feedback from the student actually completing the task.