Content-Aware Sparsity
Status: ActiveDates:
Stealth — details after publication
Projects
The work behind the work.
Active
Drafting, training, debugging. Each one is trying to surface.
Underground
Stalled, withdrawn, or didn't survive review. Each one fed the next.
-
Gradient Fan-in Asymmetry: The Structural Cause of Layer Redundancy in Deep Transformers
Status: ArchivedDates:
We argue that deep Transformer layer redundancy is caused by a structural information bottleneck in the gradient's path, not its raw magnitude. Causal interventions validate the mechanism, which we then leverage to build a superior pruning method and a more efficient, tapered architecture.
-
Self-Guided Framework for Improving Arithmetic Reasoning in Large Language Models with Reinforcement Learning
Status: ArchivedDates:
A self-guided reinforcement-learning framework that improves chain-of-thought arithmetic reasoning in LLMs by treating self-logicality across sampled rationales as the reward signal — no human-graded supervision required.
-
Dates:
By using a multiple-pivoting method to improve the quality of LLM output on low-resource languages, while performing a thorough analysis to better understand the workings of LLMs.