Our QuickLLaMA beat StreamLLM series and established new SOTA on LongBench, InfiniteBench and Needle-in-a-Haystack task.

In this work, we find surprisingly that simply using reconstruction-based methods benefits the model in learning intrinsic data distributions of the ID dataset, which could boost the performance of OOD detection significantly.

In this study, we harness features acquired through self-supervised learning. We introduce a straightforward yet potent metric, Cluster Distance Difference, to identify diverse data and introduce a novel framework, Balancing Active Learning (BAL).

In our study, we conducted a comprehensive analysis, exploring distinct pretraining tasks and employing various OOD score functions. The results highlight that the feature representations pre-trained through reconstruction yield a notable enhancement and narrow the performance gap among various score functions.

We introduce a pioneering framework for Modular-of-Thought instruction tuning, designed to promote the decomposition of tasks into logical sub-tasks and sub-modules, significantly improving both the modularity and correctness of the generated solutions.

Our research develops a general-purpose robotic coding algorithm that enables robots to leverage basic skills to tackle increasingly complex tasks.

In our work, we disentangle the ill-posed optimization problem into two parallel processes: one performs semantic matching individually, and the other judges reliability for improving discrimination ability.

Our research aims to bridge the domain gap between natural and artificial scenarios with efficient tuning strategies through leveraging the potential of language models.