Yuanxin Ouyang’s research while affiliated with Beihang University and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (130)


Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt
  • Preprint

May 2025

·

1 Read

Keqin Peng

·

Liang Ding

·

Yuanxin Ouyang

·

[...]

·

Dacheng Tao

Reasoning Large Language Models (RLLMs) have demonstrated impressive performance on complex tasks, largely due to the adoption of Long Chain-of-Thought (Long CoT) reasoning. However, they often exhibit overthinking -- performing unnecessary reasoning steps even after arriving at the correct answer. Prior work has largely focused on qualitative analyses of overthinking through sample-based observations of long CoTs. In contrast, we present a quantitative analysis of overthinking from the perspective of self-doubt, characterized by excessive token usage devoted to re-verifying already-correct answer. We find that self-doubt significantly contributes to overthinking. In response, we introduce a simple and effective prompting method to reduce the model's over-reliance on input questions, thereby avoiding self-doubt. Specifically, we first prompt the model to question the validity of the input question, and then respond concisely based on the outcome of that evaluation. Experiments on three mathematical reasoning tasks and four datasets with missing premises demonstrate that our method substantially reduces answer length and yields significant improvements across nearly all datasets upon 4 widely-used RLLMs. Further analysis demonstrates that our method effectively minimizes the number of reasoning steps and reduces self-doubt.


Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding

February 2025

·

2 Reads

Large language models (LLMs) excel at a range of tasks through in-context learning (ICL), where only a few task examples guide their predictions. However, prior research highlights that LLMs often overlook input-label mapping information in ICL, relying more on their pre-trained knowledge. To address this issue, we introduce In-Context Contrastive Decoding (ICCD), a novel method that emphasizes input-label mapping by contrasting the output distributions between positive and negative in-context examples. Experiments on 7 natural language understanding (NLU) tasks show that our ICCD method brings consistent and significant improvement (up to +2.1 improvement on average) upon 6 different scales of LLMs without requiring additional training. Our approach is versatile, enhancing performance with various demonstration selection methods, demonstrating its broad applicability and effectiveness. The code and scripts will be publicly released.










Citations (55)


... Sequential pattern mining and sequence modeling techniques [26,30] have been employed to address sequential data in course recommendations. Additionally, heterogeneous information network analysis and knowledge graph techniques [7,49] can be utilized to process relational data for course recommendations. ...

Reference:

A hierarchical attention neural network with multi-view fusion for online course recommendation
Course Recommendation System Based on Course Knowledge Graph Generated by Large Language Models
  • Citing Conference Paper
  • December 2024

... In-Context Learning (ICL) [1] has become a pivotal method for enhancing LLM performance, enabling models to effectively perform specific tasks by including demonstration samples in prompts without requiring additional training [2]. However, ICL faces several limitations: performance varies considerably depending on the order and selection of demonstration samples [3][4][5], the maximum context length constraint of LLMs makes it challenging to handle tasks involving long-context reasoning or diverse demonstration sets, and processing numerous demonstration significantly reduces computational efficiency [6,7]. ...

Revisiting Demonstration Selection Strategies in In-Context Learning
  • Citing Conference Paper
  • January 2024

... Recent studies demonstrate LLMs can enhance learning experiences across various scenarios [6,17]. Particularly, recent studies have explored AI applications, including personalized suggestions [32], and learning behavior analysis [3]. ...

A Review of Data Mining in Personalized Education: Current Trends and Future Prospects
  • Citing Article
  • March 2024

Frontiers of Digital Education

... A graph refinement, also known as graph rewiring, is crucial for further enhancing the expressiveness of the graph structure, reducing noisy elements, and mitigating potential bottleneck issues while modeling it with Graph Neural Networks (GNNs) (Akansha 2024;Li et al. 2024b). This adoption is still in its infancy in GNNs for brain disease classification. ...

Addressing Over-Squashing in GNNs with Graph Rewiring and Ordered Neurons
  • Citing Conference Paper
  • June 2024

... Effective prompt design has been shown to improve the quality of GenAI translations. This includes integrating contextual information about the source text (Peng et al., 2023) and specifying the translation's purpose and target audience (Yamada, 2023). With reference to previous prompt designs for translation tasks (Calvo-Ferrer, 2023;He, 2024;Tian, 2023), this study developed and tested six prompts. ...

Towards Making the Most of ChatGPT for Machine Translation
  • Citing Preprint
  • March 2023

... Not only do these challenges necessitate adaptive strategies for event organizers, but they also present opportunities for innovation in sustainability . Despite the increasing relevance of this topic, the intersection of climate risks, low-carbon policies, and sports economics remains underexplored, highlighting the need for comprehensive analytical frameworks that assess both the risks and economic implications of climate action in the sports industry (Peng et al., 2023). ...

Towards Making the Most of ChatGPT for Machine Translation
  • Citing Conference Paper
  • January 2023

... If left unaddressed, popularity bias tends to worsen over time [8], leading to the notorious Matthew Effect [11]. To mitigate popularity bias, previous studies have employed various approaches such as inverse propensity score (IPS) [12,13], regularization [14], causal graphs [15][16][17][18][19], and debiased contrastive loss [20,21]. The IPS approach assigns lower weights to more popular items, but it does not always work well in practice due to its high sensitivity to the weighting strategy. ...

PopDCL: Popularity-aware Debiased Contrastive Loss for Collaborative Filtering
  • Citing Conference Paper
  • Full-text available
  • October 2023

... Aligning questions with the syllabus is important for test focus, leading to studies training classifiers or ranking models to determine question relevance [32,33]. Personalized education demands generating customized questions for students, prompting the development of knowledgetracking models based on student answer histories or few-shot knowledge-tracking models incorporating sequences of student states and questions [34,35]. However, none of the previous work in Question Generation was presented in Turkish. ...

Multi-Source Soft Labeling and Hard Negative Sampling for Retrieval Distractor Ranking
  • Citing Article
  • January 2023

IEEE Transactions on Learning Technologies

... For example, these methods can't capture fine-grained item attributes, thus may recommend a pink dress for the user, while the user's actual intent is a blue and cotton dress. Some previous work has demonstrated the actual preferences of the user may be more complex including multiple dimensions [26][27][28]. Therefore, we regard that it is necessary to model and align user preference reflected in their behaviors from the perspective of fine-grained item attributes. ...

Multi-level and Multi-interest User Interest Modeling for News Recommendation
  • Citing Chapter
  • August 2023

Lecture Notes in Computer Science

... Their training objectives are achieved by indiscriminately minimizing the output distributions between the student and teacher model for each token. In fact, due to the token imbalance nature (Piantadosi, 2014) and the truth that different tokens contribute differently to the sentence meaning (Chen et al., 2020), adaptively reweighting the token-level loss would promote the model training, as evidenced by its effectiveness in sequence-tosequence training (Zhang et al., 2022;Peng et al., 2023b). It motivates us to speculate that indiscriminately adopting the same distillation mode to each token might be sub-optimal. ...

Token-Level Self-Evolution Training for Sequence-to-Sequence Learning
  • Citing Conference Paper
  • January 2023