School of Engineering Department of Computer Science and Engineering 136 Logical Inference and Rationales in Large Foundation Models Supervisor: SONG Yangqiu / CSE Student: GAO Yitang / COMP Course: UROP 1100, Spring Large foundation models are neural networks with very many parameters that are pretrained on huge datasets. They can understand and generate natural language surprisingly well. However, giving these models logical inference skills and clear rationales is still a key challenge. Traditional AI uses symbolic methods—like explicit logic rules (e.g., weakening and contraction) or knowledge graphs. However, these methods struggle with the ambiguity and scale of real-world text. Recent work shows that neural methods can mimic logical reasoning. For example, Chain-of-Thought prompting. This survey reviews techniques for logical inference in NLP, from pretraining to fine-tuning. Logical Inference and Rationales in Large Foundation Models Supervisor: SONG Yangqiu / CSE Student: LI Chengxi / COMP Course: UROP 1100, Spring UROP 2100, Summer The rapid development of Large Language Models (LLMs) has significantly advanced automated programming, yet current models often struggle with complex, multi-step tasks requiring iterative refinement, adaptive search, and cross-domain generalization. This work investigates tree-based search strategies for orchestrating LLM-driven programming, integrating insights from AI-Driven Exploration (AIDE) and genetic programming approaches such as Genesys. We introduce a research framework aimed at aligning task characteristics with optimal search patterns, exploring dynamic node architectures, and enabling adaptive strategy modification based on real-time feedback. To evaluate performance, we examine both general coding benchmarks (LiveCodeBench) and domain-specific efficiency benchmarks (KernelBench), ensuring a comprehensive assessment across correctness, adaptability, and computational performance. Logical Inference and Rationales in Large Foundation Models Supervisor: SONG Yangqiu / CSE Student: LI Haohan / COGBM Course: UROP 1100, Spring During this semester’s undergraduate research program, under the supervision of professor Song Yangqiu and PhD Candidate Zheng Tianshi, I investigated how to adapt the BERT model (the base version with 110 million parameters) for various downstream tasks, including sentiment analysis (sentence classification), paraphrase detection (sentence pair classification), and semantic textual similarity (sentence-pair regression). I experimented with different architectures and methodologies to enhance multi-task performance, such as multitask BERT fine-tuning with mixed loss, task-specific heads using mean pooling and adaptive training for imbalanced datasets across tasks. The model is evaluated on the Stanford Sentiment Treebank (SST) dataset, the Quora dataset, and the SemEval STS Benchmark dataset, showing how combined methodologies can improve model performance.
RkJQdWJsaXNoZXIy NDk5Njg=