School of Engineering Department of Computer Science and Engineering 99 Parameter-Efficient and Memory-Efficient Fine-Tuning Supervisor: CHEN Long / CSE Student: GAO Yitang / COMP Course: UROP 1100, Spring UROP 1000, Summer As transformer-based pretrained language models (PLMs) expand exponentially in terms of parameters, exemplified by large language models (LLMs) boasting billions of parameters, their application to natural language processing (NLP) tasks has achieved unprecedented success. However, despite these remarkable advancements, the enormous size and computational demands of these models present significant challenges when it comes to adapting them for specific downstream tasks, particularly in resource-constrained environments. Parameter Efficient Fine-Tuning (PEFT) serves as a highly effective solution, as it reduces the number of fine-tuning parameters and memory usage while maintaining performance. This report offers a comprehensive review of PEFT methods for PLMs, summarizing their various approaches and overall effectiveness in different scenarios. Deep Video Super-resolution Supervisor: CHEN Qifeng / CSE Student: DING Yiyi / COMP Course: UROP 1100, Fall Recent advancements in Large Language Models (LLMs) have demonstrated their potential in reasoning tasks. Autonomous LLM agents utilize these models to enhance problem-solving, planning, memorization, and tool-use capabilities. However, LLMs currently exhibit limited one-shot performance in arithmetic operations, symbolic comprehension, and common-sense reasoning, leading to issues of hallucination and inconsistency in their output. Prompt engineering techniques, which guide LLMs to mimic human thinking processes, have been proposed to improve the performance of autonomous LLM agents in complex reasoning tasks. This project aims to evaluate the effectiveness of various prompt engineering techniques in improving autonomous LLM agents’ performance on complex reasoning tasks, specifically focusing on arithmetic expressions. Furthermore, the project aims to identify the key factors that influence the performance. Deep Video Super-resolution Supervisor: CHEN Qifeng / CSE Student: FEI Yang / COSC Course: UROP 2100, Fall UROP 3100, Spring Text-to-video generation tasks are gaining popularity due to their remarkable outcomes. In this paper, we explore an advanced Video Variational Autoencoder (VAE) architecture, known as CausalVideoVAE, introduced by the Open-Sora project. Our training approach incorporates GAN loss and utilizes a larger, more diverse dataset. We stress the importance of properly managing training epochs and extensive video datasets to capture diverse features. Additionally, we suggest an innovative training strategy that balances temporal dynamics and detailed visual representations, offering potential improvements to Video VAE architectures and eventually facilitating more effective and efficient learning in video generation tasks.
RkJQdWJsaXNoZXIy NDk5Njg=