School of Engineering Department of Computer Science and Engineering 126 Foundation Model-as-a-Service at Edge Supervisor: GUO Song / CSE Student: ZHOU Xintong / COMP Course: UROP 1000, Summer This report summarizes my UROP project, which investigated methods to enable voice assistants to interpret fuzzy or implicit natural-language expressions and translate them into concrete smart-home actions. During the summer I completed four principal tasks: a focused literature review on large-language-model (LLM) based autonomous agents; study and local deployment experiments with LangGraph; implementation of a voice-driven prototype that integrates streaming speech recognition and AI-based intent parsing; and continued coursework in machine learning to reinforce theoretical foundations. Technical outcomes comprise a modular prototype, deterministic fallback rules for degraded API conditions, and practical experience in prompt engineering, system integration, and robustness engineering. The report documents methods, implementation details, evaluation, and prospective directions for future work. Safe Diffusion Models for Robust AI Generation Supervisor: GUO Song / CSE Student: HUANG Hao / COMP Course: UROP 1100, Fall The report aims to generate high-quality images from a combination of input image and a given prompt by combining the structure of BLIP-Diffusion and Noise collage. This study is implementing experiments to explore if we could integrate these methodologies to enhance multimodal generation where both text and image modalities interact seamlessly. Noise Collage provides a novel layout-aware text-to-image diffusion model structure and helps avoid condition mismatches, while the BLIP-Diffusion model integrates pretrained vision-language models (BLIP-2) with diffusion frameworks to enable controllable text-to-image generation and subject-driven image editing. By combining the approaches, we are seeking to overcome challenges such as modality suppression, fidelity, and quality in multi-subject generation. We hope the proposed framework contributes to the development of multimodal generation and personalization. Safe Diffusion Models for Robust AI Generation Supervisor: GUO Song / CSE Student: LEE Pak Nin / COMP Course: UROP 1100, Spring This is the final report for the UROP project Safe Diffusion Models for Robust AI Generation (ProjectID: Fall2024-25-2051) under the supervision by Prof. GUO, Song. This project addresses critical safety challenges in deploying diffusion models for generative AI applications, such as healthcare and autonomous systems. Three main mitigation strategies were investigated in this project: pre-inference model unlearning to eliminate unsafe data patterns, on-inference concept editing to suppress harmful outputs during generation, and post-inference adversarial filtering to eliminate residual hazard. My contributions to this project include, reviewing literature, testing with diffusion models, prototyping an adversarial filter, and exploring concept editing techniques.
RkJQdWJsaXNoZXIy NDk5Njg=