School of Engineering Department of Computer Science and Engineering 114 Designing Conversational Agents for Neurocognitive Disorders Screening Supervisor: MA Xiaojuan / CSE Student: LIANG Danxuan / COMP ZENG Yuhang / DSCT Course: UROP 3200, Spring UROP 3200, Spring This report focuses on the complete automation of the HK-GSDT (The Hong Kong Grocery Shopping Dialog Task) based on the previous system, up to the Wizard of Oz stage. With the designed framework of scaffolding questions, prompts, and the UI for subject, our goal is to enable a dialogue process between the subject and the CA without the need for human intervention. This report includes the detailed implementation process of removing the existing wizard and automating their respective duties. Additionally, the report outlines a systematic research to verify the validity of the automated system by re-enacting the NCD screening task using previous HK-GSDT video data in the system. Handling User Challenge in Human-agent Interaction Supervisor: MA Xiaojuan / CSE Student: LAM Yeung Kong Sunny / COMP Course: UROP 1100, Summer In this research project, we intend to build one of the key parts for an innovative agent system based on socially acceptable robot interaction in human environments. The agent utilizes the knowledge graphgrounded large language models (LLMs) and affordance-based reasoning for the interaction between the robot and human. Central to this approach is the concept of affordance - The potential actions or uses an object or interface element suggests to users based on its design and context, guiding intuitive interaction without explicit instructions (IxDF 2016). The agent operates in three main steps. First, it constructs a knowledge graph based in a physical environment. Second, it monitors and updates dynamically the graph to reflect the object’s transformation in the environment. Lastly, it will be given a task and retrieves relevant knowledge from the graph to ground an LLM to generate appropriate solution of affordance for the interaction task. Handling User Challenge in Human-agent Interaction Supervisor: MA Xiaojuan / CSE Student: LO Wang Ip / DSCT Course: UROP 1100, Summer Posture estimation has been proposed since the 1980s and 1990s and it has become more mature nowadays. There are many object detection algorithms, such as Mediapipe and VLM (Vision Language Model) that can detect human posture and objects in a given photo or video source accurately. In order to achieve various practical tasks, it is essential to design posture recognition algorithms. During this study, I will design a posture recognition algorithm to distinguish sitting, lying and standing posture for real-time video. In this report, I will introduce the target of the project, the details for what I have done during these months, results of the tasks and what I have done well and not.
RkJQdWJsaXNoZXIy NDk5Njg=