17 Covering 70% of the Earth’s surface, the oceans are rich in biodiversity and play a pivotal role in regulating the global climate, yet they remain largely unexplored and unknown due to their vastness and inaccessibility.The analysis and interpretation of marine imagery have garnered increasing attention from both computer vision and marine biology communities. However, developing a foundation model for marine visual analysis poses considerable challenges, primarily due to the scarcity of labeled data and the discrepancies between underwater photographs and general in-air images. To address these challenges, the research team led by Prof. SaiKitYEUNG, Professor of the Division of Integrative Systems and Design, has developed MarineInst, a groundbreaking foundation model for the analysis of the marine realms with instance visual description, enabling effective segmentation and description of marine object instances, alongside the largest marine image dataset to date. Building on their previous project, MarineGPT—the first visionlanguage model specifically for the marine domain with extensive marine knowledge, the research team has created MarineInst20M, a dataset featuring a wide spectrum of marine images with highquality semantic instance masks.These marks are constructed by a mixture of human-annotated instance masks and model-generated instance masks from the team’s automatic procedure of binary instance filtering, totaling 2.42 million images and 19.2 million masks. With this extensive marine image dataset, MarineInst facilitates the generation of meaningful and comprehensive semantic captions faithful to each generated instance mask.The dataset and model support a wide range of marine visual analysis tasks, from image-level scene understanding to regional mask-level instance understanding.This new model exhibits strong generalization ability and flexibility to support various downstream tasks with state-of-the-art performance. This project paves the way for future marine image analysis, helping scientists unveil the mysteries of the oceans, and leveraging technology to advance our understanding of marine ecosystems and conservation. Division of Integrative Systems and Design New Foundation Model for Marine Image Analysis Advances Understanding of Marine Ecosystems and Conservation
RkJQdWJsaXNoZXIy NDk5Njg=