Zefan Wang

EAAI Journal 2025 Journal Article

A novel framework for assessing determinant risk factors on cyber (dis)trust behaviors of netizens in deepfakes

• Milad Taleby Ahvanooey
• Wojciech Mazurczyk
• Zefan Wang
• Jun Zhao

NeurIPS Conference 2025 Conference Paper

GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution

• Fengxiang Wang
• Mingshuo Chen
• Yueying Li
• Di Wang
• Haotian Wang
• Zonghao Guo
• Zefan Wang
• Shan Boqi

Ultra-high-resolution (UHR) remote sensing (RS) imagery offers valuable data for Earth observation but pose challenges for existing multimodal foundation models due to two key bottlenecks: (1) limited availability of UHR training data, and (2) token explosion caused by the large image size. To address data scarcity, we introduce **SuperRS-VQA** (avg. 8, 376$\times$8, 376) and **HighRS-VQA** (avg. 2, 000$\times$1, 912), the highest-resolution vision-language datasets in RS to date, covering 22 real-world dialogue tasks. To mitigate token explosion, our pilot studies reveal significant redundancy in RS images: crucial information is concentrated in a small subset of object-centric tokens, while pruning background tokens (e. g. , ocean or forest) can even improve performance. Motivated by these findings, we propose two strategies: *Background Token Pruning* and *Anchored Token Selection*, to reduce the memory footprint while preserving key semantics. Integrating these techniques, we introduce **GeoLLaVA-8K**, the first RS-focused multimodal large language model capable of handling inputs up to 8K$\times$8K resolution, built on the LLaVA framework. Trained on SuperRS-VQA and HighRS-VQA, GeoLLaVA-8K sets a new state-of-the-art on the XLRS-Bench. Datasets and code were released at https: //github. com/MiliLab/GeoLLaVA-8K.

ICLR Conference 2024 Conference Paper

Explaining Time Series via Contrastive and Locally Sparse Perturbations

• Zichuan Liu
• Yingying Zhang
• Tianchun Wang
• Zefan Wang
• Dongsheng Luo
• Mengnan Du
• Min Wu 0008
• Yi Wang 0022

Explaining multivariate time series is a compound challenge, as it requires identifying important locations in the time series and matching complex temporal patterns. Although previous saliency-based methods addressed the challenges, their perturbation may not alleviate the distribution shift issue, which is inevitable especially in heterogeneous samples. We present ContraLSP, a locally sparse model that introduces counterfactual samples to build uninformative perturbations but keeps distribution using contrastive learning. Furthermore, we incorporate sample-specific sparse gates to generate more binary-skewed and smooth masks, which easily integrate temporal trends and select the salient features parsimoniously. Empirical studies on both synthetic and real-world datasets show that ContraLSP outperforms state-of-the-art models, demonstrating a substantial improvement in explanation quality for time series data. The source code is available at \url{https://github.com/zichuan-liu/ContraLSP}.

NeurIPS Conference 2024 Conference Paper

Protecting Your LLMs with Information Bottleneck

• Zichuan Liu
• Zefan Wang
• Linjie Xu
• Jinyu Wang
• Lei Song
• Tianchun Wang
• Chunlin Chen
• Wei Cheng

The advent of large language models (LLMs) has revolutionized the field of natural language processing, yet they might be attacked to produce harmful content. Despite efforts to ethically align LLMs, these are often fragile and can be circumvented by jailbreaking attacks through optimized or manual adversarial prompts. To address this, we introduce the Information Bottleneck Protector (IBProtector), a defense mechanism grounded in the information bottleneck principle, and we modify the objective to avoid trivial solutions. The IBProtector selectively compresses and perturbs prompts, facilitated by a lightweight and trainable extractor, preserving only essential information for the target LLMs to respond with the expected answer. Moreover, we further consider a situation where the gradient is not visible to be compatible with any LLM. Our empirical evaluations show that IBProtector outperforms current defense methods in mitigating jailbreak attempts, without overly affecting response quality or inference speed. Its effectiveness and adaptability across various attack methods and target LLMs underscore the potential of IBProtector as a novel, transferable defense that bolsters the security of LLMs without requiring modifications to the underlying models.