Author name cluster

Nathan Jacobs

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

1 author row

AAAI Conference 2026 Conference Paper

Beta Distribution Learning for Reliable Roadway Crash Risk Assessment

Ahmad Elallaf
Nathan Jacobs
Xinyue Ye
Mei Chen
Gongbo Liang

Roadway traffic accidents represent a global health crisis, responsible for over a million deaths annually and costing many countries up to 3% of their GDP. Traditional traffic safety studies often examine risk factors in isolation, overlooking the spatial complexity and contextual interactions inherent in the built environment. Furthermore, conventional Neural Network-based risk estimators typically generate point estimates without conveying model uncertainty, limiting their utility in critical decision-making. To address these shortcomings, we introduce a novel geospatial deep learning framework that leverages satellite imagery as a comprehensive spatial input. This approach enables the model to capture the nuanced spatial patterns and embedded environmental risk factors that contribute to fatal crash risks. Rather than producing a single deterministic output, our model estimates a full Beta probability distribution over fatal crash risk, yielding accurate and uncertainty-aware predictions--a critical feature for trustworthy AI in safety-critical applications. Our model outperforms baselines by achieving a 17-23% improvement in recall, a key metric for flagging potential dangers, while delivering superior calibration. By providing reliable and interpretable risk assessments from satellite imagery alone, our method enables safer autonomous navigation and offers a highly scalable tool for urban planners and policymakers to enhance roadway safety equitably and cost-effectively.

PDF Details DOI

AAMAS Conference 2026 Conference Paper

DiffVAS: Diffusion-Guided Visual Active Search in Partially Observable Environments

Anindya Sarkar
Srikumar Sastry
Aleksis Pirinen
Nathan Jacobs
Yevgeniy Vorobeychik

Visualactivesearch(VAS)hasbeenintroducedasamodelingframework that leverages visual cues to direct aerial (e. g. , UAV-based) exploration and pinpoint areas of interest within extensive geospatialregions. PotentialapplicationsofVASincludedetectinghotspots for rare wildlife poaching, aiding search-and-rescue missions, and uncovering illegal trafficking of weapons, among other uses. Previous VAS approaches assume that the entire search space is known upfront, which is often unrealistic due to constraints such as a restricted field of view and high acquisition costs, and they typically learn policies tailored to specific target objects, which limits their ability to search for multiple target categories simultaneously. In this work, we propose DiffVAS, a target-conditioned policy that searches for diverse objects simultaneously according to task requirements in partially observable environments, which advances the deployment of visual active search policies in real-world applications. DiffVAS leverages a diffusion model to reconstruct the entire geospatial area from sequentially observed partial glimpses, which enables a target-conditioned reinforcement learning-based planning module to effectively reason and guide subsequent search steps. Extensive experiments demonstrate that DiffVAS excels in searching diverse objects in partially observable environments, significantly surpassing state-of-the-art methods on several datasets. Code and models are available at this link.

PDF

TMLR Journal 2026 Journal Article

GroundingBooth: Grounding Text-to-Image Customization

Zhexiao Xiong
Wei Xiong
Jing Shi
He Zhang
Yizhi Song
Nathan Jacobs

Recent approaches in text-to-image customization have primarily focused on preserving the identity of the input subject, but often fail to control the spatial location and size of objects. We introduce GroundingBooth, which achieves zero-shot, instance-level spatial grounding on both foreground subjects and background objects in the text-to-image customization task. Our proposed grounding module and subject-grounded cross-attention layer enable the creation of personalized images with accurate layout alignment, identity preservation, and strong text-image coherence. In addition, our model seamlessly supports personalization with multiple subjects. Our model shows strong results in both layout-guided image synthesis and text-to-image customization tasks. The project page is available at https://groundingbooth.github.io.

PDF Details

AAAI Conference 2025 Conference Paper

Active Geospatial Search for Efficient Tenant Eviction Outreach

Anindya Sarkar
Alex DiChristofano
Sanmay Das
Patrick J. Fowler
Nathan Jacobs
Yevgeniy Vorobeychik

Tenant evictions threaten housing stability and are a major concern for many cities. An open question concerns whether data-driven methods enhance outreach programs that target at-risk tenants to mitigate their risk of eviction. We propose a novel active geospatial search (AGS) modeling framework for this problem. AGS integrates property-level information in a search policy that identifies a sequence of rental units to canvas to both determine their eviction risk and provide support if needed. We propose a hierarchical reinforcement learning approach to learn a search policy for AGS that scales to large urban areas containing thousands of parcels, balancing exploration and exploitation and accounting for travel costs and a budget constraint. Crucially, the search policy adapts online to newly discovered information about evictions. Evaluation using eviction data for a large urban area demonstrates that the proposed framework and algorithmic approach are considerably more effective at sequentially identifying eviction cases than baseline methods.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Fields of The World: A Machine Learning Benchmark Dataset for Global Agricultural Field Boundary Segmentation

Hannah Kerner
Snehal Chaudhari
Aninda Ghosh
Caleb Robinson
Adeel Ahmad
Eddie Choi
Nathan Jacobs
Chris Holmes

Crop field boundaries are foundational datasets for agricultural monitoring and assessments but are expensive to collect manually. Machine learning (ML) methods for automatically extracting field boundaries from remotely sensed images could help realize the demand for these datasets at a global scale. However, current ML methods for field instance segmentation lack sufficient geographic coverage, accuracy, and generalization capabilities. Further, research on improving ML methods is restricted by the lack of labeled datasets representing the diversity of global agricultural fields. We present Fields of The World (FTW)---a novel ML benchmark dataset for agricultural field instance segmentation spanning 24 countries on four continents (Europe, Africa, Asia, and South America). FTW is an order of magnitude larger than previous datasets with 70,462 samples, each containing instance and semantic segmentation masks paired with multi-date, multi-spectral Sentinel-2 satellite images. We provide results from baseline models for the new FTW benchmark, show that models trained on FTW have better zero-shot and fine-tuning performance in held-out countries than models that aren't pre-trained with diverse datasets, and show positive qualitative zero-shot results of FTW models in a real-world scenario -- running on Sentinel-2 scenes over Ethiopia.

PDF Details DOI

TMLR Journal 2025 Journal Article

Mixed-View Panorama Synthesis using Geospatially Guided Diffusion

Zhexiao Xiong
Xin Xing
Scott Workman
Subash Khanal
Nathan Jacobs

We introduce the task of mixed-view panorama synthesis, where the goal is to synthesize a novel panorama given a small set of input panoramas and a satellite image of the area. This contrasts with previous work which only uses input panoramas (same-view synthesis), or an input satellite image (cross-view synthesis). We argue that the mixed-view setting is the most natural to support panorama synthesis for arbitrary locations worldwide. A critical challenge is that the spatial coverage of panoramas is uneven, with few panoramas available in many regions of the world. We introduce an approach that utilizes diffusion-based modeling and an attention-based architecture for extracting information from all available input imagery. Experimental results demonstrate the effectiveness of our proposed method. In particular, our model can handle scenarios when the available panoramas are sparse or far from the location of the panorama we are attempting to synthesize.

PDF Details

NeurIPS Conference 2025 Conference Paper

QuARI: Query Adaptive Retrieval Improvement

Eric Xing
Abby Stylianou
Robert Pless
Nathan Jacobs

Massive-scale pretraining has made vision-language models increasingly popular for image-to-image and text-to-image retrieval across a broad collection of domains. However, these models do not perform well when used for challenging retrieval tasks, such as instance retrieval in very large-scale image collections. Recent work has shown that linear transformations of VLM features trained for instance retrieval can improve performance by emphasizing subspaces that relate to the domain of interest. In this paper, we explore a more extreme version of this specialization by learning to map a given query to a query-specific feature space transformation. Because this transformation is linear, it can be applied with minimal computational cost to millions of image embeddings, making it effective for large-scale retrieval or re-ranking. Results show that this method consistently outperforms state-of-the-art alternatives, including those that require many orders of magnitude more computation at query time.

PDF Details

AAMAS Conference 2024 Conference Paper

Geospatial Active Search for Preventing Evictions

Anindya Sarkar
Alex DiChristofano
Sanmay Das
Patrick J. Fowler
Nathan Jacobs
Yevgeniy Vorobeychik

Evictions are a threat to housing stability and a major concern for many cities. An open question is whether data-driven methods can enhance door-to-door outreach programs to target at-risk tenants. We model this problem using a new framework we term geospatial active search. Geospatial Active Search integrates visual information such as satellite imagery along with tabular data such as property and neighborhood-level information to create an online exploration plan. We develop an approach for the implementation of Geospatial Active Search in St. Louis to find properties containing tenants who will have an eviction filed against them.

PDF

NeurIPS Conference 2024 Conference Paper

GOMAA-Geo: GOal Modality Agnostic Active Geo-localization

Anindya Sarkar
Srikumar Sastry
Aleksis Pirinen
Chongjie Zhang
Nathan Jacobs
Yevgeniy Vorobeychik

We consider the task of active geo-localization (AGL) in which an agent uses a sequence of visual cues observed during aerial navigation to find a target specified through multiple possible modalities. This could emulate a UAV involved in a search-and-rescue operation navigating through an area, observing a stream of aerial images as it goes. The AGL task is associated with two important challenges. Firstly, an agent must deal with a goal specification in one of multiple modalities (e. g. , through a natural language description) while the search cues are provided in other modalities (aerial imagery). The second challenge is limited localization time (e. g. , limited battery life, urgency) so that the goal must be localized as efficiently as possible, i. e. the agent must effectively leverage its sequentially observed aerial views when searching for the goal. To address these challenges, we propose GOMAA-Geo -- a goal modality agnostic active geo-localization agent -- for zero-shot generalization between different goal modalities. Our approach combines cross-modality contrastive learning to align representations across modalities with supervised foundation model pretraining and reinforcement learning to obtain highly effective navigation and localization policies. Through extensive evaluations, we show that GOMAA-Geo outperforms alternative learnable approaches and that it generalizes across datasets -- e. g. , to disaster-hit areas without seeing a single disaster scenario during training -- and goal modalities -- e. g. , to ground-level imagery or textual descriptions, despite only being trained with goals specified as aerial views. Our code is available at: https: //github. com/mvrl/GOMAA-Geo.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

A Partially-Supervised Reinforcement Learning Framework for Visual Active Search

Anindya Sarkar
Nathan Jacobs
Yevgeniy Vorobeychik

Visual active search (VAS) has been proposed as a modeling framework in which visual cues are used to guide exploration, with the goal of identifying regions of interest in a large geospatial area. Its potential applications include identifying hot spots of rare wildlife poaching activity, search-and-rescue scenarios, identifying illegal trafficking of weapons, drugs, or people, and many others. State of the art approaches to VAS include applications of deep reinforcement learning (DRL), which yield end-to-end search policies, and traditional active search, which combines predictions with custom algorithmic approaches. While the DRL framework has been shown to greatly outperform traditional active search in such domains, its end-to-end nature does not make full use of supervised information attained either during training, or during actual search, a significant limitation if search tasks differ significantly from those in the training distribution. We propose an approach that combines the strength of both DRL and conventional active search approaches by decomposing the search policy into a prediction module, which produces a geospatial distribution of regions of interest based on task embedding and search history, and a search module, which takes the predictions and search history as input and outputs the search distribution. In addition, we develop a novel meta-learning approach for jointly learning the resulting combined policy that can make effective use of supervised information obtained both at training and decision time. Our extensive experiments demonstrate that the proposed representation and meta-learning frameworks significantly outperform state of the art in visual active search on several problem domains.

PDF Details

JBHI Journal 2022 Journal Article

Contrastive Cross-Modal Pre-Training: A General Strategy for Small Sample Medical Imaging

Gongbo Liang
Connor Greenwell
Yu Zhang
Xin Xing
Xiaoqin Wang
Ramakanth Kavuluru
Nathan Jacobs

A key challenge in training neural networks for a given medical imaging task is the difficulty of obtaining a sufficient number of manually labeled examples. In contrast, textual imaging reports are often readily available in medical records and contain rich but unstructured interpretations written by experts as part of standard clinical practice. We propose using these textual reports as a form of weak supervision to improve the image interpretation performance of a neural network without requiring additional manually labeled examples. We use an image-text matching task to train a feature extractor and then fine-tune it in a transfer learning setting for a supervised task using a small labeled dataset. The end result is a neural network that automatically interprets imagery without requiring textual reports during inference. We evaluate our method on three classification tasks and find consistent performance improvements, reducing the need for labeled data by 67%–98%.

Details DOI