Arrow Research search

Author name cluster

Ryosuke Shibasaki

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

31 papers
2 author rows

Possible papers

31

NeurIPS Conference 2025 Conference Paper

Continuous Domain Generalization

  • Zekun CAI
  • Yiheng YAO
  • Guangji Bai
  • Renhe Jiang
  • Xuan Song
  • Ryosuke Shibasaki
  • Liang Zhao

Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains as discrete or as evolving along a single axis (e. g. , time). This oversimplification fails to capture the complex, multidimensional nature of real-world variation. This paper introduces the task of Continuous Domain Generalization (CDG), which aims to generalize predictive models to unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on a low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables structure-preserving parameter transitions by enforcing geometric continuity and algebraic consistency. To handle noisy or incomplete domain variation descriptors, we introduce a gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.

PDF Details

NeurIPS Conference 2024 Conference Paper

Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation

  • Jiawei Wang
  • Renhe Jiang
  • Chuang Yang
  • Zengqing Wu
  • Makoto Onizuka
  • Ryosuke Shibasaki
  • Noboru Koshizuka
  • Chuan Xiao

This paper introduces a novel approach using Large Language Models (LLMs) integrated into an agent framework for flexible and effective personal mobility generation. LLMs overcome the limitations of previous models by effectively processing semantic data and offering versatility in modeling various tasks. Our approach addresses three research questions: aligning LLMs with real-world urban mobility data, developing reliable activity generation strategies, and exploring LLM applications in urban mobility. The key technical contribution is a novel LLM agent framework that accounts for individual activity patterns and motivations, including a self-consistency approach to align LLMs with real-world activity data and a retrieval-augmented strategy for interpretable activity generation. We evaluate our LLM agent framework and compare it with state-of-the-art personal mobility generation approaches, demonstrating the effectiveness of our approach and its potential applications in urban mobility. Overall, this study marks the pioneering work of designing an LLM agent framework for activity generation based on real-world human activity data, offering a promising tool for urban mobility analysis.

PDF Details DOI

AIJ Journal 2024 Journal Article

Learning spatio-temporal dynamics on mobility networks for adaptation to open-world events

  • Zhaonan Wang
  • Renhe Jiang
  • Hao Xue
  • Flora D. Salim
  • Xuan Song
  • Ryosuke Shibasaki
  • Wei Hu
  • Shaowen Wang

As a decisive part in the success of Mobility-as-a-Service (MaaS), spatio-temporal dynamics modeling on mobility networks is a challenging task particularly considering scenarios where open-world events drive mobility behavior deviated from the routines. While tremendous progress has been made to model high-level spatio-temporal regularities with deep learning, most, if not all of the existing methods are neither aware of the dynamic interactions among multiple transport modes on mobility networks, nor adaptive to unprecedented volatility brought by potential open-world events. In this paper, we are therefore motivated to improve the canonical spatio-temporal network (ST-Net) from two perspectives: (1) design a heterogeneous mobility information network (HMIN) to explicitly represent intermodality in multimodal mobility; (2) propose a memory-augmented dynamic filter generator (MDFG) to generate sequence-specific parameters in an on-the-fly fashion for various scenarios. The enhanced event-aware spatio-temporal network, namely EAST-Net, is evaluated on several real-world datasets with a wide variety and coverage of open-world events. Both quantitative and qualitative experimental results verify the superiority of our approach compared with the state-of-the-art baselines. What is more, experiments show generalization ability of EAST-Net to perform zero-shot inference over different open-world events that have not been seen.

Details DOI

AAAI Conference 2022 Conference Paper

Event-Aware Multimodal Mobility Nowcasting

  • Zhaonan Wang
  • Renhe Jiang
  • Hao Xue
  • Flora D. Salim
  • Xuan Song
  • Ryosuke Shibasaki

As a decisive part in the success of Mobility-as-a-Service (MaaS), spatio-temporal predictive modeling for crowd movements is a challenging task particularly considering scenarios where societal events drive mobility behavior deviated from the normality. While tremendous progress has been made to model high-level spatio-temporal regularities with deep learning, most, if not all of the existing methods are neither aware of the dynamic interactions among multiple transport modes nor adaptive to unprecedented volatility brought by potential societal events. In this paper, we are therefore motivated to improve the canonical spatio-temporal network (ST-Net) from two perspectives: (1) design a heterogeneous mobility information network (HMIN) to explicitly represent intermodality in multimodal mobility; (2) propose a memory-augmented dynamic filter generator (MDFG) to generate sequence-specific parameters in an on-the-fly fashion for various scenarios. The enhanced event-aware spatiotemporal network, namely EAST-Net, is evaluated on several real-world datasets with a wide variety and coverage of societal events. Both quantitative and qualitative experimental results verify the superiority of our approach compared with the state-of-the-art baselines. Code and data are published on https: //github. com/underdoc-wang/EAST-Net.

PDF Details

TIST Journal 2022 Journal Article

Predicting Citywide Crowd Dynamics at Big Events: A Deep Learning System

  • Renhe Jiang
  • Zekun CAI
  • Zhaonan Wang
  • Chuang Yang
  • Zipei Fan
  • Quanjun Chen
  • Xuan Song
  • Ryosuke Shibasaki

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g., police) and public service operators (e.g., subway/bus operator) to protect people’s safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the “deep” trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent, which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

Details DOI

AAAI Conference 2021 Conference Paper

Social-DPF: Socially Acceptable Distribution Prediction of Futures

  • Xiaodan Shi
  • Xiaowei Shao
  • Guangming Wu
  • Haoran Zhang
  • Zhiling Guo
  • Renhe Jiang
  • Ryosuke Shibasaki

We consider long-term path forecasting problems in crowds, where future sequence trajectories are generated given a short observation. Recent methods for this problem have focused on modeling social interactions and predicting multi-modal futures. However, it is not easy for machines to successfully consider social interactions, such as avoiding collisions while considering the uncertainty of futures under a highly interactive and dynamic scenario. In this paper, we propose a model that incorporates multiple interacting motion sequences jointly and predicts multi-modal socially acceptable distributions of futures. Specifically, we introduce a new aggregation mechanism for social interactions, which selectively models long-term inter-related dynamics between movements in a shared environment through a message passing mechanism. Moreover, we propose a loss function that not only accesses how accurate the estimated distributions of the futures are but also considers collision avoidance. We further utilize mixture density functions to describe the trajectories and learn multi-modality of future paths. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to forecast socially acceptable distributions in complex scenarios.

PDF Details

AAAI Conference 2020 Conference Paper

Multimodal Interaction-Aware Trajectory Prediction in Crowded Space

  • Xiaodan Shi
  • Xiaowei Shao
  • Zipei Fan
  • Renhe Jiang
  • Haoran Zhang
  • Zhiling Guo
  • Guangming Wu
  • Wei Yuan

Accurate human path forecasting in complex and crowded scenarios is critical for collision avoidance of autonomous driving and social robots navigation. It still remains as a challenging problem because of dynamic human interaction and intrinsic multimodality of human motion. Given the observation, there is a rich set of plausible ways for an agent to walk through the circumstance. To address those issues, we propose a spatio-temporal model that can aggregate the information from socially interacting agents and capture the multimodality of the motion patterns. We use mixture density functions to describe the human path and predict the distribution of future paths with explicit density. To integrate more factors to model interacting people, we further introduce a coordinate transformation to represent the relative motion between people. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to forecast various plausible futures in complex scenarios and achieves state-of-the-art performance.

PDF Details

AAAI Conference 2018 Conference Paper

DeepUrbanMomentum: An Online Deep-Learning System for Short-Term Urban Mobility Prediction

  • Renhe Jiang
  • Xuan Song
  • Zipei Fan
  • Tianqi Xia
  • Quanjun Chen
  • Satoshi Miyazawa
  • Ryosuke Shibasaki

Big human mobility data are being continuously generated through a variety of sources, some of which can be treated and used as streaming data for understanding and predicting urban dynamics. With such streaming mobility data, the online prediction of short-term human mobility at the city level can be of great significance for transportation scheduling, urban regulation, and emergency management. In particular, when big rare events or disasters happen, such as large earthquakes or severe traffic accidents, people change their behaviors from their routine activities. This means people’s movements will almost be uncorrelated with their past movements. Therefore, in this study, we build an online system called DeepUrban- Momentum to conduct the next short-term mobility predictions by using (the limited steps of) currently observed human mobility data. A deep-learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data for a huge urban area. Experimental results demonstrate the superior performance of our proposed model as compared to the existing approaches. Lastly, we apply our system to a real emergency scenario and demonstrate that our system is applicable in the real world.

PDF Details

IJCAI Conference 2016 Conference Paper

A Collaborative Filtering Approach to Citywide Human Mobility Completion from Sparse Call Records

  • Zipei Fan
  • Ayumi Arai
  • Xuan Song
  • Apichon Witayangkurn
  • Hiroshi Kanasugi
  • Ryosuke Shibasaki

Most of human mobility big datasets available by now, for example call detail records or twitter data with geotag, are always sparse and heavily biased. As a result, using such kind of data to directly represent real-world human mobility is unreliable and problematic. However, difficult though it is, a completion of human mobility turns out to be a promising way to minimize the issues of sparsity and bias. In this paper, we model the completion problem as a recommender system and therefore solve this problem in a collaborative filtering (CF) framework. We propose a spatio-temporal CF that simultaneously infers the topic distribution over users, time-of-days, days as well as locations, and then use the topic distributions to estimate a posterior over locations and infer the optimal location sequence in a Hidden Markov Model considering the spatio-temporal continuity. We apply and evaluate our algorithm using a real-world Call Detail Records dataset from Bangladesh and gives an application on Dynamic Census, which incorporates the survey data from cell phone users to generate an hourly population distribution with attributes.

PDF Details

IJCAI Conference 2016 Conference Paper

DeepTransport: Prediction and Simulation of Human Mobility and Transportation Mode at a Citywide Level

  • Xuan Song
  • Hiroshi Kanasugi
  • Ryosuke Shibasaki

Traffic congestion causes huge economic loss worldwide in every year due to wasted fuel, excessive air pollution, lost time, and reduced productivity. Understanding how humans move and select the transportation mode throughout a large-scale transportation network is vital for urban congestion prediction and transportation scheduling. In this study, we collect big and heterogeneous data (e. g. , GPS records and transportation network data), and we build an intelligent system, namely DeepTransport, for simulating and predicting human mobility and transportation mode at a citywide level. The key component of DeepTransport is based on the deep learning architecture that that aims to understand human mobility and transportation patterns from big and heterogeneous data. Based on the learning model, given any time period, specific location of the city or people's observed movements, our system can automatically simulate or predict the persons' future movements and their transportation mode in the large-scale transportation network. Experimental results and validations demonstrate the efficiency and superior performance of our system, and suggest that human transportation mode may be predicted and simulated more easily than previously thought.

PDF Details

AAAI Conference 2016 Conference Paper

Learning Deep Representation from Big and Heterogeneous Data for Traffic Accident Inference

  • Quanjun Chen
  • Xuan Song
  • Harutoshi Yamada
  • Ryosuke Shibasaki

With the rapid development of urbanization and public transportation system, the number of traffic accidents have significantly increased globally over the past decades and become a big problem for human society. Facing these possible and unexpected traffic accidents, understanding what causes traffic accident and early alarms for some possible ones will play a critical role on planning effective traffic management. However, due to the lack of supported sensing data, research is very limited on the field of updating traffic accident risk in real-time. Therefore, in this paper, we collect big and heterogeneous data (7 months traffic accident data and 1.6 million users’ GPS records) to understand how human mobility will affect traffic accident risk. By mining these data, we develop a deep model of Stack denoise Autoencoder to learn hierarchical feature representation of human mobility. And these features are used for efficient prediction of traffic accident risk level. Once the model has been trained, our model can simulate corresponding traffic accident risk map with given real-time input of human mobility. The experimental results demonstrate the efficiency of our model and suggest that traffic accident risk can be significantly more predictable through human mobility.

PDF Details

TIST Journal 2016 Journal Article

Prediction and Simulation of Human Mobility Following Natural Disasters

  • Xuan Song
  • Quanshi Zhang
  • Yoshihide Sekimoto
  • Ryosuke Shibasaki
  • Nicholas Jing Yuan
  • Xing Xie

In recent decades, the frequency and intensity of natural disasters has increased significantly, and this trend is expected to continue. Therefore, understanding and predicting human behavior and mobility during a disaster will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, such research is very difficult to perform owing to the uniqueness of various disasters and the unavailability of reliable and large-scale human mobility data. In this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users 1 over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data) to study human mobility following natural disasters. An empirical analysis is conducted to explore the basic laws governing human mobility following disasters, and an effective human mobility model is developed to predict and simulate population movements. The experimental results demonstrate the efficiency of our model, and they suggest that human mobility following disasters can be significantly more predictable and be more easily simulated than previously thought.

Details DOI

AAAI Conference 2015 Conference Paper

A Simulator of Human Emergency Mobility Following Disasters: Knowledge Transfer from Big Disaster Data

  • Xuan Song
  • Quanshi Zhang
  • Yoshihide Sekimoto
  • Ryosuke Shibasaki
  • Nicholas Jing Yuan
  • Xing Xie

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, understanding and simulating of human emergency mobility following disasters will become the critical issue for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, due to the uniqueness of various disasters and the unavailability of reliable and large scale human mobility data, such kind of research is very difficult to be performed. Hence, in this paper, we collect big and heterogeneous data (e. g. 1. 6 million users’ GPS records in three years, 17520 times of Japan earthquake data in four years, news reporting data, transportation network data and etc.) to capture and analyze human emergency mobility following different disasters. By mining these big data, we aim to understand what basic laws govern human mobility following disasters, and develop a general model of human emergency mobility for generating and simulating large amount of human emergency movements. The experimental results and validations demonstrate the efficiency of our simulation model, and suggest that human mobility following disasters may be significantly more predictable and can be easier simulated than previously thought.

PDF Details

TIST Journal 2015 Journal Article

From RGB-D Images to RGB Images

  • Quanshi Zhang
  • Xuan Song
  • Xiaowei Shao
  • Huijing Zhao
  • Ryosuke Shibasaki

Mining object-level knowledge, that is, building a comprehensive category model base, from a large set of cluttered scenes presents a considerable challenge to the field of artificial intelligence. How to initiate model learning with the least human supervision (i.e., manual labeling) and how to encode the structural knowledge are two elements of this challenge, as they largely determine the scalability and applicability of any solution. In this article, we propose a model-learning method that starts from a single-labeled object for each category, and mines further model knowledge from a number of informally captured, cluttered scenes. However, in these scenes, target objects are relatively small and have large variations in texture, scale, and rotation. Thus, to reduce the model bias normally associated with less supervised learning methods, we use the robust 3D shape in RGB-D images to guide our model learning, then apply the properly trained category models to both object detection and recognition in more conventional RGB images. In addition to model training for their own categories, the knowledge extracted from the RGB-D images can also be transferred to guide model learning for a new category, in which only RGB images without depth information in the new category are provided for training. Preliminary testing shows that the proposed method performs as well as fully supervised learning methods.

Details DOI

AAAI Conference 2014 Conference Paper

Intelligent System for Urban Emergency Management during Large-Scale Disaster

  • Xuan Song
  • Quanshi Zhang
  • Yoshihide Sekimoto
  • Ryosuke Shibasaki

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, urban emergency management has become the especially important issue for the whole governments around the world. In this paper, we present a novel intelligent system for urban emergency management during the large-scale disasters. The proposed system stores and manages the global positioning system (GPS) records from mobile devices used by approximately 1. 6 million people throughout Japan over one year. By mining and analyzing population movements after the Great East Japan Earthquake, our system can automatically learn a probabilistic model to better understand and simulate human mobility during the emergency situations. Based on the learning model, population mobility in various urban areas impacted by the earthquake throughout Japan can be automatically simulated or predicted. On the basis of such kind of system, it is easy for us to find some new features or population mobility patterns after the recent and unprecedented composite disasters, which are likely to provide valuable experience and play a vital role for future disaster management worldwide.

PDF Details

ICRA Conference 2014 Conference Paper

Start from minimum labeling: Learning of 3D object models and point labeling from a large and complex environment

  • Quanshi Zhang
  • Xuan Song 0001
  • Xiaowei Shao
  • Huijing Zhao
  • Ryosuke Shibasaki

A large category model base can provide object-level knowledge for various perception tasks of the intelligent vehicle system. The automatic and efficient construction of such a model base is highly desirable but challenging. This paper presents a novel semi-supervised approach to discover possible prototype models of 3D object structures from the point cloud of a large and complex environment, given a limited number of seeds in an object category. Our method incrementally trains the models while simultaneously collecting object samples. Considering the bias problem of model learning caused by bias accumulation in a sample collection, we propose to gradually differentiate the standard category model into several sub-category models to represent different intra-category structural styles. Thus, new sub-categories are discovered and modeled, old models are improved, and redundant models for similar structures are deleted iteratively during the learning process. This multiple-model strategy provides several interactive options for the category boundary to deal with the bias problem. Experimental results demonstrate the effectiveness and high efficiency of our approach to model mining from “big point cloud data”.

Details

TIST Journal 2013 Journal Article

A fully online and unsupervised system for large and high-density area surveillance

  • Xuan Song
  • Xiaowei Shao
  • Quanshi Zhang
  • Ryosuke Shibasaki
  • Huijing Zhao
  • Jinshi Cui
  • Hongbin Zha

For reasons of public security, an intelligent surveillance system that can cover a large, crowded public area has become an urgent need. In this article, we propose a novel laser-based system that can simultaneously perform tracking, semantic scene learning, and abnormality detection in a fully online and unsupervised way. Furthermore, these three tasks cooperate with each other in one framework to improve their respective performances. The proposed system has the following key advantages over previous ones: (1) It can cover quite a large area (more than 60×35m), and simultaneously perform robust tracking, semantic scene learning, and abnormality detection in a high-density situation. (2) The overall system can vary with time, incrementally learn the structure of the scene, and perform fully online abnormal activity detection and tracking. This feature makes our system suitable for real-time applications. (3) The surveillance tasks are carried out in a fully unsupervised manner, so that there is no need for manual labeling and the construction of huge training datasets. We successfully apply the proposed system to the JR subway station in Tokyo, and demonstrate that it can cover an area of 60×35m, robustly track more than 150 targets at the same time, and simultaneously perform online semantic scene learning and abnormality detection with no human intervention.

Details DOI

TIST Journal 2013 Journal Article

An online system for multiple interacting targets tracking

  • Xuan Song
  • Huijing Zhao
  • Jinshi Cui
  • Xiaowei Shao
  • Ryosuke Shibasaki
  • Hongbin Zha

Multitarget tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This article presents a promising online system to deal with these problems. The novelty of this system is that laser and vision are integrated with tracking and online learning to complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers online for “possible interacting targets”. When the targets are in close proximity, the classifiers learned online are used alongside visual information to assist in tracking. Therefore, this mode of cooperation not only deals with various tough problems encountered in tracking, but also ensures that the entire process can be completely online and automatic. Experimental results demonstrate that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a good trade-off between tracking accuracy and the time-cost factor.

Details DOI

IS Journal 2013 Journal Article

Large-Scale Auto-GPS Analysis for Discerning Behavior Change during Crisis

  • Teerayut Horanont
  • Apichon Witayangkurn
  • Yoshihide Sekimoto
  • Ryosuke Shibasaki

Auto-GPS is a new type of mobile sensing data used to discern human mobility and behavior during a large-scale crisis. Using data collected after the 2011 Great Japan Earthquake, useful information is revealed on how humans react in disaster scenarios and how the evacuation process can be monitored in near real time.

Details DOI

ICRA Conference 2013 Conference Paper

Unsupervised 3D category discovery and point labeling from a large urban environment

  • Quanshi Zhang
  • Xuan Song 0001
  • Xiaowei Shao
  • Huijing Zhao
  • Ryosuke Shibasaki

The building of an object-level knowledge base is the foundation of a new methodology for many perception tasks in artificial intelligence, and is an area that has received increasing attention in recent years. In this paper, we propose, for the first time, to mine category shape patterns directly from a large urban environment, thus constructing a category structure base. Conventionally, category patterns are learned from a large collection of object samples, but automatic object collection requires prior knowledge of category structures. To solve this chicken-and-egg problem, we learn shape patterns from raw segmentations, and then refine these segmentations based on the pattern knowledge. In the process, we solve two challenging problems of knowledge mining. First, as some categories have large intra-category structure variations, we design an entropy-based method to determine the structure variation for each category, in order to establish the correct range of sample collection. Second, because incorrect segmentation is unavoidable without prior knowledge, we propose a novel unsupervised method that uses a pattern competition strategy to identify and subtract shape patterns formed by incorrectly segmented objects. This ensures that shape patterns are meaningful at the object level. Experimental results demonstrated the effectiveness of the proposed method for category structure mining in a large urban environment.

Details

ICRA Conference 2012 Conference Paper

Laser-based intelligent surveillance and abnormality detection in extremely crowded scenarios

  • Xuan Song 0001
  • Xiaowei Shao
  • Quanshi Zhang
  • Ryosuke Shibasaki
  • Huijing Zhao
  • Hongbin Zha

Abnormal activity detection plays a crucial role in surveillance applications, and a surveillance system that can perform robustly in the extremely crowded area has become an urgent need for public security. In this paper, we propose a novel laser-based system which can simultaneously perform the tracking, semantic scene learning and abnormality detection in the large and crowded environment. In our system, a novel abnormality detection model is proposed, and it considers and combines various factors that will influence human activity. Moreover, this model intensively investigate the relationship between pedestrians' social behaviors and their walking scenarios. We successfully applied the proposed system to the JR subway station of Tokyo, which can cover a 60×35m area, robustly track more than 180 targets at the same time and simultaneously perform the online semantic scene learning and abnormality detection with no human intervention.

Details

IROS Conference 2011 Conference Paper

3D crowd surveillance and analysis using laser range scanners

  • Xiaowei Shao
  • Huijing Zhao
  • Ryosuke Shibasaki
  • Yun Shi
  • Kiyoshi Sakamoto

In this study, we present a novel system for crowd surveillance and quantified analysis based on laser range scanners. By mounting a laser scanner at a swinging platform, the spatial information of passengers inside the area of interest can be reconstructed in a form of 3D points. Multiple laser scanners are integrated together by semi-auto calibration procedures. Background map is generated through histogram analysis of scan maps, and is further applied for 3D moving object detection. An improved version of mean-shift clustering algorithm is proposed to extract individual passengers efficiently. In addition, quantified crowdness analysis is conducted from different aspects to indicate the situation inside the surveillance area according to the extraction results of passengers. The proposed system was tested in a central subway station in Tokyo and experimental results demonstrate the effectiveness of our proposed system.

Details

ICRA Conference 2011 Conference Paper

A novel laser-based system: Fully online detection of abnormal activity via an unsupervised method

  • Xuan Song 0001
  • Xiaowei Shao
  • Ryosuke Shibasaki
  • Huijing Zhao
  • Jinshi Cui
  • Hongbin Zha

Abnormal activity detection plays a crucial role in surveillance applications, and such system has become an urgent need for public security. In this paper, we propose a novel laser-based system, which can perform the online detection of abnormal activity with an unsupervised way. The proposed system has the following key features that make it advantageous over previous ones: (1) It can cover quite a large and crowded area, such as subway station, public square, intersection and etc. (2) The overall system can vary with time period, incrementally learn the behavior pattern of pedestrians and perform the fully online detection of abnormal activity. This feature makes our system be quite suitable for the real-time applications. (3) The abnormal activity detection is carried out with a fully unsupervised way, there is no need for manual labelling and constructing the huge training datasets. We successfully applied the proposed system into the JR subway station of Tokyo, which can cover a 60×35m area, track more 150 targets at the same time and simultaneously perform the robust detection of abnormal activity with no human intervention.

Details

ICRA Conference 2010 Conference Paper

Fusion of laser and vision for multiple targets tracking via on-line learning

  • Xuan Song 0001
  • Huijing Zhao
  • Jinshi Cui
  • Xiaowei Shao
  • Ryosuke Shibasaki
  • Hongbin Zha

Multi-target tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This paper presents a promising tracking system to deal with these problems. The novelty of this system is that laser and vision, tracking and learning are integrated and can complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers for the “possible interacting targets”. When the targets are in close proximity, the learned classifiers and visual information are used to assist in tracking. Therefore, this mode of co-operation between them not only deals with various tough problems encountered in the tracking, but also ensures that the entire process can be completely on-line and automatic. Experimental results demonstrated that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a perfect trade-off between tracking accuracy and time-cost.

Details

ICRA Conference 2009 Conference Paper

Moving object classification using horizontal laser scan data

  • Huijing Zhao
  • Quanshi Zhang
  • Masaki Chiba
  • Ryosuke Shibasaki
  • Jinshi Cui
  • Hongbin Zha

Motivated by two potential applications, i. e. enhancing driving safety and traffic data collection, a system has been developed using a single-layer horizontal laser scanner as the major sensor for both localization and perception of the surroundings in a large dynamic urban environment. This research focuses on a classification method, that given a stream of laser measurements, classify the moving object into either a person, a group of people, a bicycle or a car. In this research, a number of features are defined after examining the property of data appearance. A classification method is proposed after examining the likelihood measures between each pair of feature and class. Experimental results are presented, demonstrating that the algorithm has efficiency with respect to both driving safety and traffic data collection in highly dynamic environment.

Details

ICRA Conference 2008 Conference Paper

SLAM in a dynamic large outdoor environment using a laser scanner

  • Huijing Zhao
  • Masaki Chiba
  • Ryosuke Shibasaki
  • Xiaowei Shao
  • Jinshi Cui
  • Hongbin Zha

In this research, we propose a method of SLAM in a dynamic large outdoor environment using a laser scanner. Focus are cast on solving two major problems: 1) achieving global accuracy especially in non-cyclical environment, 2) tackling a mixture of data from both dynamic and static objects. Algorithms are developed, where GPS data and control inputs are used to diagnose pose error and guide to achieve a global accuracy; Classification of laser points and objects are conducted not in an independent module but across the processing in a framework of SLAM with moving object detection and tracking. Experiments are conducted using the data from two test-bed vehicles, and performance of the algorithms are demonstrated.

Details

IROS Conference 2007 Conference Paper

Detection and tracking of multiple pedestrians by using laser range scanners

  • Xiaowei Shao
  • Huijing Zhao
  • Katsuyuki Nakamura
  • Kyoichiro Katabira
  • Ryosuke Shibasaki
  • Yuri Nakagawa

We propose a novel system for tracking multiple pedestrians in a crowded scene by exploiting single-row laser range scanners that measure distances of surrounding objects. A walking model is built to describe the periodicity of the movement of the feet in the spatial-temporal domain, and a mean-shift clustering technique in combination with spatial- temporal correlation analysis is applied to detect pedestrians. Based on the walking model, particle filter is employed to track multiple pedestrians. Compared with camera-based methods, our system provides a novel technique to track multiple pedestrians in a relatively large area. The experiments, in which over 300 pedestrians were tracked in 5 minutes, show the validity of the proposed system.

Details

ICRA Conference 2007 Conference Paper

Monitoring a populated environment using single-row laser range scanners from a mobile platform

  • Huijing Zhao
  • Yuzhong Chen
  • Xiaowei Shao
  • Kyoichiro Katabira
  • Ryosuke Shibasaki

In this research, we proposed a system of detecting and monitoring pedestrians' motion trajectories at a populated and wide environment, such as exhibition hall, supermarket etc. , using the horizontally profiling single-row laser range scanners on a mobile platform. A simplified walking model is defined to track the rhythmic swing feet at the ground level. Pedestrians are recognized by detecting the braided styles, which is a typical appearance that could discriminate the data of moving feet with other mobile and motionless objects. Two experiments are conducted. One is at the laboratory environment, the purpose of which is to examine the algorithm in details. Another is at an exhibition hall, a populated and wide environment, the purpose is to examine whether the system could be applied for practical needs. It is a big challenge, while the system did well. Pedestrians in the exhibition hall at the moment of measurement are detected. Their motion trajectories are extracted, and associated to the background map, which is made of the motionless objects, and covers the whole exhibition hall.

Details

IROS Conference 2006 Conference Paper

Laser-based Interacting People Tracking Using Multi-level Observations

  • Jinshi Cui
  • Hongbin Zha
  • Huijing Zhao
  • Ryosuke Shibasaki

Laser based people tracking systems have been developed for mobile robotics and intelligent surveillance areas. Existing systems rely on simple laser point clustering methods to extract object locations. However, when dealing with multiple interacting people, laser points of different persons are often interlaced and undistinguishable due to measurement noise and they can not provide reliable features. It causes current systems quite fragile and unreliable. In this paper, we try to explore potentials from multi-level observations including weakly detected features, stably extracted features and foreground points. For inference, detection incorporated joint particle filter is used. And stably extracted features are utilized to properly estimate parameters of dynamic model for each target. In real experiments, we obtain raw data from multiple registered laser scanners, which measure two legs for each people. Evaluations with real data show that the proposed method is more robust and effective than existing approaches

Details

IROS Conference 2005 Conference Paper

Tracking multiple people using laser and vision

  • Jinshi Cui
  • Hongbin Zha
  • Huijing Zhao
  • Ryosuke Shibasaki

We present a novel system that aims at reliably detecting and tracking multiple people in an open area. Multiple single-row laser scanners and one video camera are utilized. Feet trajectory tracking based on registration of distance information from multiple laser scanners and visual body region tracking based on color histogram are combined in a Bayesian formulation. Results from tests in a real environment are reported to demonstrate that the system can detect and track multiple people simultaneously with reliable and real-time performance.

Details
v2026.09.13