Ramin Zabih

NeurIPS Conference 2024 Conference Paper

Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Models

• Ali Behrouz
• Michele Santacatterina
• Ramin Zabih

Modeling multivariate time series is a well-established problem with a wide range of applications from healthcare to financial markets. It, however, is challenging as it requires methods to (1) have high expressive power of representing complicated dependencies along the time axis to capture both long-term progression and seasonal patterns, (2) capture the inter-variate dependencies when it is informative, (3) dynamically model the dependencies of variate and time dimensions, and (4) have efficient training and inference for very long sequences. Traditional State Space Models (SSMs) are classical approaches for univariate time series modeling due to their simplicity and expressive power to represent linear dependencies. They, however, have fundamentally limited expressive power to capture non-linear dependencies, are slow in practice, and fail to model the inter-variate information flow. Despite recent attempts to improve the expressive power of SSMs by using deep structured SSMs, the existing methods are either limited to univariate time series, fail to model complex patterns (e. g. , seasonal patterns), fail to dynamically model the dependencies of variate and time dimensions, and/or are input-independent. We present Chimera, an expressive variation of the 2-dimensional SSMs with careful design of parameters to maintain high expressive power while keeping the training complexity linear. Using two SSM heads with different discretization processes and input-dependent parameters, Chimera is provably able to learn long-term progression, seasonal patterns, and desirable dynamic autoregressive processes. To improve the efficiency of complex 2D recurrence, we present a fast training using a new 2-dimensional parallel selective scan. Our experimental evaluation shows the superior performance of Chimera on extensive and diverse benchmarks, including ECG and speech time series classification, long-term and short-term time series forecasting, and time series anomaly detection.

NeurIPS Conference 2023 Conference Paper

Test-Time Distribution Normalization for Contrastively Learned Visual-language Models

• Yifei Zhou
• Juntao Ren
• Fengyu Li
• Ramin Zabih
• Ser Nam Lim

Advances in the field of visual-language contrastive learning have made it possible for many downstream applications to be carried out efficiently and accurately by simply taking the dot product between image and text representations. One of the most representative approaches proposed recently known as CLIP has quickly garnered widespread adoption due to its effectiveness. CLIP is trained with an InfoNCE loss that takes into account both positive and negative samples to help learn a much more robust representation space. This paper however reveals that the common downstream practice of taking a dot product is only a zeroth-order approximation of the optimization goal, resulting in a loss of information during test-time. Intuitively, since the model has been optimized based on the InfoNCE loss, test-time procedures should ideally also be in alignment. The question lies in how one can retrieve any semblance of negative samples information during inference in a computationally efficient way. We propose Distribution Normalization (DN), where we approximate the mean representation of a batch of test samples and use such a mean to represent what would be analogous to negative samples in the InfoNCE loss. DN requires no retraining or fine-tuning and can be effortlessly applied during inference. Extensive experiments on a wide variety of downstream tasks exhibit a clear advantage of DN over the dot product on top of other existing test-time augmentation methods.

AAAI Conference 1990 Conference Paper

Some Applications of Graph Bandwidth to Constraint Satisfaction Problems

• Ramin Zabih

Bandwidth is a fundamental concept in graph theory which has some surprising applications to a class of AI search problems. Graph bandwidth provides a link between the syntactic structure of a constraint satisfaction problem (CSP) and the complexity of the underlying search task. Bandwidth can be used to define a new class of easy CSP’ s, namely those that have limited constraint graph bandwidth. These CSP’ s can be solved in polynomial time, essentially by divide and conquer. This in turn suggests that bandwidth provides a mathematical measure of the decomposability of a search problem. In addition, bandwidth supplies a measure for comparing different search orderings for a given CSP. Statistical analysis suggests that backtracking with small bandwidth orderings leads to a more efficient search than that obtained under orderings with larger bandwidths. Small bandwidth orderings also limit the pruning that can be done by intelligent backtracking. If small bandwidth orderings are indeed advantageous, then a large number of heuristics developed in numerical analysis to find such orderings may find applicability to solving constraint satisfaction problems.

IJCAI Conference 1989 Conference Paper

An Algebraic Approach to Constraint Satisfaction Problems

• Igor Rivin
• Ramin Zabih

AAAI Conference 1988 Conference Paper

A Rearrangement Search Strategy for Determining Propositional Satisfiability

• Ramin Zabih

We present a simple algorithm for determining the satisfiability of propositional formulas in Conjunctive Normal Form. As the procedure searches for a satisfying truth assignment it dynamically rearranges the order in which variables are considered. The choice of which variable to assign a truth value next is guided by an upper bound on the size of the search remaining; the procedure makes the choice which yields the smallest upper bound on the size of the remaining search. We describe several upper bound functions and discuss the tradeoff between accurate upper bound functions and the overhead required to compute the upper bounds. Experimental data shows that for one easily computed upper bound the reduction in the size of the search spa, ce more than compensates for the 0verhea. d involved in selecting the next variable.

AAAI Conference 1987 Conference Paper

Non-Deterministic Lisp with Dependency-Directed Backtracking

• Ramin Zabih

Extending functional Lisp with McCarthy’s non-deterministic operator AMB yields a language which can concisely express search problems. Dependency-directed backtracking is a powerful search strategy. We describe a non-deterministic Lisp dialect called SCHEMER and show that it can provide automatic dependency-directed backtracking. The resulting language provides a convenient interface to this efficient backtracking strategy.