Arrow Research search

Author name cluster

Amit Sheth

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

42 papers
1 author row

Possible papers

42

AAAI Conference 2026 System Paper

CausalPulse: Agentic Copilot for Root Cause Analysis in Smart Manufacturing

  • Chathurangi Shyalika
  • Utkarshani Jaimini
  • Cory Henson
  • Amit Sheth

Modern manufacturing systems demand real-time, trustworthy, and interpretable insights into anomalies and their underlying causes. However, conventional pipelines treat anomaly detection, causal inference, and decision-making as siloed tasks, lacking integration, explainability, and adaptability. We present CausalPulse, an intelligent, multi-agent copilot for automated Root Cause Analysis (RCA) in industrial settings. Built on a modular and extensible architecture, the system leverages standard agentic protocols, including Model Context Protocol (MCP), Agent2Agent (A2A), and LangGraph for dynamic tool and agent discovery and seamless orchestration of tasks. Agents dynamically interact to perform data preprocessing, anomaly detection, causal discovery, and root cause analysis through a neurosymbolic workflow that combines symbolic reasoning with neural methods. Intelligent postprocessing pipelines enable automatic chaining of agent tasks, enhancing contextual awareness and adaptability. CausalPulse is evaluated using both an academic public dataset (i.e., Future Factories) and an industrial proprietary dataset (i.e., Planar Oxygen Sensor Element) and shows that the system outperforms traditional baselines in interpretability, trustworthiness, and operational utility.

PDF Details DOI

AAAI Conference 2026 System Paper

Chatsparent: An Interactive System for Detecting and Mitigating Cognitive Fatigue in LLMs

  • Riju Marwah
  • Vishal Pallagani
  • Ritvik Garimella
  • Amit Sheth

LLMs are increasingly being deployed as chatbots, but today’s interfaces offer little to no friction: users interact through seamless conversations that conceal when the model is drifting, hallucinating or failing. This lack of transparency fosters blind trust, even as models produce unstable or repetitive outputs. We introduce an interactive demo that surfaces and mitigates cognitive fatigue, a failure mode where LLMs gradually lose coherence during auto-regressive generation. Our system, Chatsparent, instruments real-time, token-level signals of fatigue, including attention-to-prompt decay, embedding drift, and entropy collapse, and visualizes them as a unified fatigue index. When fatigue thresholds are crossed, the interface allows users to activate lightweight interventions such as attention resets, entropy-regularized decoding, and self-reflection checkpoints. The demo streams live text and fatigue signals, allowing users to observe when fatigue arises, how it affects output quality, and how interventions restore stability. By turning passive chatbot interaction into an interactive diagnostic experience, our system empowers users to better understand LLM behavior while improving reliability at inference time.

PDF Details DOI

AAAI Conference 2026 Conference Paper

DETONATE – A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization

  • Renjith Prasad Kaippilly Mana
  • Abhilekh Borah
  • Hasnat Md Abdullah
  • Chathurangi Shyalika
  • Gurpreet Singh
  • Ritvik Garimella
  • Rajarshi Roy
  • Harshul Raj Surana

Alignment is crucial for text-to-image (T2I) models to ensure that the generated images faithfully capture user intent while maintaining safety and fairness. Direct Preference Optimization (DPO) has emerged as a key alignment technique for large language models (LLMs), and its influence is now extending to T2I systems. This paper introduces DPO-Kernels for T2I models, a novel extension of DPO that enhances alignment across three key dimensions: (i) Hybrid Loss, which integrates embedding-based objectives with the traditional probability-based loss to improve optimization; (ii) Kernelized Representations, leveraging Radial Basis Function (RBF), Polynomial, and Wavelet kernels to enable richer feature transformations, ensuring better separation between safe and unsafe inputs; and (iii) Divergence Selection, expanding beyond DPO’s default Kullback–Leibler (KL) regularizer by incorporating alternative divergence measures such as Wasserstein and Rényi divergences to enhance stability and robustness in alignment training. We introduce DETONATE, the first large-scale benchmark of its kind, comprising approximately 100K curated image pairs, categorized as chosen and rejected. This benchmark encapsulates three critical axes of social bias and discrimination: Race, Gender, and Disability. The prompts are sourced from the hate speech datasets, while the images are generated using state-of-the-art T2I models, including Stable Diffusion 3.5 Large (SD-3.5), Stable Diffusion XL (SD-XL), and Midjourney. Furthermore, to evaluate alignment beyond surface metrics, we introduce the Alignment Quality Index (AQI) for T2I systems: a novel geometric measure that quantifies latent space separability of safe/unsafe image activations, revealing hidden model vulnerabilities. While alignment techniques often risk overfitting, we empirically demonstrate that DPO-Kernels preserve strong generalization bounds using the theory of Heavy-Tailed Self-Regularization (HT-SR).

PDF Details DOI

AAAI Conference 2026 System Paper

In-Situ Eval: A Modular Framework for Custom and Real-Time RAG Benchmarking

  • Ritvik Garimella
  • Kaushik Roy
  • Chathurangi Shyalika
  • Amit Sheth

Retrieval-Augmented Generation (RAG) has become the standard approach for integrating domain knowledge into Large Language Models (LLMs). However, fair comparison of RAG pipelines remains difficult: data preparation is often ad hoc, subsampling methods are opaque, parameters vary across implementations, and evaluation is fragmented. We present In-Situ Eval, a unified and reproducible framework that operationalizes the full RAG pipeline with configurable subsampling strategies and both RAG-specific and generic evaluation metrics. The platform supports two execution modes: an offline Dataset mode for evaluating precomputed outputs, and a live Retrieval mode for benchmarking RAG variants with state-of-the-art LLMs. Users can flexibly select datasets, retrieval techniques, models, and metrics, enabling side-by-side comparisons, ablations, and targeted analyses. This holistic approach reduces computational costs, clarifies the impact of subsampling techniques, and provides actionable insights for real-world deployments. By facilitating transparent, customizable, and interactive benchmarking, In-Situ Eval empowers both researchers and practitioners to make informed decisions in adapting RAG pipelines to domain-specific needs.

PDF Details DOI

AAAI Conference 2026 System Paper

PAL: Personal Adaptive Learner

  • Megha Chakraborty
  • Darssan L. Eswaramoorthi
  • Madhur Thareja
  • Het Riteshkumar Shah
  • Finlay Palmer
  • Aryaman Bahl
  • Michelle A Ihetu
  • Amit Sheth

AI-driven education platforms have made some progress in personalisation, yet most remain constrained to static adaptation—predefined quizzes, uniform pacing, or generic feedback—limiting their ability to respond to learners’ evolving understanding. This shortfall highlights the need for systems that are both context-aware and adaptive in real time. We introduce PAL (Personal Adaptive Learner), an AI-powered platform that transforms lecture videos into interactive learning experiences. PAL continuously analyzes multimodal lecture content and dynamically engages learners through questions of varying difficulty, adjusting to their responses as the lesson unfolds. At the end of a session, PAL generates a personalized summary that reinforces key concepts while tailoring examples to the learner’s interests. By uniting multimodal content analysis with adaptive decision-making, PAL contributes a novel framework for responsive digital learning. Our work demonstrates how AI can move beyond static personalization toward real-time, individualized support, addressing a core challenge in AI-enabled education.

PDF Details DOI

IS Journal 2025 Journal Article

Cognitive Neurosymbolic Artificial Intelligence for Complex Decision-Making: Integrating Foundation Models, Cognitive Architectures, and Knowledge

  • Yuxin Zi
  • Kaushik Roy
  • Amit Sheth

Advances in deep learning and knowledge representation have driven the adoption of intelligent decision support systems in high-stakes domains such as health-care applications. However, traditional statistical approaches often yield opaque, correlation-driven predictions, making it difficult for experts to interpret results and take meaningful actions. To address these limitations, we propose a cognitive neurosymbolic artificial intelligence (AI) framework by combining multimodal perception, neurosymbolic reasoning (with symbolic controls), and knowledge, this paradigm enhances accuracy and robustness in complex decision-making environments. This integrated framework ensures that every recommendation follows human-like decision-making, while being grounded in verifiable, expert knowledge, a crucial attribute for critical application scenarios such as health care. We demonstrate this approach through a mental health support system, showcasing how cognitive neurosymbolic AI can improve decision-making outcomes, personalized interventions, longitudinal tracking, and actionable insights for critical applications.

Details DOI

IS Journal 2025 Journal Article

From Morphemes to Knowledge Graphs: Enabling Abstractions in Large Language Models With Neurosymbolic AI

  • Thilini Wijesiriwardene
  • Krishnaprasad Thirunarayanan
  • Amit Sheth

Recent advances in large language models (LLMs) have revolutionized natural language processing, achieving impressive performance across a wide range of linguistic tasks. However, these successes often mask a critical limitation: Current evaluation paradigms provide little insight into how well LLMs handle linguistic abstractions, the very cognitive capability that underlies generalization, analogy-making, and systematic reasoning. Without a principled framework for evaluating abstraction, it remains unclear whether LLMs truly engage in abstractions and their nature, how consistently they do so, and to what extent these behaviors reflect genuine abstraction capabilities versus surface-level pattern matching. We propose a structured taxonomy of linguistic abstractions in natural language processing, spanning levels from morphology to knowledge graphs (KGs), organized along two key dimensions: linguistic granularity and contextual dependence. This taxonomy supports a more nuanced evaluation of LLMs’ abstraction capability and helps identify where current models fall short. In particular, we highlight the limitations of LLMs at higher levels of abstraction—such as semantic, topical, taxonomic, and KG levels—where relational composition, context sensitivity, and symbolic structures are critical. To remedy these weaknesses, we advocate for the integration of neurosymbolic artificial intelligence (AI) systems that combine neural representations with symbolic reasoning.

Details DOI

PRL Workshop 2025 Workshop Paper

Inductive Logic Programming for Heuristic Search

  • Rojina Panta
  • Vedant Khandelwal
  • Celeste Veronese
  • Amit Sheth
  • Daniele Meli
  • Forest Agostinelli

Pathfinding problems are found through computing, chemistry, mathematics, and robotics. Solving pathfinding problems is typically achieved through heuristic search, which is guided by a heuristic function that can be learned using deep neural networks. However, since deep neural networks are typically not explainable, the extraction of new knowledge from these learned heuristic functions is cumbersome. On the other hand, to the best of our knowledge, it has yet to be shown how heuristic functions represented as logic programs, which have been shown to be explainable, can be learned. In this work, we present an algorithm to learn heuristic functions represented as logic programs using dynamic programming and inductive logic programming. Furthermore, we build on dynamic programming concepts to improve the learned logic programs by reusing predicates learned for solving simpler pathfinding problem instances to solve more complex instances. We use the 8-puzzle to demonstrate the effectiveness of our algorithm. Code — https: //github. com/Rojina99/HeurSearchILP

PDF Details

IS Journal 2025 Journal Article

NeuroSymbolic Knowledge-Grounded Planning and Reasoning in Artificial Intelligence Systems

  • Amit Sheth
  • Vedant Khandelwal
  • Kaushik Roy
  • Vishal Pallagani
  • Megha Chakraborty

Decision-support systems in AI-assisted health care require robust, interpretable, and user-centric processes that effectively handle natural language inputs. While large language models (LLMs) excel at generating coherent text, they struggle with complex reasoning and multistep planning tasks. In response, we propose a neurosymbolic framework that integrates LLMs with symbolic knowledge graphs, graph-based reasoners, and constraint-aware planning modules. This hybrid approach leverages LLMs for initial plan formulation while refining outcomes with structured, domain-specific representations that enforce safety standards, ensure regulatory compliance, and maintain logical consistency. Demonstrated through examples in health care and manufacturing, our method bridges the gap between unstructured language generation and formal reasoning, enhancing reliability in high-stakes applications and supporting dynamic, context-aware decision-making. The framework offers a scalable, trustworthy solution for complex, constraint-driven environments. By combining generative creativity with formal logic, our approach addresses the key limitations of LLMs, making it suitable for diverse, high-impact domains.

Details DOI

IJCAI Conference 2025 Conference Paper

NSF-MAP: Neurosymbolic Multimodal Fusion for Robust and Interpretable Anomaly Prediction in Assembly Pipelines

  • Chathurangi Shyalika
  • Renjith Prasad
  • Fadi El Kalach
  • Revathy Venkataramanan
  • Ramtin Zand
  • Ramy Harik
  • Amit Sheth

In modern assembly pipelines, identifying anomalies is crucial in ensuring product quality and operational efficiency. Conventional single-modality methods fail to capture the intricate relationships required for precise anomaly prediction in complex predictive environments with abundant data and multiple modalities. This paper proposes a neurosymbolic AI and fusion-based approach for multimodal anomaly prediction in assembly pipelines. We introduce a time series and image-based fusion model that leverages decision-level fusion techniques. Our research builds upon three primary novel approaches in multimodal learning: time series and image-based decision-level fusion modeling, transfer learning for fusion, and knowledge-infused learning. We evaluate the novel method using our derived and publicly available multimodal dataset and conduct comprehensive ablation studies to assess the impact of our preprocessing techniques and fusion model compared to traditional baselines. The results demonstrate that a neurosymbolic AI-based fusion approach that uses transfer learning can effectively harness the complementary strengths of time series and image data, offering a robust and interpretable approach for anomaly prediction in assembly pipelines with enhanced performance. \noindent The datasets, codes to reproduce the results, supplementary materials, and demo are available at https: //github. com/ChathurangiShyalika/NSF-MAP.

PDF Details DOI

AAMAS Conference 2025 Conference Paper

SmartPilot: Agent-Based CoPilot for Intelligent Manufacturing

  • Chathurangi Shyalika
  • Renjith Prasad
  • Alaa Al Ghazo
  • Darssan L. Eswaramoorthi
  • Sara Shree Muthuselvam
  • Amit Sheth

In the dynamic landscape of Industry 4. 0, achieving efficiency, precision, and adaptability is essential for optimizing manufacturing operations. SmartPilot is a neurosymbolic and agent-based CoPilot designed to enhance real-time decision-making capabilities in manufacturing. The system addresses three key challenges: anomaly prediction, production forecasting, and domain-specific question answering through an agent-based framework. SmartPilot leverages multimodal data and a compact architecture optimized for edge devices. This paper highlights its innovative combination of agent-based design and neurosymbolic reasoning to enable contextual decision-making in complex environments. The demonstration video1, datasets, and supplementary materials are available at https: //github. com/ChathurangiShyalika/SmartPilot.

PDF

IS Journal 2024 Journal Article

Causal Neurosymbolic AI: A Synergy Between Causality and Neurosymbolic Methods

  • Utkarshani Jaimini
  • Cory Henson
  • Amit Sheth

Causal neurosymbolic AI (NeSyAI) combines the benefits of causality with NeSyAI. More specifically, it 1) enriches NeSyAI systems with explicit representations of causality, 2) integrates causal knowledge with domain knowledge, and 3) enables the use of NeSyAI techniques for causal AI tasks. The explicit causal representation yields insights that predictive models may fail to analyze from observational data. It can also assist people in decision-making scenarios where discerning the cause of an outcome is necessary to choose among various interventions.

Details DOI

AAAI Conference 2024 System Paper

GEAR-Up: Generative AI and External Knowledge-Based Retrieval: Upgrading Scholarly Article Searches for Systematic Reviews

  • Kaushik Roy
  • Vedant Khandelwal
  • Valerie Vera
  • Harshul Surana
  • Heather Heckman
  • Amit Sheth

This paper addresses the time-intensive nature of systematic reviews (SRs) and proposes a solution leveraging advancements in Generative AI (e.g., ChatGPT) and external knowledge augmentation (e.g., Retrieval-Augmented Generation). The proposed system, GEAR-Up, automates query development and translation in SRs, enhancing efficiency by enriching user queries with context from language models and knowledge graphs. Collaborating with librarians, qualitative evaluations demonstrate improved reproducibility and search strategy quality. Access the demo at https://youtu.be/zMdP56GJ9mU.

PDF Details DOI

IS Journal 2024 Journal Article

Grounding From an AI and Cognitive Science Lens

  • Goonmeet Bajaj
  • Valerie L. Shalin
  • Srinivasan Parthasarathy
  • Amit Sheth

Grounding is a challenging problem, requiring a formal definition and different levels of abstraction. This article explores grounding from both cognitive science and machine learning perspectives. It identifies the subtleties of grounding, its significance for collaborative agents, and similarities and differences in grounding approaches in both communities. The article examines the potential of neurosymbolic approaches tailored for grounding tasks, showcasing how they can more comprehensively address grounding. Finally, we discuss areas for further exploration and development in grounding.

Details DOI

IS Journal 2024 Journal Article

Neurosymbolic AI for Enhancing Instructability in Generative AI

  • Amit Sheth
  • Vishal Pallagani
  • Kaushik Roy

Generative AI, especially via large language models (LLMs), has transformed content creation across text, images, and music, showcasing capabilities in following instructions through prompting, largely facilitated by instruction tuning. Instruction tuning is a supervised fine-tuning method where LLMs are trained on datasets formatted with specific tasks and corresponding instructions. This method systematically enhances the model’s ability to comprehend and execute the provided directives. Despite these advancements, LLMs still face challenges in consistently interpreting complex, multistep instructions and generalizing them to novel tasks, which are essential for broader applicability in real-world scenarios. This article explores why neurosymbolic AI offers a better path to enhance the instructability of LLMs. We explore the use of a symbolic task planner to decompose high-level instructions into structured tasks, a neural semantic parser to ground these tasks into executable actions, and a neuro-symbolic executor to implement these actions while dynamically maintaining an explicit representation of state.

Details DOI

IS Journal 2024 Journal Article

Neurosymbolic Value-Inspired Artificial Intelligence (Why, What, and How)

  • Amit Sheth
  • Kaushik Roy

The rapid progression of artificial intelligence (AI) systems, facilitated by the advent of large language models (LLMs), has resulted in their widespread application to provide human assistance across diverse industries. This trend has sparked significant discourse centered around the ever-increasing need for LLM-based AI systems to function among humans as a part of human society. Toward this end, neurosymbolic AI systems are attractive because of their potential to enable and interpretable interfaces for facilitating value-based decision making by leveraging explicit representations of shared values. In this article, we introduce substantial extensions to Kahneman’s System 1 and System 2 framework and propose a neurosymbolic computational framework called value-inspired AI (VAI). It outlines the crucial components essential for the robust and practical implementation of VAI systems, representing and integrating various dimensions of human values. Finally, we further offer insights into the current progress made in this direction and outline potential future directions for the field.

Details DOI

IS Journal 2023 Journal Article

A Semantic Web Approach to Fault Tolerant Autonomous Manufacturing

  • Fadi El Kalach
  • Ruwan Wickramarachchi
  • Ramy Harik
  • Amit Sheth

The next phase of manufacturing is centered on making the switch from traditional automated to autonomous systems. Future factories are required to be agile, allowing for more customized production and resistance to disturbances. Such production lines would be able to reallocate resources as needed and minimize downtime while keeping up with market demands. These systems must be capable of complex decision-making based on parameters, such as machine status, sensory/IoT data, and inspection results. Current manufacturing lines lack this complex capability and instead focus on low-level decision-making on the machine level without utilizing the generated data to its full extent. This article presents progress toward this autonomy by introducing Semantic Web capabilities applied to managing the production line. Finally, a full autonomous manufacturing use case is also developed to showcase the value of Semantic Web in a manufacturing context. This use case utilizes diverse data sources and domain knowledge to complete a manufacturing process despite malfunctioning equipment. It highlights the benefit of Semantic Web in manufacturing by integrating the heterogeneous information required for the process to be completed. This provides an approach to autonomous manufacturing not yet fully realized at the intersection of Semantic Web and manufacturing.

Details DOI

AAAI Conference 2023 System Paper

CLUE-AD: A Context-Based Method for Labeling Unobserved Entities in Autonomous Driving Data

  • Ruwan Wickramarachchi
  • Cory Henson
  • Amit Sheth

Generating high-quality annotations for object detection and recognition is a challenging and important task, especially in relation to safety-critical applications such as autonomous driving (AD). Due to the difficulty of perception in challenging situations such as occlusion, degraded weather, and sensor failure, objects can go unobserved and unlabeled. In this paper, we present CLUE-AD, a general-purpose method for detecting and labeling unobserved entities by leveraging the object continuity assumption within the context of a scene. This method is dataset-agnostic, supporting any existing and future AD datasets. Using a real-world dataset representing complex urban driving scenes, we demonstrate the applicability of CLUE-AD for detecting unobserved entities and augmenting the scene data with new labels.

PDF Details DOI

AAAI Conference 2023 System Paper

Demo Alleviate: Demonstrating Artificial Intelligence Enabled Virtual Assistance for Telehealth: The Mental Health Case

  • Kaushik Roy
  • Vedant Khandelwal
  • Raxit Goswami
  • Nathan Dolbir
  • Jinendra Malekar
  • Amit Sheth

After the pandemic, artificial intelligence (AI) powered support for mental health care has become increasingly important. The breadth and complexity of significant challenges required to provide adequate care involve: (a) Personalized patient understanding, (b) Safety-constrained and medically validated chatbot patient interactions, and (c) Support for continued feedback-based refinements in design using chatbot-patient interactions. We propose Alleviate, a chatbot designed to assist patients suffering from mental health challenges with personalized care and assist clinicians with understanding their patients better. Alleviate draws from an array of publicly available clinically valid mental-health texts and databases, allowing Alleviate to make medically sound and informed decisions. In addition, Alleviate's modular design and explainable decision-making lends itself to robust and continued feedback-based refinements to its design. In this paper, we explain the different modules of Alleviate and submit a short video demonstrating Alleviate's capabilities to help patients and clinicians understand each other better to facilitate optimal care strategies.

PDF Details DOI

IS Journal 2023 Journal Article

Neurosymbolic Artificial Intelligence (Why, What, and How)

  • Amit Sheth
  • Kaushik Roy
  • Manas Gaur

Humans interact with the environment using a combination of perception—transforming sensory inputs from their environment into symbols, and cognition—mapping symbols to knowledge about the environment for supporting abstraction, reasoning by analogy, and long-term planning. Human perception-inspired machine perception, in the context of artificial intelligence (AI), refers to large-scale pattern recognition from raw data using neural networks trained using self-supervised learning objectives such as next-word prediction or object recognition. On the other hand, machine cognition encompasses more complex computations, such as using knowledge of the environment to guide reasoning, analogy, and long-term planning. Humans can also control and explain their cognitive functions. This seems to require the retention of symbolic mappings from perception outputs to knowledge about their environment. For example, humans can follow and explain the guidelines and safety constraints driving their decision making in safety-critical applications such as health care, criminal justice, and autonomous driving.

Details DOI

IS Journal 2023 Journal Article

Why Do We Need Neurosymbolic AI to Model Pragmatic Analogies?

  • Thilini Wijesiriwardene
  • Amit Sheth
  • Valerie L. Shalin
  • Amitava Das

A hallmark of intelligence is the ability to use a familiar domain to make inferences about a less familiar domain, known as analogical reasoning. In this article, we delve into the performance of large language models (LLMs) in dealing with progressively complex analogies expressed in unstructured text. We discuss analogies at four distinct levels of complexity: lexical, syntactic, semantic, and pragmatic. As the analogies become more complex, they require increasingly extensive, diverse knowledge beyond the textual content, unlikely to be found in the lexical co-occurrence statistics that power LLMs. We discuss neurosymbolic AI techniques that combine statistical and symbolic AI, informing the representation of unstructured text to highlight and augment relevant content, provide abstraction, and guide the mapping process. This maintains the efficiency of LLMs while preserving the ability to explain analogies for pedagogical applications.

Details DOI

IS Journal 2022 Journal Article

Knowledge-Based Entity Prediction for Improved Machine Perception in Autonomous Systems

  • Ruwan Wickramarachchi
  • Cory Henson
  • Amit Sheth

Knowledge-based entity prediction (KEP) is a novel task that aims to improve machine perception in autonomous systems. KEP leverages relational knowledge from heterogeneous sources in predicting potentially unrecognized entities. In this article, we provide a formal definition of KEP as a knowledge completion task. Three potential solutions are then introduced, which employ several machine learning and data mining techniques. Finally, the applicability of KEP is demonstrated on two autonomous systems from different domains; namely, autonomous driving and smart manufacturing. We argue that in complex real-world systems, the use of KEP would significantly improve machine perception while pushing the current technology one step closer to achieving full autonomy.

Details DOI

IS Journal 2021 Journal Article

Cognitive Digital Twins for Smart Manufacturing

  • Muhammad Intizar Ali
  • Pankesh Patel
  • John G. Breslin
  • Ramy Harik
  • Amit Sheth

Smart manufacturing or Industry 4. 0, a trend initiated a decade ago, aims to revolutionize traditional manufacturing using technology driven approaches. Modern digital technologies such as the Industrial Internet of Things (IIoT), Big Data analytics, augmented/virtual reality, and artificial intelligence (AI) are the key enablers of new smart manufacturing approaches. The digital twin is an emerging concept whereby a digital replica can be built of any physical object. Digital twins are becoming mainstream; many organizations have started to rely on digital twins to monitor, analyze, and simulate physical assets and processes.

Details DOI

IS Journal 2019 Journal Article

Extending Patient-Chatbot Experience with Internet-of-Things and Background Knowledge: Case Studies with Healthcare Applications

  • Amit Sheth
  • Hong Yung Yip
  • Saeedeh Shekarpour

Presents case studies in the healthcare industry that focus on the use of Chatbots to improve patient monitoring and medical services. The transition towards personalized health management requires public awareness about management strategies of self-monitoring, self-appraisal, and self-management, eventually paving a way to more timely interventions and higher quality patient–clinician interactions. A key enabler is patient generated health data, fueled in good part by the growth in wearable devices including smart watches and other Internet-of- Things (IoT) for health-tracking. These tracking devices provide “low-level” monitoring signals indicating health conditions such as sleep apnea and heart rhythm disorder. However, to make more sense of IoT data, it is imperative that we develop cognitive approaches where they mine, interlink, and abstract diverse IoT data. These cognitive approaches often needs to keep the user closely engaged to acquire more information, to obtain feedback, to collect verbal health conditions, and to provide intervention and management actions. The chatbot technology was initially introduced as an artificial conversational agent to simulate conversations with a user using voice or text interactions.

Details DOI

IS Journal 2018 Journal Article

From Raw Data to Smart Manufacturing: AI and Semantic Web of Things for Industry 4.0

  • Pankesh Patel
  • Muhammad Intizar Ali
  • Amit Sheth

AI techniques combined with recent advancements in the Internet of Things, Web of Things, and Semantic Web-jointly referred to as the Semantic Web-promise to play an important role in Industry 4. 0. As part of this vision, the authors present a Semantic Web of Things for Industry 4. 0 (SWeTI) platform. Through realistic use case scenarios, they showcase how SweTI technologies can address Industry 4. 0s challenges, facilitate cross-sector and cross-domain integration of systems, and develop intelligent and smart services for smart manufacturing.

Details DOI

IS Journal 2018 Journal Article

How Will the Internet of Things Enable Augmented Personalized Health?

  • Amit Sheth
  • Utkarshani Jaimini
  • Hong Yung Yip

The Internet of Things refers to network-enabled technologies, including mobile and wearable devices, which are capable of sensing and actuation as well as interaction and communication with other similar devices over the Internet. The IoT is profoundly redefining the way we create, consume, and share information. Ordinary citizens increasingly use these technologies to track their sleep, food intake, activity, vital signs, and other physiological statuses. This activity is complemented by IoT systems that continuously collect and process environment-related data that has a bearing on human health. This synergy has created an opportunity for a new generation of healthcare solutions.

Details DOI

IS Journal 2017 Journal Article

Challenges of Sentiment Analysis for Dynamic Events

  • Monireh Ebrahimi
  • Amir Hossein Yazdavar
  • Amit Sheth

Efforts to assess people's sentiments on Twitter have suggested that Twitter could be a valuable resource for studying political sentiment and that it reflects the offline political landscape. Many opinion mining systems and tools provide users with people's attitudes toward products, people, or topics and their attributes/aspects. However, although it may appear simple, using sentiment analysis to predict election results is difficult, since it is empirically challenging to train a successful model to conduct sentiment analysis on tweet streams for a dynamic event such as an election. This article highlights some of the challenges related to sentiment analysis encountered during monitoring of the presidential election using Kno. e. sis's Twitris system.

Details DOI

IS Journal 2017 Journal Article

IoT Quality Control for Data and Application Needs

  • Tanvi Banerjee
  • Amit Sheth

The amount of Internet of Things (IoT) data is growing rapidly. Although there is a growing understanding of the quality of such data at the device and network level, important challenges in interpreting and evaluating the quality at informational and application levels remain to be explored. This article discusses some of these challenges and solutions of IoT systems at the different OSI layers to understand the factors affecting the quality of the overall system. With the help of two IoT-enabled digital health applications, the authors investigate the role of semantics in measuring the data quality of the system, as well as integrating multimodal data for clinical decision support. They also discuss the extension of IoT to the Internet of Everything by including human-in-the-loop to enhance the system accuracy. This paradigm shift through the confluence of sensors and data analytics can lead to accelerated innovation in applications by overcoming the limitations of the current systems, leading to unprecedented opportunities in healthcare.

Details DOI

IS Journal 2017 Journal Article

On Using the Intelligent Edge for IoT Analytics

  • Pankesh Patel
  • Muhammad Intizar Ali
  • Amit Sheth

This article presents a flexible architecture for Internet of Things (IoT) data analytics using the concept of fog computing. The authors identify different actors and their roles in order to design adaptive IoT data analytics solutions. The presented approach can be used to effectively design robust IoT applications that require a tradeoff between cloud- and edge-based computing depending on dynamic application requirements. The potential use cases of this technology can be found in scenarios such as smart cities, security surveillance, and smart manufacturing, where the quality of user experience is important.

Details DOI

IJCAI Conference 2017 Conference Paper

Relatedness-based Multi-Entity Summarization

  • Kalpa Gunaratna
  • Amir Hossein Yazdavar
  • Krishnaprasad Thirunarayan
  • Amit Sheth
  • Gong Cheng

Representing world knowledge in a machine processable format is important as entities and their descriptions have fueled tremendous growth in knowledge-rich information processing platforms, services, and systems. Prominent applications of knowledge graphs include search engines (e. g. , Google Search and Microsoft Bing), email clients (e. g. , Gmail), and intelligent personal assistants (e. g. , Google Now, Amazon Echo, and Apple's Siri). In this paper, we present an approach that can summarize facts about a collection of entities by analyzing their relatedness in preference to summarizing each entity in isolation. Specifically, we generate informative entity summaries by selecting: (i) inter-entity facts that are similar and (ii) intra-entity facts that are important and diverse. We employ a constrained knapsack problem solving approach to efficiently compute entity summaries. We perform both qualitative and quantitative experiments and demonstrate that our approach yields promising results compared to two other stand-alone state-of-the-art entity summarization approaches.

PDF Details

AAAI Conference 2017 Conference Paper

RQUERY: Rewriting Natural Language Queries on Knowledge Graphs to Alleviate the Vocabulary Mismatch Problem

  • Saeedeh Shekarpour
  • Edgard Marx
  • Sšren Auer
  • Amit Sheth

For non-expert users, a textual query is the most popular and simple means for communicating with a retrieval or question answering system. However, there is a risk of receiving queries which do not match with the background knowledge. Query expansion and query rewriting are solutions for this problem but they are in danger of potentially yielding a large number of irrelevant words, which in turn negatively influences runtime as well as accuracy. In this paper, we propose a new method for automatic rewriting input queries on graph-structured RDF knowledge bases. We employ a Hidden Markov Model to determine the most suitable derived words from linguistic resources. We introduce the concept of triplebased co-occurrence for recognizing co-occurred words in RDF data. This model was bootstrapped with three statistical distributions. Our experimental study demonstrates the superiority of the proposed approach to the traditional n-gram model.

PDF Details

IS Journal 2016 Journal Article

Internet of Things to Smart IoT Through Semantic, Cognitive, and Perceptual Computing

  • Amit Sheth

Rapid growth in the Internet of Things (IoT) has resulted in a massive growth of data generated by these devices and sensors put on the Internet. Physical-cyber-social (PCS) big data consist of this IoT data, complemented by relevant Web-based and social data of various modalities. Smart data is about exploiting this PCS big data to get deep insights and make it actionable, and making it possible to facilitate building intelligent systems and applications. This article discusses key AI research in semantic computing, cognitive computing, and perceptual computing. Their synergistic use is expected to power future progress in building intelligent systems and applications for rapidly expanding markets in multiple industries. Over the next two years, this column on IoT will explore many challenges and technologies on intelligent use and applications of IoT data.

Details DOI

IS Journal 2016 Journal Article

On Searching the Internet of Things: Requirements and Challenges

  • Payam Barnaghi
  • Amit Sheth

Internet of Things (IoT) data services are designed to be available to devices and users on request at any time and at any location. Quality, latency, trust, availability, reliability, and continuity are among the key parameters that impact efficient access and use of IoT data and services. However, current data and service search, discovery, and access methods and solutions are more suited for fewer and/or static (or stored) data and service resources. IoT resources differ in terms of the number of resources and the complexity and amount of data. Efficient discovery, ranking, selection, access, integration, and interpretation and understanding of the data and services requires coordinated efforts among network, data/service provider resources, and core IoT components. This article describes some of the requirements and discusses the key challenges to build scalable and efficient search and discovery mechanisms for the IoT.

Details DOI

AAAI Conference 2016 Conference Paper

Understanding City Traffic Dynamics Utilizing Sensor and Textual Observations

  • Pramod Anantharam
  • Krishnaprasad Thirunarayan
  • Surendra Marupudi
  • Amit Sheth
  • Tanvi Banerjee

Understanding speed and travel-time dynamics in response to various city related events is an important and challenging problem. Sensor data (numerical) containing average speed of vehicles passing through a road link can be interpreted in terms of traffic related incident reports from city authorities and social media data (textual), providing a complementary understanding of traffic dynamics. State-of-the-art research is focused on either analyzing sensor observations or citizen observations; we seek to exploit both in a synergistic manner. We demonstrate the role of domain knowledge in capturing the non-linearity of speed and travel-time dynamics by segmenting speed and travel-time observations into simpler components amenable to description using linear models such as Linear Dynamical System (LDS). Specifically, we propose Restricted Switching Linear Dynamical System (RSLDS) to model normal speed and travel time dynamics and thereby characterize anomalous dynamics. We utilize the city traffic events extracted from text to explain anomalous dynamics. We present a large scale evaluation of the proposed approach on a real-world traffic and twitter dataset collected over a year with promising results.

PDF Details

TIST Journal 2015 Journal Article

Extracting City Traffic Events from Social Streams

  • Pramod Anantharam
  • Payam Barnaghi
  • Krishnaprasad Thirunarayan
  • Amit Sheth

Cities are composed of complex systems with physical, cyber, and social components. Current works on extracting and understanding city events mainly rely on technology-enabled infrastructure to observe and record events. In this work, we propose an approach to leverage citizen observations of various city systems and services, such as traffic, public transport, water supply, weather, sewage, and public safety, as a source of city events. We investigate the feasibility of using such textual streams for extracting city events from annotated text. We formalize the problem of annotating social streams such as microblogs as a sequence labeling problem. We present a novel training data creation process for training sequence labeling models. Our automatic training data creation process utilizes instance-level domain knowledge (e.g., locations in a city, possible event terms). We compare this automated annotation process to a state-of-the-art tool that needs manually created training data and show that it has comparable performance in annotation tasks. An aggregation algorithm is then presented for event extraction from annotated text. We carry out a comprehensive evaluation of the event annotation and event extraction on a real-world dataset consisting of event reports and tweets collected over 4 months from the San Francisco Bay Area. The evaluation results are promising and provide insights into the utility of social stream for extracting city events.

Details DOI

AAAI Conference 2015 Conference Paper

FACES: Diversity-Aware Entity Summarization Using Incremental Hierarchical Conceptual Clustering

  • Kalpa Gunaratna
  • Krishnaparasad Thirunarayan
  • Amit Sheth

Semantic Web documents that encode facts about entities on the Web have been growing rapidly in size and evolving over time. Creating summaries on lengthy Semantic Web documents for quick identification of the corresponding entity has been of great contemporary interest. In this paper, we explore automatic summarization techniques that characterize and enable identification of an entity and create summaries that are human friendly. Specifically, we highlight the importance of diversified (faceted) summaries by combining three dimensions: diversity, uniqueness, and popularity. Our novel diversity-aware entity summarization approach mimics human conceptual clustering techniques to group facts, and picks representative facts from each group to form concise (i. e. , short) and comprehensive (i. e. , improved coverage through diversity) summaries. We evaluate our approach against the state-of-the-art techniques and show that our work improves both the quality and the efficiency of entity summarization.

PDF Details

JBHI Journal 2014 Journal Article

Semantics Driven Approach for Knowledge Acquisition From EMRs

  • Sujan Perera
  • Cory Henson
  • Krishnaprasad Thirunarayan
  • Amit Sheth
  • Suhas Nair

Semantic computing technologies have matured to be applicable to many critical domains such as national security, life sciences, and health care. However, the key to their success is the availability of a rich domain knowledge base. The creation and refinement of domain knowledge bases pose difficult challenges. The existing knowledge bases in the health care domain are rich in taxonomic relationships, but they lack nontaxonomic (domain) relationships. In this paper, we describe a semiautomatic technique for enriching existing domain knowledge bases with causal relationships gleaned from Electronic Medical Records (EMR) data. We determine missing causal relationships between domain concepts by validating domain knowledge against EMR data sources and leveraging semantic-based techniques to derive plausible relationships that can rectify knowledge gaps. Our evaluation demonstrates that semantic techniques can be employed to improve the efficiency of knowledge acquisition.

Details DOI

IS Journal 2013 Journal Article

From Data to Actionable Knowledge: Big Data Challenges in the Web of Things [Guest Editors' Introduction]

  • Payam Barnaghi
  • Amit Sheth
  • Cory Henson

Extending the current Internet and providing connection, communication, and internetworking between devices and physical objects, or "things, " is a growing trend that's often referred to as the Internet of Things (IoT). Integrating real-world data into the Web, with its large repositories of data, and providing Web-based interactions between humans and IoT resources is what the Web of Things (WoT) stands for. Here, the guest editors describe the Big Data issues in the WoT, discuss the challenges of extracting actionable knowledge and insights from raw sensor data, and introduce the theme articles in this special issue.

Details DOI

IS Journal 2013 Journal Article

Physical-Cyber-Social Computing: An Early 21st Century Approach

  • Amit Sheth
  • Pramod Anantharam
  • Cory Henson

Technology plays an increasingly important role in facilitating and improving personal and social activities, engagements, decision making, interaction with physical and social worlds, insight generation, and just about anything that humans, as intelligent beings, seek to do. The term computing for human experience (CHE) captures technology's human-centric role, emphasizing the unobtrusive, supportive, and assistive part technology plays in improving human experience. Here, the authors present an emerging paradigm called physical-cyber-social (PCS) computing, supporting the CHE vision, which encompasses a holistic treatment of data, information, and knowledge from the PCS worlds to integrate, correlate, interpret, and provide contextually relevant abstractions to humans. They also outline the types of computational operators that make up PCS computing.

Details DOI

IS Journal 2007 Journal Article

Semantic Web Services, Part 2

  • David Martin
  • John Domingue
  • Amit Sheth
  • Steve Battle
  • Katia Sycara
  • Dieter Fensel

In part 2 of this Trends & Controversies installment, we continue exploring the state of the art, current practices, and future directions for Semantic Web services. SWS aims to bring Semantic Web technology - for representing, sharing, and reasoning about knowledge - to bear in Web service contexts. The objective is to enable a fuller, more flexible automation of service provision and use and the construction of more powerful tools and methodologies for working with services.

Details DOI