Yuval Shahar

AIIM Journal 2022 Journal Article

A dual-layer context-based architecture for the detection of anomalous instructions sent to medical devices

• Tom Mahler
• Erez Shalom
• Yuval Elovici
• Yuval Shahar

Complex medical devices are controlled by instructions sent from a host personal computer (PC) to the device. Anomalous instructions can introduce many potentially harmful threats to patients (e. g. , radiation overexposure), to physical device components (e. g. , manipulation of device motors), or to functionality (e. g. , manipulation of medical images). Threats can occur due to cyber-attacks, human error (e. g. , using the wrong protocol, or misconfiguring the protocol's parameters by a technician), or host PC software bugs. Thus, anomalous instructions might represent an intentional threat to the patient or to the device, a human error, or simply a non-optimal operation of the device. To protect medical devices, we propose a new dual-layer architecture. The architecture analyzes the instructions sent from the host PC to the physical components of the device, to detect anomalous instructions using two detection layers: (1) an unsupervised context-free (CF) layer that detects anomalies based solely on the instruction's content and inter-correlations; and (2) a supervised context-sensitive (CS) layer that detects anomalies in both the clinical objective and patient contexts using a set of supervised classifiers pre-trained for each specific context. The proposed dual-layer architecture was evaluated in the computed tomography (CT) domain, using 4842 CT instructions that we recorded, including two types of CF anomalous instructions, four types of clinical objective context instructions and four types of patient context instructions. The CF layer was evaluated using 14 unsupervised anomaly detection algorithms. The CS layer was evaluated using six supervised classification algorithms applied to each context (i. e. , clinical objective or patient). Adding the second CS supervised layer to the architecture improved the overall anomaly detection performance (by improving the detection of CS anomalous instructions [when they were not also CF anomalous]) from an F1 score baseline of 72. 6%, to an improved F1 score of 79. 1% to 99. 5% (depending on the clinical objective or patient context used). Adding, the semantics-oriented CS layer enables the detection of CS anomalies using the semantics of the device's procedure, which is not possible when using just the purely syntactic CF layer. However, adding the CS layer also introduced a somewhat increased false positive rate (FPR), and thus reduced somewhat the specificity of the overall process. We conclude that by using both the CF and CS layers, a dual-layer architecture can better detect anomalous instructions to medical devices. The increased FPR might be reduced, in the future, through the use of stronger models, and by training them on more data. The improved accuracy, and the potential capability of adding explanations to both layers, might be useful for creating decision support systems for medical device technicians.

AIIM Journal 2022 Journal Article

Distributed application of guideline-based decision support through mobile devices: Implementation and evaluation

• Erez Shalom
• Ayelet Goldstein
• Elior Ariel
• Moshe Sheinberger
• Valerie Jones
• Boris Van Schooten
• Yuval Shahar

Background Traditionally guideline (GL)-based Decision Support Systems (DSSs) use a centralized infrastructure to generate recommendations to care providers, rather than to patients at home. However, managing patients at home is often preferable, reducing costs and empowering patients. Thus, we wanted to explore an option in which patients, in particular chronic patients, might be assisted by a local DSS, which interacts as needed with the central DSS engine, to manage their disease outside the standard clinical settings. Objectives To design, implement, and demonstrate the technical and clinical feasibility of a new architecture for a distributed DSS that provides patients with evidence-based guidance, offered through applications running on the patients' mobile devices, monitoring and reacting to changes in the patient's personal environment, and providing the patients with appropriate GL-based alerts and personalized recommendations; and increase the overall robustness of the distributed application of the GL. Methods We have designed and implemented a novel projection–callback (PCB) model, in which small portions of the evidence-based guideline's procedural knowledge are projected from a projection engine within the central DSS server, to a local DSS that resides on each patient's mobile device. The local DSS applies the knowledge using the mobile device's local resources. The GL projections generated by the projection engine are adapted to the patient's previously defined preferences and, implicitly, to the patient's current context, in a manner that is embodied in the projected therapy plans. When appropriate, as defined by a temporal pattern within the projected plan, the local DSS calls back the central DSS, requesting further assistance, possibly another projection. To support the new model, the initial specification of the GL includes two levels: one for the central DSS, and one for the local DSS. We have implemented a distributed GL-based DSS using the projection–callback model within the MobiGuide EU project, which automatically manages chronic patients at home using sensors on the patients and their mobile phone. We assessed the new GL specification process, by specifying two very different, complex GLs: for Gestational Diabetes Mellitus, and for Atrial Fibrillation. Then, we evaluated the new computational architecture by applying the two GLs to the automated clinical management, at real time, of patients in two different countries: Spain and Italy, respectively. Results The specification using the new projection-callback model was found to be quite feasible. We found significant differences between the distributed versions of the two GLs, suggesting further research directions and possibly additional ways to analyze and characterize GLs. Applying the two GLs to the two patient populations proved highly feasible as well. The mean time between the central and local interactions was quite different for the two GLs: 3. 95 ± 1. 95 days in the case of the gestational diabetes domain, and 23. 80 ± 12. 47 days, in the case of the atrial fibrillation domain, probably corresponding to the difference in the distributed specifications of the two GLs. Most of the interaction types were due to projections to the local DSS (83%); others were data notifications, mostly to change context (17%). Some of the data notifications were triggered due to technical errors. The robustness of the distributed architecture was demonstrated through the successful recovery from multiple crashes of the local DSS. Conclusions The new projection-callback model has been demonstrated to be feasible, from specification to distributed application. Different GLs might significantly differ, however, in their distributed specification and application characteristics. Distributed medical DSSs can facilitate the remote management of chronic patients by enabling the central DSSs to delegate, in a dynamic fashion, determined by the patient's context, much of the monitoring and treatment management decisions to the mobile device. Patients can be kept in their home environment, while still maintaining, through the projection-callback mechanism, several of the advantages of a central DSS, such as access to the patient's longitudinal record, and to an up-to-date evidence-based GL repository.

AIIM Journal 2017 Journal Article

Evaluation of an automated knowledge-based textual summarization system for longitudinal clinical data, in the intensive care domain

• Ayelet Goldstein
• Yuval Shahar
• Efrat Orenbuch
• Matan J. Cohen

Objectives To examine the feasibility of the automated creation of meaningful free-text summaries of longitudinal clinical records, using a new general methodology that we had recently developed; and to assess the potential benefits to the clinical decision-making process of using such a method to generate draft letters that can be further manually enhanced by clinicians. Methods We had previously developed a system, CliniText (CTXT), for automated summarization in free text of longitudinal medical records, using a clinical knowledge base. In the current study, we created an Intensive Care Unit (ICU) clinical knowledge base, assisted by two ICU clinical experts in an academic tertiary hospital. The CTXT system generated free-text summary letters from the data of 31 different patients, which were compared to the respective original physician-composed discharge letters. The main evaluation measures were (1) relative completeness, quantifying the data items missed by one of the letters but included by the other, and their importance; (2) quality parameters, such as readability; (3) functional performance, assessed by the time needed, by three clinicians reading each of the summaries, to answer five key questions, based on the discharge letter (e. g. , “What are the patient’s current respiratory requirements? ”), and by the correctness of the clinicians’ answers. Results Completeness: In 13/31 (42%) of the letters the number of important items missed in the CTXT-generated letter was actually less than or equal to the number of important items missed by the MD-composed letter. In each of the MD-composed letters, at least two important items that were mentioned by the CTXT system were missed (a mean of 7. 2±5. 74). In addition, the standard deviation in the number of missed items in the MD letters (STD=15. 4) was much higher than the standard deviation in the CTXT-generated letters (STD=5. 3). Quality: The MD-composed letters obtained a significantly better grade in three out of four measured parameters. However, the standard variation in the quality of the MD-composed letters was much greater than the standard variation in the quality of the CTXT-generated letters (STD=6. 25 vs. STD=2. 57, respectively). Functional evaluation: The clinicians answered the five questions on average 40% faster (p< 0. 001) when using the CTXT-generated letters than when using the MD-composed letters. In four out of the five questions the clinicians’ correctness was equal to or significantly better (p <0. 005) when using the CTXT-generated letters than when using the MD-composed letters. Conclusions An automatic knowledge-based summarization system, such as the CTXT system, has the capability to model complex clinical domains, such as the ICU, and to support interpretation and summarization tasks such as the creation of a discharge summary letter. Based on the results, we suggest that the use of such systems could potentially enhance the standardization of the letters, significantly increase their completeness, and reduce the time to write the discharge summary. The results also suggest that using the resultant structured letters might reduce the decision time, and enhance the decision quality, of decisions made by other clinicians.

AIIM Journal 2017 Journal Article

Inter-labeler and intra-labeler variability of condition severity classification models using active and passive learning methods

• Nir Nissim
• Yuval Shahar
• Yuval Elovici
• George Hripcsak
• Robert Moskovitch

Background and objectives Labeling instances by domain experts for classification is often time consuming and expensive. To reduce such labeling efforts, we had proposed the application of active learning (AL) methods, introduced our CAESAR-ALE framework for classifying the severity of clinical conditions, and shown its significant reduction of labeling efforts. The use of any of three AL methods (one well known [SVM-Margin], and two that we introduced [Exploitation and Combination_XA]) significantly reduced (by 48% to 64%) condition labeling efforts, compared to standard passive (random instance-selection) SVM learning. Furthermore, our new AL methods achieved maximal accuracy using 12% fewer labeled cases than the SVM-Margin AL method. However, because labelers have varying levels of expertise, a major issue associated with learning methods, and AL methods in particular, is how to best to use the labeling provided by a committee of labelers. First, we wanted to know, based on the labelers’ learning curves, whether using AL methods (versus standard passive learning methods) has an effect on the Intra-labeler variability (within the learning curve of each labeler) and inter-labeler variability (among the learning curves of different labelers). Then, we wanted to examine the effect of learning (either passively or actively) from the labels created by the majority consensus of a group of labelers. Methods We used our CAESAR-ALE framework for classifying the severity of clinical conditions, the three AL methods and the passive learning method, as mentioned above, to induce the classifications models. We used a dataset of 516 clinical conditions and their severity labeling, represented by features aggregated from the medical records of 1. 9 million patients treated at Columbia University Medical Center. We analyzed the variance of the classification performance within (intra-labeler), and especially among (inter-labeler) the classification models that were induced by using the labels provided by seven labelers. We also compared the performance of the passive and active learning models when using the consensus label. Results The AL methods: produced, for the models induced from each labeler, smoother Intra-labeler learning curves during the training phase, compared to the models produced when using the passive learning method. The mean standard deviation of the learning curves of the three AL methods over all labelers (mean: 0. 0379; range: [0. 0182 to 0. 0496]), was significantly lower (p =0. 049) than the Intra-labeler standard deviation when using the passive learning method (mean: 0. 0484; range: [0. 0275–0. 0724). Using the AL methods resulted in a lower mean Inter-labeler AUC standard deviation among the AUC values of the labelers’ different models during the training phase, compared to the variance of the induced models’ AUC values when using passive learning. The Inter-labeler AUC standard deviation, using the passive learning method (0. 039), was almost twice as high as the Inter-labeler standard deviation using our two new AL methods (0. 02 and 0. 019, respectively). The SVM-Margin AL method resulted in an Inter-labeler standard deviation (0. 029) that was higher by almost 50% than that of our two AL methods The difference in the inter-labeler standard deviation between the passive learning method and the SVM-Margin learning method was significant (p =0. 042). The difference between the SVM-Margin and Exploitation method was insignificant (p =0. 29), as was the difference between the Combination_XA and Exploitation methods (p =0. 67). Finally, using the consensus label led to a learning curve that had a higher mean intra-labeler variance, but resulted eventually in an AUC that was at least as high as the AUC achieved using the gold standard label and that was always higher than the expected mean AUC of a randomly selected labeler, regardless of the choice of learning method (including a passive learning method). Using a paired t-test, the difference between the intra-labeler AUC standard deviation when using the consensus label, versus that value when using the other two labeling strategies, was significant only when using the passive learning method (p =0. 014), but not when using any of the three AL methods. Conclusions The use of AL methods, (a) reduces intra-labeler variability in the performance of the induced models during the training phase, and thus reduces the risk of halting the process at a local minimum that is significantly different in performance from the rest of the learned models; and (b) reduces Inter-labeler performance variance, and thus reduces the dependence on the use of a particular labeler. In addition, the use of a consensus label, agreed upon by a rather uneven group of labelers, might be at least as good as using the gold standard labeler, who might not be available, and certainly better than randomly selecting one of the group’s individual labelers. Finally, using the AL methods: when provided by the consensus label reduced the intra-labeler AUC variance during the learning phase, compared to using passive learning.

JMLR Journal 2012 Journal Article

Mal-ID: Automatic Malware Detection Using Common Segment Analysis and Meta-Features

• Gil Tahan
• Lior Rokach
• Yuval Shahar

This paper proposes several novel methods, based on machine learning, to detect malware in executable files without any need for preprocessing, such as unpacking or disassembling. The basic method (Mal-ID) is a new static (form-based) analysis methodology that uses common segment analysis in order to detect malware files. By using common segment analysis, Mal-ID is able to discard malware parts that originate from benign code. In addition, Mal-ID uses a new kind of feature, termed meta-feature, to better capture the properties of the analyzed segments. Rather than using the entire file, as is usually the case with machine learning based techniques, the new approach detects malware on the segment level. This study also introduces two Mal-ID extensions that improve the Mal-ID basic method in various aspects. We rigorously evaluated Mal-ID and its two extensions with more than ten performance measures, and compared them to the highly rated boosted decision tree method under identical settings. The evaluation demonstrated that Mal-ID and the two Mal-ID extensions outperformed the boosted decision tree method in almost all respects. In addition, the results indicated that by extracting meaningful features, it is sufficient to employ one simple detection rule for classifying executable files. [abs] [ pdf ][ bib ] &copy JMLR 2012. ( edit, beta )

AIIM Journal 2011 Journal Article

Artificial Intelligence in Medicine AIME 2009

• Yuval Shahar
• Carlo Combi

AIIM Journal 2010 Journal Article

Intelligent visualization and exploration of time-oriented data of multiple patients

• Denis Klimov
• Yuval Shahar
• Meirav Taieb-Maimon

Objective Clinicians and medical researchers alike require useful, intuitive, and intelligent tools to process large amounts of time-oriented multiple-patient data from multiple sources. For analyzing the results of clinical trials or for quality assessment purposes, an aggregated view of a group of patients is often required. To meet this need, we designed and developed the VISualizatIon of Time-Oriented RecordS (VISITORS) system, which combines intelligent temporal analysis and information visualization techniques. The VISITORS system includes tools for intelligent retrieval, visualization, exploration, and analysis of raw time-oriented data and derived (abstracted) concepts for multiple patient records. To derive meaningful interpretations from raw time-oriented data (known as temporal abstractions), we used the knowledge-based temporal-abstraction method. Methods The main module of the VISITORS system is an interactive, ontology-based exploration module, which enables the user to visualize raw data and abstract (derived) concepts for multiple patient records, at several levels of temporal granularity; to explore these concepts; and to display associations among raw and abstract concepts. A knowledge-based delegate function is used to convert multiple data points into one delegate value representing each temporal granule. To select the population of patients to explore, the VISITORS system includes an ontology-based temporal-aggregation specification language and a graphical expression-specification module. The expressions, applied by an external temporal mediator, retrieve a list of patients, a list of relevant time intervals, and a list of time-oriented patients’ data sets, by using an expressive set of time and value constraints. Results Functionality and usability evaluation of the interactive exploration module was performed on a database of more than 1000 oncology patients by a group of 10 users—five clinicians and five medical informaticians. Both types of users were able in a short time (mean of 2. 5±0. 2min per question) to answer a set of clinical questions, including questions that require the use of specialized operators for finding associations among derived temporal abstractions, with high accuracy (mean of 98. 7±2. 4 on a predefined scale from 0 to 100). There were no significant differences between the response times and between accuracy levels of the exploration of the data using different time lines, i. e. , absolute (i. e. , calendrical) versus relative (referring to some clinical key event). A system usability scale (SUS) questionnaire filled out by the users demonstrated the VISITORS system to be usable (mean score for the overall group: 69. 3), but the clinicians’ usability assessment was significantly lower than that of the medical informaticians. Conclusions We conclude that intelligent visualization and exploration of longitudinal data of multiple patients with the VISITORS system is feasible, functional, and usable.

UAI Conference 2010 Conference Paper

Irregular-Time Bayesian Networks

• Michael Ramati
• Yuval Shahar

In many fields observations are performed irregularly along time, due to either measurement limitations or lack of a constant immanent rate. While discrete-time Markov models (as Dynamic Bayesian Networks) introduce either inefficient computation or an information loss to reasoning about such processes, continuous-time Markov models assume either a discrete state space (as Continuous-Time Bayesian Networks), or a flat continuous state space (as stochastic differential equations). To address these problems, we present a new modeling class called Irregular-Time Bayesian Networks (ITBNs), generalizing Dynamic Bayesian Networks, allowing substantially more compact representations, and increasing the expressivity of the temporal dynamics. In addition, a globally optimal solution is guaranteed when learning temporal systems, provided that they are fully observed at the same irregularly spaced time-points, and a semiparametric subclass of ITBNs is introduced to allow further adaptation to the irregular nature of the available data.

AIIM Journal 2008 Journal Article

Evaluation of an architecture for intelligent query and exploration of time-oriented clinical data

• Susana B. Martins
• Yuval Shahar
• Dina Goren-Bar
• Maya Galperin
• Herbert Kaizer
• Lawrence V. Basso
• Deborah McNaughton
• Mary K. Goldstein

Objective Evaluate KNAVE-II, a knowledge-based framework for visualization, interpretation, and exploration of longitudinal clinical data, clinical concepts and patterns. KNAVE-II mediates queries to a distributed temporal-abstraction architecture (IDAN), which uses a knowledge-based problem-solving method specializing in on-the-fly computation of clinical queries. Methods A two-phase, balanced cross-over study to compare efficiency and satisfaction of a group of clinicians when answering queries of variable complexity about time-oriented clinical data, typical for oncology protocols, using KNAVE-II, versus standard methods: both paper charts and a popular electronic spreadsheet (ESS) in Phase I; an ESS in Phase II. The measurements included the time required to answer and the correctness of answer for each query and each complexity category, and for all queries, assessed versus a predetermined gold standard set by a domain expert. User satisfaction was assessed by the Standard Usability Score (SUS) tool-specific questionnaire and by a “Usability of Tool Comparison” comparative questionnaire developed for this study. Results In both evaluations, subjects answered higher-complexity queries significantly faster using KNAVE-II than when using paper charts or an ESS up to a mean of 255s difference per query versus the ESS for hard queries (p =0. 0003) in the second evaluation. Average correctness scores when using KNAVE-II versus paper charts, in the first phase, and the ESS, in the second phase, were significantly higher over all queries. In the second evaluation, 91. 6% (110/120) of all of the questions asked within queries of all levels produced correct answers using KNAVE-II, opposed to only 57. 5% (69/120) using the ESS (p <0. 0001). User satisfaction with KNAVE-II was significantly superior compared to using either a paper chart or the ESS (p =0. 006). Clinicians ranked KNAVE-II superior to both paper and the ESS. Conclusions An evaluation of the functionality and usability of KNAVE-II and its supporting knowledge-based temporal-mediation architecture has produced highly encouraging results regarding saving of physician time, enhancement of accuracy of clinical assessment, and user satisfaction.

AIIM Journal 2006 Journal Article

Distributed, intelligent, interactive visualization and exploration of time-oriented clinical data and their abstractions

• Yuval Shahar
• Dina Goren-Bar
• David Boaz
• Gil Tahan

Objectives We present KNAVE-II, an intelligent interface to a distributed architecture specific to the tasks of query, knowledge-based interpretation, summarization, visualization, interactive exploration of large numbers of distributed time-oriented clinical data, and dynamic sensitivity analysis of these data. KNAVE-II main contributions to the fields of temporal reasoning and intelligent user interfaces are: (1) the capability for interactive computation and visualization of domain specific temporal abstractions, supported by ALMA – a computational engine that applies the domain knowledge base to the clinical time-oriented database. (2) Semantic (ontology-based) navigation and exploration of the data, knowledge, and temporal abstractions, supported by the IDAN mediator, a distributed architecture that enables runtime access to domain-specific knowledge bases that are maintained by expert physicians. Methods and materials KNAVE-II was designed according to 12 requirements that were defined through iterative cycles of design and user-centered evaluation. The complete architecture has been implemented and evaluated in a cross-over study design that compared the KNAVE-II module versus two existing methods: paper charts and an Excel electronic spreadsheet. A small group of clinicians answered the same queries, using the domain of oncology and a set of 1000 patients followed after bone-marrow transplantation. Results The results show that users are able to perform medium to hard difficulty level queries faster and more accurately by using KNAVE-II than paper charts and Excel. Moreover, KNAVE-II was ranked first in preference by all users, along all usability dimensions. Conclusions Initial evaluation of KNAVE-II and its supporting knowledge based temporal-mediation architecture, by applying it to a large data base of patients monitored several years after bone marrow transplantation (BMT), has produced highly encouraging results.

AIIM Journal 2006 Journal Article

Multiple hierarchical classification of free-text clinical guidelines

• Robert Moskovitch
• Shiva Cohen-Kashi
• Uzi Dror
• Iftah Levy
• Amit Maimon
• Yuval Shahar

Objective Manual classification of free-text documents within a predefined hierarchy, commonly required in the medical domain, is highly time consuming task. We present an approach based on supervised learning to automate the classification of clinical guidelines into predefined hierarchical conceptual categories. Methods and material Given a set of hierarchically categorized documents in the training stage the learning algorithm exploits the hierarchical structure of the concepts in order to overcome the low number of training examples. The classification task is thus decomposed into a continuous decision process, unlike searching within a decision tree, which follows the concept hierarchy and makes a single decision at each node on the path, multiple paths can be chosen. Classification is based on applying a similarity function at each concept. Several evaluation measures were used, based on the intended use of the hierarchy. In addition, conservative and aggressive stop-criterion strategies for stopping the search through the concept hierarchy were formulated. An evaluation of the approach, including several training methods and multiple evaluation measures, has been performed using a training set of 1136 guidelines from the National Guideline Clearing House set. Results Based on a test collection consisting of 1038 clinical practice guidelines (CPGs) classified along two hierarchies, of roughly 5000 concepts, in which each CPG was classified by a mean of 10 concepts, a variable precision was observed from 44% to 60% depending on the settings of the training methods. Conclusion These results demonstrate the feasibility of the approach, especially when considering the low ratio of guidelines to classification indices (concepts) in the evaluation data set used here.

AIIM Journal 2005 Journal Article

A framework for distributed mediation of temporal-abstraction queries to clinical databases

• David Boaz
• Yuval Shahar

Objective: The specification and creation of a distributed system that integrates medical knowledge bases with time-oriented clinical databases; the goal is to answer complex temporal queries regarding both raw data and its abstractions, such as are often required in medical applications. Methods: (1) Specification, design, and implementation of a generalized access method to a set of heterogeneous clinical data sources, by using a virtual medical-record interface and by mapping the local terms to a set of standardized medical vocabularies; (2) specification of a generalized interface to a set of knowledge sources; (3) specification and implementation of a service, called ALMA that computes complex time-oriented medical queries that include both raw data and abstractions derivable from it; (4) design and implementation of a mediator, called IDAN, that answers raw-data and abstract queries by integrating the appropriate clinical data with the relevant medical knowledge and uses the computation service to answer the queries; (5) an expressive language that enables definition of time-dependent medical queries, which are referred to the mediator; (6) evaluation of the effect of the system, when combined with a new visual interface, called KNAVE-II, on the speed and accuracy of answering a set of complex queries in an oncology sub domain, by a group of clinicians, compared to answering these queries using paper or an electronic spreadsheet. Results: We have implemented the full IDAN architecture. The IDAN/KNAVE-II combination significantly increased the accuracy and speed of answering complex queries about both the data and their abstractions, compared to the standard tools. Conclusion: The implemented architecture proves the feasibility of the distributed integration of medical knowledge sources with clinical data of heterogeneous sources. The results suggest that the proposed IDAN modular architecture has potential significance for supporting the automation of clinical tasks such as diagnosis, monitoring, therapy, and quality assessment.

TIME Conference 1999 Conference Paper

A Constraint-Based Specification of Periodic Patterns in Time-Oriented Data

• Shubha Chakravarty
• Yuval Shahar

We use a constraint based language to specify periodic temporal patterns. The Constraint-based Pattern Specification Language (CAPSUL) is simple to use, but allows a wide variety of patterns to be expressed. CAPSUL solves problems such as: (1) how to use calendar based constraints to define repetition of a periodic event; (2) what temporal relations must exist between consecutive repeats of a pattern; and (3) how expressivity is limited if the same temporal relations must hold between each pair of intervals in the pattern. We implemented CAPSUL in a temporal-abstraction system called Resume, and used it in a graphical knowledge acquisition tool to acquire domain-specific knowledge from experts about patterns to be found in large databases. We summarize the results of preliminary experiments using the pattern specification and pattern detection tools on data about patients who have cancer and have been seen at the University of Chicago bone marrow transplantation center.

AIIM Journal 1999 Journal Article

Representation of change in controlled medical terminologies

• Diane E Oliver
• Yuval Shahar
• Edward H Shortliffe
• Mark A Musen

Computer-based systems that support health care require large controlled terminologies to manage names and meanings of data elements. These terminologies are not static, because change in health care is inevitable. To share data and applications in health care, we need standards not only for terminologies and concept representation, but also for representing change. To develop a principled approach to managing change, we analyze the requirements of controlled medical terminologies and consider features that frame knowledge-representation systems have to offer. Based on our analysis, we present a concept model, a set of change operations, and a change-documentation model that may be appropriate for controlled terminologies in health care. We are currently implementing our modeling approach within a computational architecture.

ICAPS Conference 1998 Conference Paper

AsbruView: Visualization of Time-Oriented, Skeletal Plans

• Silvia Miksch
• Robert Kosara
• Yuval Shahar
• Peter D. Johnson 0001

Skeletal plans are a powerful way to reuse existing domain-specific procedural knowledge. The main drawbacks are that the compositions and the interdependencies of different skeletal plans and their components are not lucid. The aim of this paper is to overcome these limitations and to present the visualization of time-oriented, skeletal plans. Within the Asgaard project, we have developed a time-oriented and intention-based language, called Asbru, to represent such skeletal plans. The Asbru syntax is defined in Backus-Naur form (BNF). Reading BNF or similar forms are next to impossible even for domain experts. We explored different representations and automated knowledge-acquisition tools. However, the domain experts did not accept any of these representations. Consequently, we investigated different metaphor graphics and ended up with a plan visualization utilizing the metaphors of "tracks" and "traffic", called AsbruView. We formatively evaluated different approaches of this plan visualization with physicians applying treatment protocols of mechanical ventilated newborn infants.

TIME Conference 1998 Conference Paper

Model-Based Visualization of Temporal Abstractions

• Yuval Shahar
• Cleve Cheng

We describe a domain-independent framework (KNAVE) specific to the task of interpretation, summerization, visualization, explanation, and interactive navigation in a context-sensitive manner through time-oriented raw data and the multiple levels of higher-level, interval-based concepts that can be abstracted from these data. The KNAVE domain-independent navigation operators access the domain-specific knowledge base, which is modeled by the formal ontology of the knowledge-based temporal-abstraction method; the method generates the temporal abstractions from the time-oriented database. Thus, domain-specific knowledge underlies the semantics of the domain-independent visualization and navigation processes. By accessing the domain-specific temporal abstraction knowledge base and the domain-specific time-oriented database, KNAVE modules enable users to query for domain-specific temporal abstractions and to change the focus of the visualization, thus reusing for a different task (visualization and navigation) the domain model that has been acquired from the domain experts for the purpose of temporal abstraction. Initial evaluation of the KNAVE prototype has been encouraging. The KNAVE methodology has potentially broad implications for tasks such as planning, monitoring, explanation, and data mining.

AIIM Journal 1998 Journal Article

The Asgaard project: a task-specific framework for the application and critiquing of time-oriented clinical guidelines

• Yuval Shahar
• Silvia Miksch
• Peter Johnson

Clinical guidelines can be viewed as generic skeletal-plan schemata that represent clinical procedural knowledge and that are instantiated and refined dynamically by care providers over significant time periods. In the Asgaard project, we are investigating a set of tasks that support the application of clinical guidelines by a care provider other than the guideline's designer. We are focusing on the application of the guideline, recognition of care providers' intentions from their actions, and critique of care providers' actions given the guideline and the patient's medical record. We are developing methods that perform these tasks in multiple clinical domains, given an instance of a properly represented clinical guideline and an electronic medical patient record. In this paper, we point out the precise domain-specific knowledge required by each method, such as the explicit intentions of the guideline designer (represented as temporal patterns to be achieved or avoided). We present a machine-readable language, called Asbru, to represent and to annotate guidelines based on the task-specific ontology. We also introduce an automated tool for the acquisition of clinical guidelines based on the same ontology, developed using the PROTÉGÉ-II framework.

UAI Conference 1998 Conference Paper

Utility Elicitation as a Classification Problem

• Urszula Chajewska
• Lise Getoor
• Joseph Norman
• Yuval Shahar

We investigate the application of classification techniques to utility elicitation. In a decision problem, two sets of parameters must generally be elicited: the probabilities and the utilities. While the prior and conditional probabilities in the model do not change from user to user, the utility models do. Thus it is necessary to elicit a utility model separately for each new user. Elicitation is long and tedious, particularly if the outcome space is large and not decomposable. There are two common approaches to utility function elicitation. The first is to base the determination of the users utility function solely ON elicitation OF qualitative preferences.The second makes assumptions about the form AND decomposability OF the utility function.Here we take a different approach: we attempt TO identify the new USERs utility function based on classification relative to a database of previously collected utility functions. We do this by identifying clusters of utility functions that minimize an appropriate distance measure. Having identified the clusters, we develop a classification scheme that requires many fewer and simpler assessments than full utility elicitation and is more robust than utility elicitation based solely on preferences. We have tested our algorithm on a small database of utility functions in a prenatal diagnosis domain and the results are quite promising.

AIJ Journal 1997 Journal Article

A framework for knowledge-based temporal abstraction

• Yuval Shahar

A new domain-independent knowledge-based inference structure is presented, specific to the task of abstracting higher-level concepts from time-stamped data. The framework includes a model of time, parameters, events and contexts. A formal specification of a domain's temporal abstraction knowledge supports acquisition, maintenance, reuse and sharing of that knowledge. The knowledge-based temporal abstraction method decomposes the temporal abstraction task into five subtasks. These subtasks are solved by five domain-independent temporal abstraction mechanisms. The temporal abstraction mechanisms depend on four domain-specific knowledge types: structural, classification (functional), temporal semantic (logical) and temporal dynamic (probabilistic) knowledge. Domain values for all knowledge types are specified when a temporal abstraction system is developed. The knowledge-based temporal abstraction method has been implemented in the RÉSUMÉ system and has been evaluated in several clinical domains (protocol-based care, monitoring of children's growth and therapy of diabetes) and in an engineering domain (monitoring of traffic control), with encouraging results.

TIME Conference 1997 Conference Paper

Knowledge-Based Temporal Interpolation

• Yuval Shahar

Temporal interpolation is the task of bridging gaps between time-oriented concepts in a context-sensitive manner. It is a subtask important for solving the temporal-abstraction task-abstraction of interval-based, higher-level concepts from time-stamped data. We present a knowledge-based approach to the temporal-interpolation task and discuss in detail the precise knowledge required by that approach, its theoretical foundations, and the implications of the approach. The temporal-interpolation computational mechanism we discuss relies, among other knowledge types, on a temporal-persistence model. The temporal-persistence model employs local temporal-persistence functions that are temporally bidirectional (i. e. , extend a belief measure in a predicate both into the future and into the past) and global, maximal-gap temporal-persistence functions that bridge gaps between interval-based predicates. We investigate the quantitative and qualitative properties implied by both types of persistence functions. We have implemented our approach and evaluated it in several different domains. We discuss its implications for acquisition, maintenance, reuse, and sharing of temporal-abstraction knowledge.

TIME Conference 1996 Conference Paper

Dynamic Temporal Interpretation Contexts for Temporal Abstraction

• Yuval Shahar

The temporal-abstraction task is the task of abstracting higher-level concepts from time-stamped data in a context-sensitive manner. We have developed and implemented a formal knowledge-based framework for decomposing and solving that task that supports acquisition, maintenance, reuse, and sharing of temporal-abstraction knowledge. We present the logical model underlying the representation and runtime formation of interpretation contexts. Interpretation contexts are relevant for abstraction of time-oriented data and are induced by input data, concluded abstractions, external events, goals of the temporal-abstraction process, and certain combinations of interpretation contexts. Knowledge about interpretation contexts is represented as a context ontology and as a dynamic induction relation over interpretation contexts and other proposition types. Induced interpretation contexts are either basic, composite, generalized, or nonconvex. We discuss the advantages of separating explicitly interpretation-context propositions from the propositions inducing them and from the abstractions created within them.

AIIM Journal 1996 Journal Article

Knowledge-based temporal abstraction in clinical domains

• Yuval Shahar
• Mark A Musen

We have defined a knowledge-based framework for the creation of abstract, interval-based concepts from time-stamped clinical data, the knowledge-based temporal-abstraction (KBTA) method. The KBTA method decomposes its task into five subtasks; for each subtask we propose a formal solving mechanism. Our framework emphasizes explicit representation of knowledge required for abstraction of time-oriented clinical data, and facilitates its acquisition, maintenance, reuse and sharing. The RÉSUMÉ system implements the KBTA method. We tested RÉSUMÉ in several clinical-monitoring domains, including the domain of monitoring patients who have insulin-dependent diabetes. We acquired from a diabetes-therapy expert diabetes-therapy temporal-abstraction knowledge. Two diabetes-therapy experts (including the first one) created temporal abstractions from about 800 points of diabetic-patients' data. RÉSUMÉ generated about 80% of the abstractions agreed by both experts; about 97% of the generated abstractions were valid. We discuss the advantages and limitations of the current architecture.

AIIM Journal 1995 Journal Article

Ontology-based configuration of problem-solving methods and generation of knowledge-acquisition tools: application of PROTÉGÉ-II to protocol-based decision support

• Samson W Tu
• Henrik Eriksson
• John H Gennari
• Yuval Shahar
• Mark A Musen

PROTÉGÉ-II is a suite of tools and a methodology for building knowledge-based systems and domain-specific knowledge-acquisition tools. In this paper, we show how PROTÉGÉ-II can be applied to the task of providing protocol-based decision support in the domain of treating HIV-infected patients. To apply PROTÉGÉ-II, (1) we construct a decomposable problem-solving method called episodic skeletal-plan refinement, (2) we build an application ontology that consists of the terms and relations in the domain, and of method-specific distinctions not already captured in the domain terms, and (3) we specify mapping relations that link terms from the application ontology to the domain-independent terms used in the problem-solving method. From the application ontology, we automatically generate a domain-specific knowledge-acquisition tool that is custom-tailored for the application. The knowledge-acquisition tool is used for the creation and maintenance of domain knowledge used by the problem-solving method. The general goal of the PROTÉGÉ-II approach is to produce systems and components that are reusable and easily maintained. This is the rationale for constructing ontologies and problem-solving methods that can be composed from a set of smaller-grained methods and mechanisms. This is also why we tightly couple the knowledge-acquisition tools to the application ontology that specifies the domain terms used in the problem-solving systems. Although our evaluation is still preliminary, for the application task of providing protocol-based decision support, we show that these goals of reusability and easy maintenance can be achieved. We discuss design decisions and the tradeoffs that have to be made in the development of the system.

AIJ Journal 1995 Journal Article

Task modeling with reusable problem-solving methods

• Henrik Eriksson
• Yuval Shahar
• Samson W. Tu
• Angel R. Puerta
• Mark A. Musen

Problem-solving methods for knowledge-based systems establish the behavior of such systems by defining the roles in which domain knowledge is used and the ordering of inferences. Developers can compose problem-solving methods that accomplish complex application tasks from primitive, reusable methods. The key steps in this development approach are task analysis, method selection (from a library), and method configuration. Protégé-ii is a knowledge-engineering environment that allows developers to select and configure problem-solving methods. In addition, Protégé-ii generates domain-specific knowledge-acquisition tools that domain specialists can use to create knowledge bases on which the methods may operate. The board-game method is a problem-solving method that defines control knowledge for a class of tasks that developers can model in a highly specific way. The method adopts a conceptual model of problem solving in which the solution space is construed as a “game board” on which the problem solver moves “playing pieces” according to prespecified rules. This familiar conceptual model simplifies the developer's cognitive demands when configuring the board-game method to support new application tasks. We compare configuration of the board-game method to that of a chronological-backtracking problem-solving method for the same application tasks (for example, towers of Hanoi and the Sisyphus room-assignment problem). We also examine how method designers can specialize problem-solving methods by making ontological commitments to certain classes of tasks. We exemplify this technique by specializing the chronological-backtracking method to the board-game method.