Authors: Yao Ming, Panpan Xu, Furui Cheng, Huamin Qu, Liu Ren
Abstract: Recently we have witnessed a growing adoption of deep sequence models in many application domains including predictive health care, natural language analysis and machine log analysis due to their superior performance compared to traditional machine learning models. However, the intricate process of training and fine-tuning the models confines their accessibility to expert machine learning practitioners. The black-box nature of many deep sequence models (e.g. LSTMs) also makes it a difficult task to incorporate domain-specific knowledge or constraints known to the domain experts in the model. In ProtoVis (Prototype Visualization), we tackle the challenge of directly involving the domain experts in steering a deep sequence model without relying on machine learning practitioners as intermediaries. Our approach utilizes ProSeNet (Prototype Sequence Network), which combines deep sequence modeling with prototype learning for both predictive accuracy and interpretability. Prototype learning is a form of case-based reasoning which imitates the common human problem-solving process of consulting past experiences to solve new problems. The key component of ProSeNet is a small set of exemplar cases (i.e. prototypes) which are constructed using historical data. In ProtoVis they serve both as an efficient visual summary of the original data and explanations of the model decisions. With ProtoVis the domain experts can inspect, critique and revise the prototypes interactively. The system then incorporates user-specified prototypes and incrementally updates the model. We conduct extensive case studies on a wide range of application domains including sentiment analysis on texts and predictive diagnostics based on vehicle fault logs. The results of the case studies and the expert interviews demonstrate that ProtoVis can indeed help domain experts obtain interpretable models with more concise prototypes. Quantitative results also show that the fine-tuned model is able to incorporate user specified prototypes with similar predictive accuracy as the original model.