Abhinav Maurya

ML/DS Researcher Data Scientist Google Seattle, WA Email:

Selected Research

• AWS Capacity Forecasting and Planning
• Simulate, Optimize, Predict: Building a Cloud Admission Control Policy That Never Stops Learning the Perfect Decisions. Abhinav Maurya, Pawel Cholewinski, Kerem Bulbul, John Dunagan. Patent pending.
• MIP Optimization-based Budget Setting for AWS EC2 Capacity Pool Protection. Abhinav Maurya. Patent pending.
• Predictive Scaling of EC2 APIs. Abhinav Maurya, Andy Becker. Research presented at VP-level all-hands meeting. Patent pending.
• Human-AI Collaboration and the Future of Work
• Understanding Human-AI Work Collaboration using a Randomized Field Study. Abhinav Maurya, Sunder Kekre, Rahul Telang. Poster presentation at ACM Collective Intelligence 2019, awarded conference registration scholarship. Presented at AAAI HCOMP 2018 Doctoral Consortium, awarded student travel grant. [ abstract | extended abstract | pdf | poster ]
  With increasing adoption of artificial intelligence technologies, there is a growing interest in understanding effective strategies for human-AI collaboration in the workplace. In this paper, we investigate whether human workers carrying out natural language data annotation tasks can perform better if aided with data-driven annotation recommendations from an ensemble of machine learning models. Our randomized experimental study uses a common natural language processing task called named entity tagging. Annotators in the treatment group are provided highlighted named entity suggestions in the text they are annotating, whereas those in the control group are not. The treatment group significantly outperforms the control group on evaluation metrics such as precision, recall, and throughput, indicating that humans were successfully able to incorporate AI suggestions in completing their work. The treatment group evaluation metrics also outperformed those of the ensemble machine learning model providing the suggestions. A careful analysis revealed that AI suggestions were more accurate than human annotations (control group) in the regime of shorter named entities, whereas humans performed better than automated suggestions in the regime of longer named entities. The treatment group effectively accepted the AI suggestions in the former regime and corrected AI mistakes in the latter regime, thereby surpassing the performance of either AI or human alone.
• Optimal Transport and Occupational Skill Analysis
• Understanding Job-Skill Relationships using Big Data and Neural Networks. Abhinav Maurya. Bloomberg Data for Good Exchange 2018. [ abstract | pdf ]
  Nonlinear vector space embeddings have shown great promise in many applications such as similarity search, analogy mappings, dataset visualization, etc. In this paper, we propose a simple distribution-to-distribution regression model based on neural networks which provides (i) job2vec: interpretable embeddings of jobs in terms of associated skills, (ii) skill2vec: converse embeddings of skills in terms of the jobs that require them, and (iii) SkillRank: a mechanism for ranking skills associated with each job that can be recommended to members aspiring for that job. Due to the simplicity of our model, it has no hyperparameters that need tuning except for the number of stochastic gradient descent iterations which is easily determined using the early stopping criterion. We dicsuss preliminary results and insights from our model using data from a major professional social network.
• Optimal Transport Embeddings for Understanding Occupations and Skills in the Labor Market. Abhinav Maurya. Working paper. [ abstract ]
  Vector space embeddings have shown great promise in many applications such as similarity search, analogy mappings, dataset visualization, etc. In this paper, we propose a simple distribution-to-distribution optimal transport approach for (i) learning occupation and skill embeddings (OccupationEmbedding and SkillEmbedding) and (ii) identifying occupation-specific skills from these embeddings using OccupationSkillRanking with the goal of career-driven skill training for unemployed and underemployed workers. Our approach beats a number of baselines on metrics evaluating the quality of occupation-personalized skill recommendations. We conjecture that the Wasserstein metric can effectively model the distance between entity histograms such as those associated with multidiscipilinary occupation histories and skill sets. Reducing such a histogram to a single embedding by using a pooling operation such as taking the mean embedding over entities can lose valuable information about the set of entities (occupations/skills), and lead to poorer performance on downstream tasks such as skill recommendation.
• Bayesian Matching Models for Closing Labor Market Skill Gaps
• Bayesian Multi-View Models for Member-Job Matching and Personalized Skill Recommendations. Abhinav Maurya, Rahul Telang. IEEE International Conference on Big Data 2017 (acceptance rate: 17.8%). Awarded IEEE Big Data student travel grant. Finalist for 2017 INFORMS Social Media Section's Best Paper Award. IZA 2018 Workshop on Matching Workers and Jobs Online. Preliminary version presented in poster sessions at Amazon Graduate Research Symposium 2017, CMU Innovation with Impact Exhibition 2017, and ISBIS 2017. Patent pending. [ abstract | pdf ]
  Inefficiencies in the labor market such as friction in matching members to jobs and the existence of skill gaps in various sectors of the economy are considered to be major problems facing economies today. The central premise of our work is that increasing the productivity of a member of the workforce (and thereby of the economy as a whole) crucially depends on identifying and recommending skills whose acquisition will yield the highest utility gains for that member. To this end, we develop novel unsupervised Bayesian multi-view models BayesMatch and NpBayesMatch which can match members to other similar members as well as relevant jobs using shared features. The matching step is followed by a skill recommendation step SkillR which makes demand-based skill recommendations to members. Our extensive quantitative evaluation using a rich dataset comprised of member profiles and job postings from a major professional networking site suggests that skill recommendations made by our algorithm are highly correlated with skills demanded in heldout future jobs compared to those made by traditional collaborative filtering algorithms that do not utilize information about skill demand. This indicates that either members of the workforce do not have skills demanded by jobs or do not have enough information about which are the best skills to signal for competing in the labor market.
• Understanding Tech Organizations using Big Data
• A Lens into Employee Peer Reviews via Sentiment-Aspect Modeling. Abhinav Maurya, Leman Akoglu, Ramayya Krishnan, Daniel Bay. ACM/IEEE ASONAM 2018. [ abstract | pdf ]
  Given a corpus of employee peer reviews from a large corporation where each review is structured into pros and cons, what are the prevalent traits that employees talk about? How can we describe the performance of an employee with just a few sentences, that help us interpret what their work is praised and criticized for? What is the best way to summarize an employee's reviews, while preserving the content and sentiment as well as possible? In this work, we study a large collection of corporation-wide employee peer reviews from a technology enterprise. Motivated by the challenges we outline in our analysis of employee review data, our work makes two main contributions in the domain of people analytics: (a) Sentiment-Aspect Model (SAM): we introduce a stylized log-linear model that identifies the hidden aspects and sentiment within an employee peer review corpus, (b) Interpretable Sentiment-Aspect Representations (Employee2Vec): using SAM, we produce a vector space embedding for each employee, containing an overall sentiment score per aspect, and (c) Summarization of Employee Peer Reviews (PeerSum): we summarize an employee's peer reviews with just a few sentences which reflect the most prevalent traits and associated sentiment for the employee as much as possible. We show that our model SAM can use the structure present in the dataset as supervision to discover meaningful latent traits and sentiment embodied in the reviews. Our employee vector representations Employee2Vec provide a compact, interpretable overview of their evaluation. The review summaries extracted by PeerSum provide text that explains the professional performance of an employee in a succinct and objectively quantifiable way. We also show how to use our techniques for people analytics tasks such as the analysis of thematic differences between departments, regions, and genders.
• Contrastive Anomaly Detection
• Semantic Scan: Detecting Subtle, Spatially Localized Events in Text Streams. Abhinav Maurya, Kenton Murray, Yandong Liu, Chris Dyer, William W. Cohen, Daniel B. Neill. Technical Report. Winner of seventh Yelp dataset challenge. [ abstract | pdf ]
  Early detection and precise characterization of emerging topics in text streams can be highly useful in applications such as timely and targeted public health interventions and discovering evolving regional business trends. Many methods have been proposed for detecting emerging events in text streams using topic modeling. However, these methods have numerous shortcomings that make them unsuitable for rapid detection of locally emerging events on massive text streams. In this paper, we describe Semantic Scan (SS) that has been developed specifically to overcome these shortcomings in detecting new spatially compact events in text streams. Semantic Scan integrates novel contrastive topic modeling with online document assignment and principled likelihood ratio-based spatial scanning to identify emerging events with unexpected patterns of keywords hidden in text streams. This enables more timely and accurate detection and characterization of anomalous, spatially localized emerging events. Semantic Scan does not require manual intervention or labeled training data, and is robust to noise in real-world text data since it identifies anomalous text patterns that occur in a cluster of new documents rather than an anomaly in a single new document. We compare Semantic Scan to alternative state-of-the-art methods such as Topics over Time, Online LDA, and Labeled LDA on two real-world tasks: (i) a disease surveillance task monitoring free-text Emergency Department chief complaints in Allegheny County, and (ii) an emerging business trend detection task based on Yelp reviews. On both tasks, we find that Semantic Scan provides significantly better event detection and characterization accuracy than competing approaches, while providing up to an order of magnitude speedup.
• Contrastive Structured Anomaly Detection for Gaussian Graphical Models. Abhinav Maurya, Mark Cheung. ICML 2016 Workshop on Anomaly Detection. ACM/IEEE ASONAM 2018. [ abstract | pdf ]
  Gaussian graphical models (GGMs) are probabilistic tools of choice for analyzing conditional dependencies between variables in complex systems. Finding changepoints in the structural evolution of a GGM is therefore essential to detecting anomalies in the underlying system modeled by the GGM. In order to detect structural anomalies in a GGM, we consider the problem of estimating changes in the precision matrix of the corresponding Gaussian distribution. We take a two-step approach to solving this problem:- (i) estimating a background precision matrix using system observations from the past without any anomalies, and (ii) estimating a foreground precision matrix using a sliding temporal window during anomaly monitoring. Our primary contribution is in estimating the foreground precision using a novel contrastive inverse covariance estimation procedure. In order to accurately learn only the structural changes to the GGM, we maximize a penalized log-likelihood where the penalty is the l1 norm of difference between the foreground precision being estimated and the already learned background precision. We modify the alternating direction method of multipliers (ADMM) algorithm for sparse inverse covariance estimation to perform contrastive estimation of the foreground precision matrix. Our results on simulated GGM data show significant improvement in precision and recall for detecting structural changes to the GGM, compared to a non-contrastive sliding window baseline.
• Bayesian Optimization of Non-convex Metrics for Binary Classification on Imbalanced Datasets
• Optimizing Predictive Precision in Imbalanced Datasets for Actionable Revenue Change Prediction. Pravar Dilip Mahajan, Abhinav Maurya, Aly Megahed, Alaa Elwany, Ray Strong, Jeanette Blomberg. 11th INFORMS Workshop on Data Mining and Decision Analytics (DMDA). Finalist for 2016 INFORMS Service Science Cluster Best Paper Award. To appear in European Journal of Operations Research (acceptance rate: ~19%, impact factor: 3.81). Patents pending. [ abstract ]
  In business environments where an organization offers contract-based periodic services to its clients, one crucial task is to predict changes in revenues generated through different clients or specific service offerings from one time epoch to another. This is commonly known as the revenue change prediction problem. In practical real-world environments, the importance of having adequate revenue change prediction capability primarily stems from scarcity of resources (in particular, sales team personnel or technical consultants) that are needed to respond to different revenue change scenarios including predicted revenue growth or shrinkage. It becomes important to make actionable decisions; that is, decisions related to prioritizing clients or service offerings to which these scarce resources are to be allocated. The contribution of the current work is twofold. First, we propose a framework for conducting revenue change prediction through casting it as a classification problem. Second, since datasets associated with revenue change prediction are typically imbalanced, we develop a new methodology for solving the classification problem such that we achieve maximum prediction precision while minimizing sacrifice in prediction accuracy. We validate our proposed framework through real-world datasets acquired from a major global provider of cloud computing services, and benchmark its performance against standard classifiers from previous works in the literature.
• Bayesian Optimization for Predicting Rare Internal Failures in Manufacturing Processes. Abhinav Maurya. IEEE International Conference on Big Data 2016 (acceptance rate: 18.7%). Winner of travel grant awarded by Robert Bosch GmbH. [ abstract | pdf ]
  Modern manufacturing processes are highly instrumented to capture data at every step along the assembly lines. Such fine-grained data can help manufacturers perform quality control by predicting which manufactured products are at risk of being defective. A predictive analytics system that identifies internal failures in products before they are shipped can result in significant savings for the manufacturer and result in better customer satisfaction. However, predicting internal failures in manufactured products is a challenging machine learning task due to two reasons: (i) the rarity of such failures in modern manufacturing processes, and (ii) the failure of traditional machine learning algorithms to optimize non-convex metrics such as Matthew's Correlation Coefficient (MCC) used to measure performance on imbalanced datasets. In this paper, we present “ImbalancedBayesOpt”, a meta-optimization algorithm that directly maximizes MCC by learning optimal weights for the positive and negative classes. We use Gradient Boosting Machine (GBM) as the base classifier for our meta-optimization algorithm due to its competitive performance on machine learning prediction tasks. Our quantitative evaluation suggests that “ImbalancedBayesOpt” can significantly improve the classification performance of base classifiers on severely imbalanced high-dimensional datasets.
• WirelessAcrossRoad: Low-cost Traffic Congestion Detection for the Developing World
• Road-RFSense: A Practical RF-Sensing Based Road Traffic Estimation System for Developing Regions. Rijurekha Sen, Abhinav Maurya, Bhaskaran Raman, Amarjeet Singh, Rupesh Mehta, Ramakrishnan Kalyanaraman. ACM Transactions on Sensor Networks, 2014. [ abstract | pdf ]
  An unprecedented rate of growth in the number of vehicles has resulted in acute road congestion problems worldwide, especially in many developing countries. In this article, we present Road-RFSense, a practical RF sensing--based road traffic estimation system for developing regions. Our first contribution is a new mechanism to sense road occupancy, based on variation in RF link characteristics, when line of sight between a transmitter-receiver pair is obstructed. We design algorithms to classify traffic states into two classes, free-flow versus congested, at timescales of 20 seconds with greater than 90% accuracy. We also present a traffic queue length measurement system, where a network of RF sensors can correlate the traffic state classification decisions of individual sensors and detect traffic queue length in real time. Deployment of our system on a Mumbai road gives correct estimates, validated against 9 hours of image-based ground truth. Our third contribution is a large-scale data-driven study, in collaboration with city traffic authorities, to answer questions regarding road-specific classification model training. Finally, we explore multilevel classification into seven different traffic states using a larger set of RF-based features and careful choice of classification algorithms.
• KyunQueue: A Sensor Network System To Monitor Road Traffic Queues. Rijurekha Sen, Abhinav Maurya, Bhaskaran Raman, Rupesh Mehta, Ramakrishnan Kalyanaraman, Nagamanoj Vankadhara, Swaroop Roy, Prashima Sharma. 10th ACM Conference on Embedded Networked Sensor Systems, 2012 (acceptance rate: 18.7%). [ abstract | pdf ]
  Unprecedented rate of growth in the number of vehicles has resulted in acute road congestion problems worldwide. Better traffic flow management, based on enhanced traffic monitoring, is being tried by city authorities. In many developing countries, the situation is worse because of greater skew in growth of traffic vs the road infrastructure. Further, the existing traffic monitoring techniques perform poorly in the chaotic non-lane based traffic here. In this paper, we present Kyun Queue, a sensor network system for real time traffic queue monitoring. Compared to existing systems, it has several advantages: it (a) works in chaotic traffic, (b) does not interrupt traffic flow during its installation and maintenance and (c) incurs low cost. Our contributions in this paper are four-fold. (1) We propose a new mechanism to sense road occupancy based on variation in RF link characteristics, when line of sight between a transmitter-receiver pair is obstructed. (2) We design algorithms to classify traffic states into congested or free-flowing at time scales of 20 seconds with above 90% accuracy. (3) We design and implement the embedded platforms needed to do the sensing, computation and communication to form a network of sensors. This network can correlate the traffic state classification decisions of individual sensors, to detect multiple levels of traffic congestion or traffic queue length on a given stretch of road, in real time. (4) Deployment of our system on a Mumbai road, after careful consideration of issues like localization and interference, gives correct estimates of traffic queue lengths, validated against 9 hours of image-based ground truth. Our system can provide input to several traffic management applications like traffic light control, incident detection, and congestion monitoring.