Using NLP to Detect Tradeoffs in Employee Reviews
Résumé
This paper presents a methodology to identify industry-specific tradeoffs using natural language processing (NLP) to analyze employee reviews from Glassdoor. The analysis is based on 400,000+ reviews from employees working in the financial sector between 2008 and 2020. For each review, the pros and cons sections are classified in a one-to-many approach in terms of the most prevalent topics in the sector. The most prevalent noun chunks within a representative sample of reviews are used as topics. The classification of reviews is based on the cosine similarity between the sentence embeddings of these topics and the sentence embeddings of the comment sections using a sentence-transformer model. Based on this classification, the count of pro-con topic pairs is tested for statistical significance against a control of randomly generated pairings from the same sample of classifications. The process is repeated 10,000 times and only pairs with a p-value < .05 (after Bonferroni-Holm correction) are considered. If both, a pro-con pair of topics and its opposite pairing are significantly frequent, this combination of topics qualifies as a tradeoff. Depending on whether the same pairing constitutes a significantly frequent pro-pro pair, it can be determined whether a solution for the tradeoff exists in the sector (at least within the limitations of this approach). Using 13 topic labels, 2 tradeoffs – work-life balance vs. opportunity (without solution in the sector) and work-life balance vs. management (with solution in the sector) – have been identified. In addition to that, 8 more pro-con pairs were identified as significant.