Sentence transformers’ job is to convert pieces of text into embedded vectors, where similar (dissimilar) pieces of text have spatially close (far) vectors. This makes them a very computationally efficient tool for tasks such as text-clustering, semantic search, retrieval augmented generation (RAG) for LLMs, etc. Yet, they suffer from a fundamental limitation: If two statements are topically similar, but stance-wise opposite, sentence transformers will still convert them into spatially close vectors. For instance, the controversial statements: “𝐼 𝑙𝑜𝑣𝑒 𝑝𝑖𝑛𝑒𝑎𝑝𝑝𝑙𝑒 𝑜𝑛 𝑝𝑖𝑧𝑧𝑎” and “𝐼 ℎ𝑎𝑡𝑒 𝑝𝑖𝑛𝑒𝑎𝑝𝑝𝑙𝑒 𝑜𝑛 𝑝𝑖𝑧𝑧𝑎” would be understood as similar statements by the model. Why? Cuz they are both talking about putting pineapple on pizza; though from different standpoints.

In this work titled: “𝑰 𝒍𝒐𝒗𝒆 𝒑𝒊𝒏𝒆𝒂𝒑𝒑𝒍𝒆 𝒐𝒏 𝒑𝒊𝒛𝒛𝒂 != 𝑰 𝒉𝒂𝒕𝒆 𝒑𝒊𝒏𝒆𝒂𝒑𝒑𝒍𝒆 𝒐𝒏 𝒑𝒊𝒛𝒛𝒂: 𝐒𝐭𝐚𝐧𝐜𝐞-𝐀𝐰𝐚𝐫𝐞 𝐒𝐞𝐧𝐭𝐞𝐧𝐜𝐞 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 𝐟𝐨𝐫 𝐎𝐩𝐢𝐧𝐢𝐨𝐧 𝐌𝐢𝐧𝐢𝐧𝐠”, we try to solve this problem by finetuning a pretrained sentence transformer to differentiate between opposing statements on similar topics spatially. We then demonstrate how it can be applied to social network analysis to detect users’ stances on controversial topics quickly, using semantic querying.

About Vahid Ghafouri

Vahid is a Ph.D. student in Telematics at IMDEA Networks Institute (+UC3M). His main area of research involves the application of NLP on social network data for measuring online polarization and radicalization.

This event will be conducted in English