Machine Intelligence to Detect, Characterise, and Defend against Influence Operations in the Information Environment
Journal Article
·
· Journal of Information Warfare
OSTI ID:1777157
- BATTELLE (PACIFIC NW LAB)
Social media has enabled a new era of manipulation in the information and cognitive domains. Deceptive content—misleading, falsified, and fabricated—is routinely created and spread in the modern social media environment with the intent to create confusion and widen political and social divides, and exploit the societal conflict exacerbated by these divides in the real-world (aka physical domain). Such disinformation campaigns demonstrate a threat to the integrity of economic, political, cultural, public health, and national security institutions around the world. In this work we overview our artificial intelligence (AI) capabilities to detect, describe, and defend against information operations on Twitter as an example social platform to understand the influence of misleading and falsified content diffusion and better enable those charged with defending against such manipulation to enable responsive parties to counter it. We first present novel linguistically-informed deep learning (DL) models for misinformation and disinformation detection, and present an in-depth linguistic analysis of psycho-linguistic markers across broad deception categories. We then demonstrate how our models perform in the multilingual and multimodal setting and categorize falsified and misleading content based on the intent to deceive. We also provide a large-scale analysis to describe user behavior and spread patterns while engaging with deceptive content and report novel findings about the immediate diffusion of deceptive content by characterizing the vulnerable sub-populations and their demographics, and explicitly measuring speed and scale of deception spread to uncover who shares deceptive content, how quickly, how much, and how evenly. In addition, we measure audience reactions to misinformation and disinformation at scale, distinguishing the reactions of users identified as bots versus humans. Finally, we take advantage of deep translation and generation models to create unique solutions for real-time defense against digital deception and discuss how to apply causal inference to prescribe and intervene into strategic communications jointly across information, cognitive, and physical domains.
- Research Organization:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1777157
- Report Number(s):
- PNNL-SA-154713
- Journal Information:
- Journal of Information Warfare, Journal Name: Journal of Information Warfare Journal Issue: 2 Vol. 20
- Country of Publication:
- United States
- Language:
- English
Similar Records
Misleading or Falsification? Inferring Deceptive Strategies and Types in Online News and Social Media
Evaluating Deception Detection Model Robustness To Linguistic Variation
Understanding and Explicitly Measuring Linguistic and Stylistic Properties of Deception via Generation and Translation
Conference
·
Fri Apr 27 00:00:00 EDT 2018
·
OSTI ID:1435892
Evaluating Deception Detection Model Robustness To Linguistic Variation
Conference
·
Thu Jun 10 00:00:00 EDT 2021
·
OSTI ID:1894779
Understanding and Explicitly Measuring Linguistic and Stylistic Properties of Deception via Generation and Translation
Conference
·
Tue Dec 22 23:00:00 EST 2020
·
OSTI ID:1866984