Contribution
A short policy-facing essay that gives “online manipulation” a definition and a harm. The definition makes covertness, not the technique, the mark of manipulation, and so separates manipulative nudges from informational ones and deception from manipulation in general. The harm is placed in autonomy, read in its competency and authenticity conditions, with the economic harm derivative of it. Four policy directions follow.
Key insight
Manipulation is influence the target does not and could not easily see. Persuasion and coercion address their targets openly; manipulation works only while it is hidden, and once an influence is known it becomes one more input to the target’s own deliberation. Digital technology suits it because it exposes vulnerabilities, adapts to them in real time, and itself recedes from attention through habit.
Assumptions
- Influence on individual decisions only (p. 3). Manipulation as steering instruments, or as “attempts at steering or controlling institutions and systems” (Russian trolls and the US media), is set aside. “It is manipulation in this sense—the attempt to influence individual decision-making and behaviour—that we focus on in what follows.”
- A broad, relational conception of autonomy (pp. 8–9). Deliberation includes “emotions, convictions, and experiences”, and “critical reflection can be counterfactual (we must in principle be able to critically reflect on and endorse our motivations for acting, but we need not actually reflect on each and every move we make)”. Persons are “socially, culturally, historically, and politically situated”.
- Liberal policy frame (p. 12). The proposals are deliberately “situated firmly in the terrain of contemporary liberal political discourse”.
Key results
- The definition (p. 4). “In our view, manipulation is hidden influence. Or more fully, manipulating someone means intentionally and covertly influencing their decision-making, by targeting and exploiting their decision-making vulnerabilities. Covertly influencing someone—imposing a hidden influence—means influencing them in a way they aren’t consciously aware of, and in a way they couldn’t easily become aware of were they to try and understand what was impacting their decision-making process.” It “hews especially close to the trickery view, but operationalises the notion of trickery more concretely”.
- Against persuasion and coercion (p. 4). Both “are both forthright forms of influence”. “Manipulation, by contrast, is hidden—we only learn that someone was trying to steer our decision-making after the fact, if we ever find out at all.” What marks it is that “when we learn we have been manipulated we feel played”; “at the time of decision we didn’t understand our own motivations.”
- Deception is a special case (p. 4). One way to covertly influence is to plant false beliefs; others play on desires and emotions or on cognitive biases (p. 5) “without the need for outright deception”.
- Overt temptation and guilting (p. 5). These are failed attempts at covert influence: “it is the attempted covertness that is central to manipulation, rather than the particular strategy”. Someone who tempts overtly is manipulative in the sense that they “frequently attempt to manipulate”, but “strictly speaking we would only say that they have succeeded in manipulating when their target is unaware of their machinations.”
- Nudges (pp. 5–6). Some are manipulative and some are not, “the distinction hinging on whether or not the nudge is hidden, and whether it exploits vulnerabilities or attempts to rectify them”. Nutrition labels are not; the cafeteria placement nudge “seems plausibly manipulative”. “just because it’s manipulative does not mean it isn’t justified”.
- Online manipulation (p. 6). “the use of information technology to
covertly influence another person’s decision-making, by targeting and
exploiting decision-making vulnerabilities.” Three reasons technology
suits it (pp. 6–7):
- surveillance puts vulnerabilities “on permanent display”, including “person-specific” ones (Cambridge Analytica’s psychometric targeting; a reported Facebook ability to detect insecure teenagers);
- platforms are “dynamic, interactive, intrusive, and adaptive choice architectures” (Yeung’s “highly personalised choice environment[s]”);
- habituated technology becomes “invisible”, so “the influences they facilitate are often hidden, and thus potentially manipulative”.
- Not every case manipulates (pp. 7–8). Dark patterns may “overtly inconvenienc[e] users, rather than hiding their intentions”; of Uber’s and Lyft’s prompts, “not all such practices necessarily manipulate—people are savvy”. Manipulation “is not a binary phenomenon” perhaps, and influences may be “more or less manipulative” (p. 10).
- The autonomy harm (pp. 9–10). To manipulate is “to intentionally alter their decision-making process without their conscious awareness”. It undermines autonomy in two ways: it “can lead them to act toward ends they haven’t chosen” and “for reasons not authentically their own”. It challenges both the competency condition and the authenticity condition.
- Welfare harm is derivative (p. 10). The economic harm “is derivative of the deeper harm to autonomy”.
- Beneficent manipulation still harms (pp. 10–11). “the fundamental harm of manipulation is to the process of decision-making, not its outcome”. The paternalist harms his target “by rendering him opaque to himself”; someone whose lifelong bad habit suddenly vanished “might also be deeply confused and suspicious about the source of the change”.
- Harm, not wrong (p. 11). “we have not demonstrated that manipulation is necessarily wrong in every case—only that it always carries a harm”. Cases where the harm is outweighed “are likely few and far between”.
- Collective harm (p. 11). “autonomy is writ small what democracy is writ large”.
- Policy (pp. 11–13). Curtail digital surveillance; problematise personalisation; promote awareness and understanding, noting that notice alone fails because neutralising manipulation needs knowing “who the manipulator is, what strategies they are deploying, and why”; and attend to context, since tolerance of influence differs between commerce and politics.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Manipulation is intentional, covert influence that exploits decision-making vulnerabilities | moderate | a stipulated definition, motivated by contrast with persuasion and coercion, p. 4 |
| C2 | Covertness, not technique, decides whether an influence manipulates; overt temptation is failed manipulation | weak | asserted, with the observation that such tactics work better concealed, p. 5 |
| C3 | Information technology is especially suited to manipulation | moderate | three mechanisms with reported examples, pp. 6–8; no measurement |
| C4 | Manipulation always carries a harm to autonomy, including when beneficent | moderate | argument from the process of decision-making and the vanished-habit case, pp. 9–11 |
| C5 | Manipulation is not wrong in every case | moderate | the life-in-danger case, p. 11 |
| C6 | Online manipulation threatens democracy | weak | the analogy of individual and collective self-government, p. 11 |
Concepts
- Manipulation. Intentionally and covertly influencing someone’s decision-making by targeting and exploiting their decision-making vulnerabilities.
- Covert (hidden) influence. Influence the target is not consciously aware of and could not easily become aware of on trying.
- Online manipulation. Manipulation by means of information technology, “online” read expansively; it “could just as easily be termed ‘digital manipulation’ or ‘automated manipulation’”.
- Decision-making vulnerability. Cognitive biases common to most people and “person-specific” fears, desires and circumstances.
- Invisibility. The phenomenologists’ “technological transparency”: a habituated device recedes from attention.
- Autonomy. After Raz, “people should make their own lives”, with competency and authenticity conditions after Christman.
Connections
The three-way division of existing accounts is taken from the 2018 version of Noggle’s SEP entry (LIT-1007), whose 2026 revision quotes this paper’s definition as “an especially influential version” of the covert-influence view. The definition follows Ware and Strudler in that family and claims kinship with the trickery view. The autonomy account draws on Raz, Christman, Mackenzie and Stoljar’s relational autonomy, and Scanlon. The technology argument draws on Calo’s “digital market manipulation”, Yeung’s “hypernudge” and Susser’s own “Invisible Influence” (2019). Coons and Weber’s volume (LIT-1003) supplies its Wood citation.
Bearing on the record
- QUESTION-070 quotes the definition correctly: the paper does require that the influence be intentional, and says so three times (pp. 4, 5, 9). But the paper never discusses the requirement. It treats firms (advertisers, Facebook, Uber, Lyft, “data collectors and their clients”) as the intending manipulators and never asks whether an adaptive system’s influence can be manipulation when no one chose it. Its own mechanisms, though, are of exactly that kind: interfaces “configured in real time” that “continue to learn about us”. So the paper supplies the definition QUESTION-070 tests, and the cases that strain it, without noticing the strain. On its definition the LIT-179 agent’s influence would be manipulation only if the intent of those who deployed it carries over to the exploit it found.
- The paper excludes manipulation of institutions and systems (p. 3), and it never considers manipulation by them. It is no evidence either way on whether an organized whole can manipulate.
- For QUESTION-071, the paper places manipulation’s harm on the authenticity line the record has set aside. It says manipulation leads people to act “for reasons not authentically their own” and challenges “our capacity to reflect on and endorse our reasons for acting as authentically on our own”. That is the authenticity condition, the family of CLAIM-016. The paper does not say how a manipulated agent’s present structure differs from an unmanipulated one, and its counterfactual-reflection condition is itself a present-structure condition of the kind THEORY-040 says a manipulator can install.
- But its criterion of manipulation is historical and epistemic, which bears on QUESTION-071’s second part. Whether an influence manipulated depends on whether the agent was, and could easily have become, aware of it. The harm is self-opacity: “at the time of decision we didn’t understand our own motivations”; the person whose habit vanished is “confused and suspicious about the source of the change”. That makes the mark of manipulation a fact about the agent’s access to the causal history of her state, not about the state. It is one answer to “what in its history would have to be recorded”: whether the influence was available to the agent’s reflection when it acted. The paper does not argue this for global manipulation. Its scope is single decisions.
- QUESTION-071 says the analyses filed with it say what makes an influence manipulation as against persuasion or coercion. For this paper that is true: openness is the line (p. 4). It is not an account of the four-case argument’s manipulation, which installs a whole character, and the paper says nothing on that.
- CLAIM-022 and THEORY-047. The vanished-habit case agrees with CLAIM-022 in kind: the agent is better off and her will works, and the paper still counts a harm in how the change came about. THEORY-047’s controlling conditions of acquisition are not discussed.
- THEORY-139. No bearing; nothing on group agency.
- No instruction for machine-learning practice. Its subject, the ethics and regulation of influence by technology, is not one the anthology’s topics hold.
Limitations
- An essay of fourteen pages, by its own account a shortened version of a longer paper. The definition is stipulated and motivated by contrast, not defended against counterexamples. The objection that nagging, peer pressure and guilt trips are manipulative though open is answered only by calling them attempted manipulation (p. 5).
- No empirical work. The technology examples are reported, and the paper concedes that some of them may not manipulate.
- The requirement of intent is stated, not examined. Non-human or distributed manipulators are not considered.
- Autonomy is assumed to be the right frame; the arguments of S. Buss and others that manipulation’s wrong has little to do with autonomy are not engaged.
Open questions
- Whether “intentionally” can be dropped or relocated while keeping the line between manipulation and harmless influence, given the adaptive systems the paper itself describes.
- How “couldn’t easily become aware of” is to be measured, and for whom. The paper’s own policy section says awareness that one is targeted is not enough.
- Whether the harm to autonomy is a harm to the agent’s present capacity or to her relation to the history of her choice. The vanished-habit case suggests the second.