Contribution
A review across six online domains (video gaming, gambling, cybersexual activities, shopping, social networking, on-demand streaming) of the features said to promote uncontrolled use. It then classifies them by the learning process they act on, drawing on model-free and model-based reinforcement learning, so that features in different domains can be compared by mechanism (Table 2, p. 8).
Key insight
A design feature can weaken control by working on cached values, which makes cues pull and habits form. Or it can work on the user’s own computation of costs and benefits, inflating the activity’s value, hiding its costs, cutting deliberation short, or never letting a goal be fully reached. Providers need no theory of either: “massive deployment allows learning by trial and error” (p. 6).
Assumptions
- Problematic use is defined clinically. Loss of control and continued use despite harm, after ICD-11’s gaming disorder: “impaired control; increasing priority given to gaming over other life interests and daily activities; and continuation or escalation of gaming despite the occurrence of negative consequences” (p. 2).
- Model-free and model-based control run together. They “are not mutually exclusive, but can operate in parallel” (p. 6), unlike automatic-against-willed dual-system models.
- Engagement is always chosen. “we posit that engaging in a potentially problematic online activity is always a choice” (p. 10). It is model-free or model-based, not compelled.
- Industry motives are taken as given. Such systems “clearly point to economic motives for exploiting these features” (p. 6). This is supported by citation, not by any analysis of firms.
Key results
By section. Nothing here is a new study.
-
Domain reviews (pp. 2–6).
- Gaming. Variable and fixed ratio schedules, avatars and owned virtual goods, loot boxes and “predatory” monetization (p. 3).
- Gambling. Time-limited inducements, constant availability of sports betting and live betting (p. 4).
- Cybersex. Algorithmic recommendation and personalized notifications (pp. 4–5).
- Shopping. Countdown timers “(which often provide false information)”, sell counts and scarcity alerts (p. 5).
- Social networking. Infinite scrolling, interest-matched feeds, likes, the “double tick”, ephemeral stories, push notifications (p. 5).
- Streaming. Autoplay, “generally reported as the biggest challenge to self-control in binge-watching”, together with smart download and skipping of recaps and credits (p. 5).
Gaming research is “limited by its reliance on cross-sectional data” (p. 4); for cybersex “empirical research is scarce and supporting evidence is mainly indirect” (p. 5).
-
Convergence (p. 6). Domains interpenetrate, with loot boxes in games and shopping in social feeds. Behavioural tracking is proposed as the common feature: it “allows service providers to maintain users within ‘online interactive loops’ to maximize data collection and refine these profiling analyses”.
-
Provider-side selection (p. 6). “Online product development requires little knowledge of underlying psychological processes because massive deployment allows learning by trial and error: design features that increase the frequency of online behaviour, time-on-device or spending (and therefore revenue) will be maintained, whereas those that do not will be abandoned”. Users and researchers, meanwhile, lack “an understanding of how it enables the manipulation of their behaviour in a progressively personalized manner”.
-
Model-free features (pp. 6–7). Habit and incentive sensitization are the model-free routes. “Incentive sensitization (intense cue-driven ‘wanting’) is hypothesized to be caused by the repeated overstimulation of mesolimbic and mesocortical dopaminergic pathways”, by drugs “as well as other reinforcers when their delivery is highly unpredictable” (p. 6). Random-ratio schedules make uncertainty irreducible and are found in gambling and, the authors say, in games and social media “where non-monetary rewards such as ‘likes’, ‘retweets’ or notifications are unpredictable” (p. 6). But “we are unaware of any formal analysis of design features related to reinforcement schedules in domains other than gambling” (p. 7). Personalized notifications and ads are “individualized or personalized cues that might generate even more craving” (p. 7).
-
Model-based features (pp. 7–10).
- Overvaluation. Immersion, social reward and fear of missing out (FOMO).
- Interference with deliberation. Autoplay, countdowns and cash-out offers “interfere with the deliberative and controlled processes needed to overcome present-oriented decision-making” (p. 8).
- Partial goal fulfilment. “Features that reflect moving goalposts are ubiquitous in video gaming, social networking and on-demand TV streaming, and they keep individuals engaged for longer than planned after initiating the activity” (p. 8), because “online environments often allow only partial satisfaction” (p. 8). Infinite scrolling “reveals some desired information, but also offers the never-ending promise of new rewards” (pp. 8–9).
- Cognitive biases. Stop buttons foster an illusion of control. “Losses disguised as wins” and near-misses “contribute to the false belief that one is progressing or learning something valuable about the game, or artificially reduce perceived losses” (p. 9).
-
The synthesis (p. 10). “manipulations that facilitate model-free control have the potential to turn an instrumental behaviour into an addictive one”. Equally, an activity can become problematic “simply because their consequences are computed to be more valuable than consequences of abstaining”, where “technology enables online activities to be adapted to match personal motivations, which might exploit vulnerabilities”.
-
Box 1 (p. 9). Prevention through self-control strategies, built on habits rather than resisting temptation. It also proposes regulation, including “legislation that prohibits web-based applications and interfaces from exploiting nudge techniques”.
-
A caveat against its own field (p. 10). Applying substance-use models to online behaviour “is highly reductionistic and tends to pathologize common behaviours”.
Claims
| id | claim | strength | support |
|---|---|---|---|
| C1 | Specific design features are associated with loss of control in each of six domains | weak to moderate by domain | cross-sectional and qualitative studies, reviewed; gambling best supported |
| C2 | Design features can be classified by whether they act on model-free or model-based learning and control | argued | theoretical framework (refs 48–49, 169–173); a taxonomy, not a finding |
| C3 | Unpredictable non-drug rewards can produce incentive sensitization | speculative for online media | cited reinforcement-schedule studies; no formal analysis outside gambling (p. 7) |
| C4 | Providers select features by their effect on use and revenue, without psychological theory | asserted | citations to industry-practice literature; no firm studied |
| C5 | Informational asymmetry lets providers personalize features to users’ vulnerabilities | asserted | argued |
| C6 | Moving goalposts and partial fulfilment prolong engagement beyond what was planned | weak | qualitative reports |
Concepts
- technology design features: structural characteristics of a site or application, as against general features of the internet such as accessibility, affordability and anonymity (the Triple A model).
- model-free / model-based control: action driven by values cached from past reinforcement, against action driven by a model of consequences and a comparison of expected utilities.
- random-ratio schedule: reward for each response with a probability that does not depend on history, so its uncertainty cannot be reduced by learning.
- partial goal fulfilment: goal attainment that reveals a further goal (levels, infinite scroll, cliffhangers, points).
- dark nudges: design that steers choice against the user’s interest.
- informational asymmetry: providers know users’ behaviour, and users and researchers do not know how that knowledge is used.
Connections
It rests on the incentive-sensitization theory (LIT-1461, THEORY-895), which it cites as one model-free route, and on the model-based and model-free framework of Daw, Niv and Dayan (LIT-580). Its provider-side selection is the user-facing half of what Doctorow calls twiddling (LIT-033, NOTE-041): back-end adjustment of “recommendation criteria” and ranking at low cost and high frequency. Doctorow argues from the firm’s incentives, these authors from users’ psychology, and neither studies a firm.
Bearing on the record
QUESTION-267: designed proxy-keyed potential in users. Yes, as description, and with a sharper map than the question has. Each of the paper’s model-based families acts on a different part of TERM-076’s regime, and the model-free family on its strength. The mapping is the record’s, not the paper’s.
-
Strength of a cue-keyed ranking (model-free). Random-ratio rewards and personalized triggers raise the pull of the service’s cues. Read with THEORY-895, this re-keys a ranking’s strength to cues, the personal case the question names. The paper itself flags this route as the least shown outside gambling (p. 7).
-
The ranks themselves (overvaluation). Immersion and social reward raise where the activity sits in the ordering.
-
The reference course (partial goal fulfilment). Moving goalposts move the reference course along the potential ordering. On the sign clause the user is kept just behind course, which keeps the ordering engaged without letting it settle at neutral. That fits CLAIM-440’s Carver-and-Scheier reading of sign as progress against a reference rate. It would also predict that these features yield mildly negative goal-conduciveness valence, not positive, during use. That is testable, and not tested here.
-
The system’s reading of its own progress (cognitive biases). Near- misses and losses disguised as wins “contribute to the false belief that one is progressing”. On CLAIM-439 this falsifies the system’s model of where it stands along the potential ordering, which is mis-felt valence, not unfelt. It is the user-side counterpart of CASE-151’s board receiving a model of the bank’s potential that misrepresented it.
-
The negotiation itself (interference with deliberation). Autoplay and countdowns do not outweigh the other perspectives. They pre-empt them: the next episode starts before any perspective that would stop it is consulted. Potential is then won by default, not in conflict. This bears on QUESTION-268. A conflict record will show no conflict where a feature has removed the occasion for one, so the measurement must count pre-empted decisions as well as decided ones.
The question’s criterion is still owed. The paper does not say when an ordering is “by their bearing on its maintenance” and when it is a proxy. Its clinical marks, “increasing priority” and “continuation or escalation … despite the occurrence of negative consequences”, are priority and persistence against harm, which is the question’s proxy-potential described from outside. They are not a criterion.
The organizational side: a firm’s metric installing potential in users. “design features that increase the frequency of online behaviour, time-on-device or spending (and therefore revenue) will be maintained, whereas those that do not will be abandoned” (p. 6). Read with the record’s account, this is two proxy-keyed potentials coupled. The firm’s potential ordering, ranked by engagement and revenue, a proxy for its viability, selects features. The selected features make a cue-keyed ranking potential in users, a proxy for their good.
- CLAIM-432. The coupling fits threat without remorse. The harm to users need not be represented anywhere in the firm, since selection by the metric needs “little knowledge of underlying psychological processes”. So the remorse that is functionally absent is a concern absent from the ordering, not one overridden. The paper’s call for regulation is the claim’s remedy, a redesign of the firm’s concerns from outside, though the paper does not put it that way.
- NOTE-041. Doctorow’s twiddling is the firm-side mechanism and this is its user-side effect. Doctorow’s third stage, in which value is drawn from users, would be the firm’s metric prevailing over the user-facing concerns that held it in the first stage. That connection is the record’s.
- CASE-151’s parallel is partial. There, signals of harm stayed inside the firm, held by people who could not act. Here the asymmetry is between the firm and its users and researchers, and the firm holds the signals.
A case worth filing, from another source. This paper documents no particular firm, so it cannot ground a documented CASE. A specimen of a platform whose own records show features kept for their effect on engagement against internal signals of harm to users would test the coupling directly. Public candidates are the internal research disclosed from Meta in 2021 and the complaints state attorneys general later filed against platforms over youth design features. Neither was read here, and both are allegations or leaks rather than findings, so the source and its standing would need to be settled first.
Not an instruction for machine-learning practice. Recommendation algorithms appear as features whose effects on users are reviewed, not as methods.
Limitations
- A narrative review. Most of the evidence is cross-sectional, qualitative or indirect, and the authors say so domain by domain.
- The taxonomy is theoretical. No feature is shown to act on the process it is filed under, and some features appear under both headings. Infinite scrolling, for example, is a reinforcement schedule for streaming but partial goal fulfilment for social networking (Table 2).
- The claim that unpredictable non-drug rewards produce sensitization rests on cited animal and laboratory work not analysed here, and the authors know of no formal analysis outside gambling.
- Provider motives are asserted from citations. No firm’s practice is examined.
Open questions
- Do moving-goalpost features produce negative or positive momentary affect during use, measured apart from choice? The record’s sign clause predicts negative.
- Can pre-empted decisions (autoplay) be distinguished from decisions won by the potential ranking in a conflict record, as QUESTION-268 would need?
- Is there a documented case of a firm keeping a feature for its engagement effect against its own evidence of harm?