Moral Mimicry
Large Language Models Produce Moral Rationalizations Tailored to Political Groups
University of California, Davis
Background
- Large language models (LLMs) can generate fluent text
- LLMs tend to reproduce social biases
- Moral discourse is important to the social environment
- Moral intuitions/preferences vary by social group
Do LLMs reproduce the moral biases associated with social groups?
Implications
Risks
- More effective disinformation
- Entrenchment of biases
- Polarization
Potential Benefits
- Moral reframing, inter-group dialogue and understanding
To avoid risks and steer towards benefit, we should understand whether LLMs reproduce moral biases.
An approach to study morality
Criteria
- Assume moral phenomena vary across social groups
- Quantify moral content in text
Solution
Moral Foundations Theory (MFT)
Moral Foundations Theory
Five Foundations
Care/ Harm
Fairness/ Cheating
Loyalty/ Betrayal
Authority/ Subversion
Sanctity/ Degradation
Five Foundations
Care/ Harm
Liberty/ Oppression
Fairness/ Cheating
Loyalty/ Betrayal
Authority/ Subversion
Sanctity/ Degradation
Five Foundations
Individualizing
Care/ Harm
Liberty/ Oppression
Fairness/ Cheating
Binding
Loyalty/ Betrayal
Authority/ Subversion
Sanctity/ Degradation
Relative importance varies between political groups
Moral Foundations differ between US Liberals and Conservatives
Moral Mimicry
General Idea: Large Language Models produce moral content that reflects the moral biases of social groups when conditioned with social group identity
This Research: LLMs produce moral content that reflects the moral biases of US liberals and conservatives when conditioned with liberal/conservative political identity
Research Questions
RQ1Can LLMs produce high-quality moral rationalizations?
RQ3Do LLMs use moral foundations differently when prompted with different political identities?
RQ6How does moral mimicry relate to model size?
Methods
Generating text from LLMs via prompting
- GPT-style models produce text via autoregressive decoding
- Based on the text I’ve already seen, what are the most likely words that follow
- Control the models by prompting
- The prompt provides some words for the model to start with
Prompting to produce moral rationalizations
Goal: Elicit moral “rationalization-like artifacts” from the model.
Prompt consists of three parts embedded in a template:
Scenario
Identity
Stance
Prompting to produce moral rationalizations
Template:
As a identity, here are the moral arguments for why scenario is stance:
Prompting to produce moral rationalizations
Prompt:
As a conservative, here are the moral arguments for why stealing from the store is immoral:
Completion:
- Stealing is a violation of the property rights of others.
- Stealing is a form of theft, which is a moral wrong.
- Stealing deprives others of their rightful possessions.
- Stealing is an act of aggression and violence.
Datasets
Models
GPT-3
Commercial model family from OpenAI
OPT [7]
Publicly-available model family from Meta AI
- Transformer-based GPT-style pretrained LLMs
- Several sizes (# of parameters) for each model
- Larger model \(\rightarrow\) more capability (usually)
- Pretrained on large corpora of web text
Measuring Moral Foundational Content in Text
Moral Foundations Dictionaries
Recap of the Methods
- Prompt the LM with scenario + identity + stance
- Measure foundation content in the output using Moral Foundations Dictionaries
- Calculate difference in foundation use for liberal vs. conservative identity
Research Questions
RQ1Can LLMs produce rationalization-like artifacts of sufficient quality for us to care?
RQ3Do LLMs use moral foundations differently when prompted with different political identities? (moral mimicry)
RQ6How is the moral mimicry capability affected by model size?
Results
RQ1: Can LLMs produce rationalization-like artifacts of sufficient quality for us to care?
Methods
Hand-label 100 examples of completions from GPT-3-175B for three text qualities:
Relevance
Coherence
Stance Agreement
Methods
Separate results by normative and non-normative stance
- Normative stance: prompt stance agrees with commonsense morality
- Non-normative stance: prompt stance disagrees with commonsense morality
Results
- Most results are relevant, coherent, and agree with stance
- Quality slightly better when GPT-3 is prompted with a normative stance
RQ3: Are LLMs moral mimics?
Results
- Care/Harm increased for liberal prompt
- Authority/Subversion increased for conservative prompt
- Modest effect sizes
RQ6: Moral Mimicry increases with model scale
OPT
- Sanctity/Degradation \(\leftrightarrow\) conservative identity
- Care/Harm, Fairness/Cheating \(\leftrightarrow\) liberal identity
RQ6: Moral Mimicry increases with model scale
GPT
- Authority/Subversion \(\leftrightarrow\) conservative identity
- Care/Harm, Fairness/Cheating \(\leftrightarrow\) liberal identity
