How approval-rating polls work — why ARS and live interviews give different numbers
An approval rating is not one number but a value each survey produces separately. The largest source of divergence is method: automated response system (ARS) polls have a machine ask and respondents press buttons, while live-interview polls have a person ask and respondents speak. That difference changes who cooperates, how candidly people state political leanings, and how the raw sample is weighted. Two pollsters surveying the same week can differ by several percentage points without either being wrong
The three lines
- Method — ARS uses a recorded voice and buttons; live interviews use a person and speech. This is the biggest single source of divergence
- Response — the two methods trade contact rate against cooperation rate. Comparing on response rate alone misleads
- Reading — an approval rating is this survey's value, not the country's. Check pollster, dates, method and margin of error
Key questions
- Why do different pollsters report different approval ratings?
- **Because methods differ, and different methods reach different people.** Four things vary. ① **Method** — ARS (recorded voice, button press) and live telephone interview (a person, spoken answers) produce different response patterns. ② **Sampling** — using carrier-supplied virtual numbers versus randomly generated numbers changes who gets reached. ③ **Question wording** — 'doing well / doing badly' and a four-point scale return different distributions from the same people. ④ **Weighting** — samples are adjusted to match sex, age and region, sometimes past vote as well, and the standards differ by house. Vary all four slightly and results diverge by several points. **None of that requires anyone to be wrong.**
- Which is more accurate, ARS or live interviews?
- **Neither can be called superior; each carries a different bias.** In an **ARS** poll there is no person to face, so respondents tend to state party or candidate preference more candidly. But finishing an automated call requires patience that correlates with **high political interest**, so those respondents are over-represented. In a **live interview**, an interviewer can draw out moderate and low-interest respondents, but people answering a human being are more likely to **deflect or conceal** a preference. In South Korea, ARS polls have tended to produce relatively more favourable results for the conservative party and live interviews the reverse — a pattern read through exactly this difference. **The reader's job is to check which method was used, not to pick a side.**
- Is a poll with a low response rate untrustworthy?
- **Response rate alone cannot answer that.** What looks like one number is the product of two: **contact rate** (share of eligible numbers dialled that reached someone) and **cooperation rate** (share of those reached who completed the survey). The two methods invert these. **Live interviews have higher cooperation and lower contact; ARS has higher contact and lower cooperation.** Multiply them the way the American Association for Public Opinion Research does and the gap narrows considerably. The real question is not whether the response rate is low, but **how different the respondents are from the electorate** — and closing that gap is what weighting does, using standards that differ by pollster.
On August 24, 2026, President Lee Jae-myung's approval rating was reported at 40.2 percent.
Some outlets the same day printed 40.6 percent.
One of those could be an error. But in polling, a difference like that is far more often produced by method than by mistake.
1. An approval rating is not one number
Start by correcting the sentence everyone uses.
"Lee Jae-myung's approval rating is 40.2 percent."
What is actually true:
"In a Realmeter survey commissioned by Energy Economic Daily, conducted August 18–21 among 2,026 South Korean voters aged 18 and over, 40.2 percent rated his performance positively."
An approval rating is not a fact about a country. It is a value a particular survey produced. Change the survey and the value changes. That is normal.
2. Four places the divergence comes from
| Element | What changes |
|---|---|
| Method | ARS or live interview — who stays on the line |
| Sampling | Carrier virtual numbers or randomly generated numbers — who gets reached |
| Question wording | Binary or four-point scale — the same person answers differently |
| Weighting | Sex, age, region, sometimes past vote — standards differ by house |
Vary all four slightly and results move by several points.
The first is the largest, and the most argued over.
3. ARS versus live interview
| ARS | Live interview | |
|---|---|---|
| Who asks | a recorded voice | a human interviewer |
| How you answer | press a button | speak |
| Duration | short | longer |
| Cost | low | high |
| Character | nobody to face | the interviewer can draw answers out |
The ARS bias
There is no person on the other end, so respondents tend to state party or candidate preference candidly. A choice that feels awkward to say aloud is easy to press.
The cost appears elsewhere. Sitting through a recorded questionnaire is tedious, and the people who do it anyway skew toward high political interest. The sample carries a surplus of engaged respondents.
The live-interview bias
An interviewer can reach moderates and the disengaged, and can clarify a question, which raises answer quality.
But the respondent is talking to a person. That produces deflection and concealment, and the effect grows in periods when stating a preference feels costly.
What has been observed in Korea
Korean ARS polls have tended to produce relatively more favourable results for the conservative party, and live interviews the reverse. One camp reads that as ARS capturing hidden conservative support; the other as ARS over-weighting the politically engaged.
This brief does not adjudicate. The reader's task is to check which method was used — not to decide which method to believe.
4. Response rate is two numbers, not one
"How can you trust a poll with a 3 percent response rate?" is half right.
Response rate is a product.
| Component | Meaning |
|---|---|
| Contact rate | Share of eligible numbers dialled that reached someone |
| Cooperation rate | Share of those reached who completed the survey |
And the two methods invert them.
| Contact rate | Cooperation rate | |
|---|---|---|
| Live interview (virtual numbers) | relatively low | high |
| ARS | high | low |
Multiplied together the way the American Association for Public Opinion Research defines it, the gap between methods narrows compared with looking at cooperation alone.
The more important point sits behind all of this. A low response rate is not itself the problem. The problem is how far respondents differ from the electorate.
A 5 percent response rate with a representative composition yields a usable number. A 20 percent rate skewed toward one leaning does not.
Closing that gap is what weighting does — and weighting standards differ by pollster.
5. Reading the margin of error
Every release carries a line like this:
"Margin of error ±2.2 percentage points at the 95 percent confidence level."
What it means:
- Repeat this survey 100 times by the same method, and 95 of those runs land within 2.2 points of the true value.
- So the range this survey asserts is 38.0 to 42.4 percent.
From which one working rule follows.
| Situation | How to read it |
|---|---|
| Difference inside the margin | You cannot say one leads. No ranking is available |
| Difference outside the margin | A difference can be asserted |
In the August 24 release, disapproval at 56.9 percent led approval at 40.2 by 16.7 points, and that lead had held outside the margin of error for four consecutive weeks. That is why this brief emphasised the streak rather than the level in "Lee Jae-myung approval falls to 40.2%" — a sustained position outside the margin says more than any single week's number.
One caution. The margin of error covers sampling variation only. Differences arising from method, wording and weighting are not in it — which is why gaps between pollsters are routinely larger than the stated margin.
6. Six things to check
| Item | Why |
|---|---|
| Pollster | Method and weighting standards differ |
| Client | Can shape survey design |
| Fieldwork dates | The dates people answered, not the release date |
| Method | ARS, live interview, or the mix ratio |
| Sample size and margin | A difference inside the margin is not a difference |
| What it is compared against | Trends require the same house and same method |
The third is ignored most often.
The survey released on August 24 was fielded August 18–21. Everything after that — the Canadian tariffs, Samsung's share collapse, the Iran sanctions campaign — is not in the number.
The sixth matters just as much. Comparing one pollster's new figure with another's old one produces noise, not a trend. Trends require a single house's own series.
7. In short
- An approval rating is a value a survey produced, not a fact about a country.
- Divergence comes from method, sampling, wording and weighting.
- Neither ARS nor live interviewing is superior. Each carries its own bias.
- Response rate is contact rate × cooperation rate. The real issue is how unrepresentative the respondents are.
- A difference inside the margin of error is not a difference — and the margin does not cover method effects.
- Trends require the same pollster and the same method.
8. What we could not confirm
- The partisan skew — a repeatedly observed tendency, not a universal rule.
- Contact and cooperation figures — taken from published explanations, not recalculated here.
- House weighting standards — disclosure varies, so no comparison was possible.
- Accuracy by method — no comparative study was available, so no judgement is offered.
Sources
- National Election Survey Deliberation Commission (Korea) — polling reference materials
- Seoul Shinmun — ARS favours the Democrats, live interviews the People Power Party: why?
- Munhwa Ilbo — ARS strength for the PPP: 'shy conservatives' or 'lower reliability'?
- MBC Poll M — Surveying the surveys
- MoneyToday — President Lee's approval 40.2%, a new low, sixth straight weekly decline (2026-08-24)
- Kyunghyang Shinmun — No convention bounce: President Lee's approval barely holds 40% (2026-08-24)