The Data Distortion Problem: When LLMs Turn Bad Information into Strategic Advice

PSA: Fake news and fabricated data feed LLMs and organically distribute false information over time.

CPI data not accounting for actual standard of living? All financial reports will be skewed its way.

Average mortgage or cost of living prices blanded into anything but true median? Expect investment math broken because of a false data set.

Any public reports for regional safety, political bias, or virtually anything mass media can fabricate, is accounted for in regurgitated data LLMs use to shape strategic advice.

I’ve had serious arguments over the past few months where LLMs stubbornly quoting public data you can’t argue with (the page/study/report exists) but does not reflect reality whatsoever.

A vocal minority can cause a ripple effect which an LLM couldn’t tell, similarly to black hat SEO polluting indices in page 1 with crappy results. But since old school SEO wasn’t shaping entire decks and complete strategies end to end, it was way easier to tell (and represented just one node off the full research report).

We live in interesting times.