Calling Bullshit
The Art of Skepticism in a Data-Driven World
Carl T. Bergstrom & Jevin D. West . 2020 . BeavLib 149.73 BER
References but no traceable footnotes or citations
- p029 Misinformation: false claims, not intentionally deceptive
- p029 Disinformation: deliberately spread falsehoods
p030 WhatsApp
p031 2017 Facebook admits 126M US users exposed to Russian propaganda
- p032 Firehose strategy disorients audiences. Eschew consistency, exhaust critical thinking
- p032 Purpose: advertising revenue
- p032 Teens in Macedonia earning $5000/mo, "Pope Francis Shocks the World, Endorses Trump for President"
- p034 mid-2017 FCC net neutrality, 21.7M citizen comments, most fraudulent, 500K similar comments sent at the same exact second 2017 July 19 2:57:15 pm EDT, 500K from Russia
- p038 Superfluous details make lies more persuasive
- p039 With nothing to say, bullshit fills space with inconsequentials
- p040 persuasive (exaggerated authority) or evasive (deflect uncomfortable questions) bullshit
- p040 jargon to exclude outsiders
p041 Bruno Latour Pandora's Hope: Essays on the Reality of Science 1999
- p041 When mechanisms (machines, theories) are well established, we should still examine inputs
p042 example black box: ANCOVA Analysis of covariance
- p043 Bullshit: biased input or obvious problems with output
- p047 detecting felons from photos: training photos of felons are scowling, nonfelon photos are attractive
- p064 Pitchers versus glasses - pitcher don't make us drink more, we order pitchers when we intend to drink more
p087 teen drivers in accidents are ×1.44 more likely to die with teen passenger, ×0.36 less likely with >35 yo passenger
p094 When a measure becomes a target, it ceases to be a good measure, because people game the score
- p095 Schools evaluated by student test scores teach "to the test" and memorization, not thinking
- p101 "50% of scientific papers never read" false, made-up statistic; actually 50% uncited after 4 years, but perhaps read many times
p102 Eugene Garfield founder of Science Citation Index couldn't correct this falsehood
p106 Selection bias; results influenced by act of sampling
p120 Berkson's paradox i.e. attractiveness and niceness; people rarely choose neither, so the remainder chosen correlate negatively
- p125 hence Yeats: "The best lack all conviction, while the worst are full of passionate intensity
- the worst who lack conviction live in Mom's basement, where they are not observed
p129 right censoring
- i.e. rap performers appear to die young because most rappers are young
- "but some of the pattern is real, trust me"
- p131 workplace wellness programs: those who opt in start out healthier
- p131 randomized control trials: wellness programs have no effect on fitness activities, employee retention, or medical costs
p135 Graphcrud: 2005 Florida "Stand Your Ground" law increased murders from 520 to 820 in 2007, but INVERTED AXIS Florida graph fools the eye and hides the long term lower murder trend that was REVERSED by the law
- p139 multiple variable relationships properly shown 0.5% in NYT, none in Washington Post or Wall Street Journal
p144 Glass slipper fits Cinderella; in the original Grimm version, the evil stepsisters amputate toes and heels trying to fit
p146 The Periodic Table of Periodic Tables by Daniel Donahoo in 2010 Wired
p148 2012 Andy Proel A Subway Map of Maps that Use Subway Maps as Metaphor
- p153 Silly unicorn drawing with business analytics buzzwords pointing at body parts
- p154 Silly bicycle drawing annotated with education buzzwords
- p155 bar graphs showing 40% to 22% differences but with the lower axis cut off just below Quebec numbers, making them look MUCH smaller
- p156 Improperly scaled horizontal bar graph showing MUCH longer bars for White and Asian American
- p159 Autism and MMR coverage versus birth year - autism range 0.1% to 0.6%, MMR coverage from 87% to 93% superimposed
- large autism prevalence after 2000 unlikely due to tiny MMR change
- p160 Thyroid cancer and Roundup glycophosphate increase simultaneously, but so does cell phone usage
- p170 Tennessee nonfarm labor increases 8%, bars increase 267%
- to be fair, they are presenting a steady increase in an observable way
- p170 Bar graph of book title sales - misleading because title label is shown as lower extension of bar
- p173 Fatal car crashes by age group misleading, page 174 car crashes per 100M miles by age group, young and old worst
- p178 Perspective pie chart shows largest group in back, seemingly smaller
p184 Did AI Chatbots develop their own language? Snopes: No
- p185 Post office machine learning to sort 0.5 billion mail pieces per day. machines manage 98% of handwritten addresses, 2% go to humans in huge Salt Lake City postal complex, some employees handle 1800 addresses per hour
p195 We prove we are human with visual CAPTCHAs, we are terrible with probabilities
- Completely Automated Public Turing Test to tell Computers and Humans Apart
p199 Carlos Guestrin machine learning; husky versus wolf image, software "learned" to recognize snow background of husky images
- p200 neural networks "detected pneumonia" in patient X-ray images from "PORTABLE" scanner because pneumonia patients were imaged in ER, not the radiology department
p201 biased data -> "machine indoctrination", calls for algorithmic transparency
- p202 Amazon hiring algorithm training process gender-biased
p204 adding variables curse of dimensionality
- p206 Science haphazard??
- p207 Science works well because it is self-correcting
- p210 epigenetics
p211 Diederik Stapel Faking Science: A True Story of Academic Fraud
- p215 Identifying criminals using huge databases creates too many "low probability" errors
one in 10 million error |
||
|
Match |
No Match |
Guilty |
1 |
0 |
Innocent |
5 |
50,000,000 |
p216p-values
p217 p-hacking modifying criteria until results seem better
- p219 p-value probability of opposite alternative being true;
- seems like those who can't imagine MANY alternatives can "achieve" lower p-values
p226 base rate fallacy ignoring general prevalence
p226 John Ioannidis Why Most Published Research Findings Are False
- p228 Clinical trials - publications strongly biased against reporting negative results, some negative results recast as positive results, the FDA view of trials is more pessimistic. Trials are expensive, the incentives are to publish
- p229 diagram showing cherry-picked journal view with far fewer negative results. FDA view, antidepressants especially paroxetine/Paxil looks bad
- p230 The smoking gun is there for everyone to see; coverups provide false "alternative" explanations
p230 Estelle Dumas-Mallet Poor replication validity of biomedical association studies reported by newspapers
- p231 Single study results often actively misunderstood by popular science writing
- p232 "Cafeteria Science" cherry-picked studies that tell a simple story without uncomfortable uncertainty.
p234 2015 astronaut Scott Kelly 3 months at ISS, "7% change of gene expression into proteins", press misintepreted as changed genome
p235 (some) open access journals have low standards ... details/examples dammit!
p235 predatory publishers
- p236 legitimate publication attracts solicitation emails from predatory publishers
p237 Scientific editor John McCool fakes a case report aboutSeinfeld fake "uromysitisis" episode, accepted for publication in predatory Urology & Nephrology Open Access Journal for $799
- p237 any scientific paper can be wrong, subject to reexamination andfuture evidence
- p239 be wary of extraordinary claims appearing in lower-tier venues - why not in higher tier?
- answer: Because opportunities are finite. How do we grow the higher tier?
p239 2012 Mehmet Oz green coffee extract for weight loss Dove Press
p240 retracted because data could not be validated
- p250 NBC tweet "foreign students aren't applying to American colleges"; actually, foreign applications are down at 39% of schools, and up at 35% of schools, chance fluctuations
- p251 National Geographic Society: 9 billion tons of plastic waste in oceans every year
- p252 only 8 billion tons of plastic produced throughout history, actual number 9 MILLION tons
p253 Fermi estimation: Lawrence Weinstein Guesstimation Multco Central 519.544 W424g 2008
- p254 entire cliffs 10km x 1km x 100m = 1e9 m³, oceans 3.6e14 m², rise = 1e9/3.6e14 m or 3 micrometers
- p255 actual erosion rate 1cm/year, 10km x 1km x 0.01m/year = 1e5 m³, rise = 0.3 nanometers
- p255 Fox News outraged that food stamp fraud in 2016 was $70M. 15% 45 million on food stamps, US pays $30B, 0.2% lost to fraud
- p256 actually, more than $900 million for just retailer cash fraud, within normal range for retail
p257 sociologist Neil Postman Bullshit and the art of crap-detection "At any given time, the chief source of bullshit with which you have to contend is yourself"
p266 Walter Lippman 1920 "There can be no liberty for a community which lacks the means by which to detect lies.”
- p266 Carelessly calling bullshit makes enemies
- p267 Reducto ad absurdum 2004 Summer Olympics
- 10.93 seconds for Women's 100 yard dash, more than 2 seconds faster than 70 years earlier
- book's point is that linear extrapolation predicts women outsprinting men in 2156 Olympics, overly simplistic
- p268 more interesting ... in about 2636, a time less than 0 seconds
p270 fMRI images of dead salmon "responding" to questions ... "there is something a bit off with regard to our uncorrected statistical approach"
p272 claims about necessary immune systems of long lived organisms refuted by trees
- p273 Wiles proof of Fermat's conjecture very difficult, many major advances
p273 Leonard Euler generalization of Fermat that an + bn + cn + dn ... = z^n requires n terms in sum ... disproved in 1966 with 27⁵ + 84⁵ + 110⁵ + 133⁵ = 144⁵
- counterexamples can be much easier to verify than positive proofs
p274 $74M Mercer Corridor Project improved travel times only 2 seconds ... but handles 30,000 more cars per daty
- p274 compared to $78M contract for Seattle Mariner's pitcher Felix Hernandez (didn't help Mariner batting prowess because pitchers don't bat)
- p276 Apple CEO Tim Cook graph crud - cumulative sales continued to rise before presentation, but quarterly sales declined
- p277 world record pace drops with age; maybe senescence, but also (classroom student points out) smaller sample size, fewer elderly runners
- p280 Scoring rhetorical points on tangential technicalities doesn't convince anyone, it just pisses people off.
- p280 debunk in private, one on one, nobody likes to be called out in public
- p281 people's belief in misinformation can be strengthened if you reiterate the myth before you debunk it.
- p281 Make sure of your facts, if you call bullshit, be correct.
- p282 Twitter is like yelling at TV set, hoping the TV people can hear you.
- p282 Wakefield MMR quoting sincere, don't argue bad intentions
- p283 Admit fault when wrong, credibility worth more than the outcome of one argument
- p283 Pick fewer battles, do homework in advance, and express yourself clearly
- p285 Don't call bullshit to signal intelligence, get a MENSA card instead
- p285 A "well actually" guy has more in common with a bullshitter, not a caller of bullshit. IS IT HELPFUL?
p286 Effective bullshit calling makes others smarter.