In November 2019, Alexey Guzey published an article titled ‘Matthew Walker's "Why We Sleep" Is Riddled with Scientific and Factual Errors’.¹ In contains extensive criticisms of Walker’s book, including an accusation of “deliberate data manipulation” (emphasis removed). Walker is a professor of neuroscience and psychology at UC Berkeley.

I have found many misquotes in Guzey’s article, which I will analyze and explain with the help of my tool Quote Checker. I believe Guzey can improve his article substantially if he fixes those misquotes. I welcome Guzey’s eye for detail and hope he extends it to quoting.

In addition, I analyze some of the changes Walker has made to his book since the publication of Guzey’s criticisms.

Note: I have not read Why We Sleep, nor do I plan to. I’ve only skimmed some of the passages Guzey criticizes.

Misquotes

Guzey quotes Walker:

On page 4, Walker writes:

the shorter your sleep, the shorter your life span

As readers, we are left wondering: did Walker start a sentence in lowercase? Did he neglect to end his sentence with proper punctuation? Or is Guzey just quoting without indicating an omission? There’s ambiguity here, and given Guzey’s allegations, some readers might not put such glaring mistakes past Walker. In reality, this quote is a misquote because it omits a part and punctuation without indication. One could validly work something like that into an inline quote, but not a block quote. The full quote reads:

Add the above health consequences up, and a proven link becomes easier to accept: the shorter your sleep, the shorter your life span.

Here’s a detailed diff:

In fairness to Guzey, he does provide the full quote at the end of his article, as he points out in footnote 1, where he addresses the possibility of missing context. But missing context isn’t my criticism here. His shorter quote is a misquote in form, even if it’s not one in content. His disclaimer re missing context would still apply regardless.

To fix the misquote, Guzey could write the following: ‘[T]he shorter your sleep, the shorter your life span.’ He knows that one can indicate a preceding omission by turning a lowercase letter into a capital one and adding brackets around it. He does it himself in several instances, eg when he accurately quotes the Encyclopedia of Sleep:

In another quote, Guzey removes italic emphasis without indication:

Guzey is inconsistent about indicating changes to emphasis: in a prior (and correct) quote, he indicates that bold emphasis is his in all quotes.

Next, Guzey quotes Walker (this time keeping italics):

Approximately 30 to 40 percent of these patients will feel better after a night without sleep … the 60 to 70 percent of patients who do not respond to the sleep deprivation will actually feel worse, deepening their depression. As a result, sleep deprivation is not a realistic or comprehensive therapy option.

The corresponding passage in Why We Sleep actually reads:

Approximately 30 to 40 percent of these patients will feel better after a night without sleep. Their lack of slumber appears to be an antidepressant.
The reason sleep deprivation is not a commonly used treatment, however, is twofold. First, as soon as these individuals do sleep, the antidepressant benefit goes away. Second, the 60 to 70 percent of patients who do not respond to the sleep deprivation will actually feel worse, deepening their depression. As a result, sleep deprivation is not a realistic or comprehensive therapy option.

Note the new paragraph. One cannot validly use an ellipsis to merge paragraphs:

I propose this fix:

Skipping some, Guzey quotes a paper by Ulrich-Michael Hemmeter et al titled ‘Sleep deprivation in depression’, published in the Expert Review of Neurotherapeutics, 10(7), 1101–1115 (2010). Guzey presents one big block quote from that paper (formatting removed):

The observation that after the recovery night a great majority of SD responders relapse into depression suggests that sleep per se may have a depressiogenic property.

The rapid effect of SD on depressive mood within hours is a fascinating experience for the patient, who may have been depressed for weeks or months …

SD is the only established antidepressant therapy that acts within hours, and therefore, can be applied in patients with treatment-resistant depression with a chance of approximately 50% of seeing an immediate, although temporary, relief from depressive symptoms without major side effects. … The experience of realizing that depression can be lifted and sleep can improve is very important for the further therapy motivation of treatment resistant depressed patients. … [Sleep deprivation] can be combined with antidepressant medication, predominantly serotonergic agents, with bright light therapy and with a phase advance of sleep cycles. All these strategies have been able to provide a chance to stabilize the SD response, at least in a subgroup of patients. [correction: John E. Richters points out that, in 2019, there’s also ketamine (a) which is approved by the FDA for treatment-resistent depression and acts within hours (a).]

But it’s not one quote. It’s actually three separate quotes. In the first instance, there’s not even an ellipsis to indicate the (massive) omission. It’s so large (seven pages) that I can’t paste it all into Quote Checker because it would go beyond fair use. So I’m breaking the big block quote up into three smaller diffs. I checked the source code of Guzey’s page and it actually has three separate blockquote elements, but they’re visually rendered as one, with the vertical bar having no interruptions or any other visual indications of separation, so his readers will mistakenly conclude that it’s a single quote.

Here’s the first part:

Seven pages later, the quote continues:

This part is fine, and the ellipsis at the end could have properly segued into the next paragraph, if only Guzey had indicated the omission at the beginning:

I’m leaving out the part at the end where Guzey includes a bracketed correction about there being an additional treatment option. I’ve also removed bold emphasis in all of Guzey’s quotes since he’s indicated his addition thereof, so it’s better to leave it out for accurate diffing.

Next, Guzey quotes Walker as stating that “sleep deprivation is not a realistic or comprehensive therapy option”, which is a correct quote. He then quotes the book Sleep, Neuronal Plasticity and Brain Function. Once again, he strings together block quotes that should be separated since they are separated by 14 pages in the book. The first part is (technically) correct:

I say “technically” because Guzey could simply remove the ellipsis and then display the second quote separately. Or he could technically post the ellipsis on its own line, but given how much content it replaces, that could be misleading. The second quote looks correct, but my visibility into that one is limited.

Next, Guzey again omits punctuation without indication, which is now a pattern:

This may not seem like a big deal but the reader is left guessing whether Walker omitted it, whether Guzey neglected to indicate an omission and the sentence actually continues, etc. A proper block quote either ends in the same punctuation as the original or it indicates the change. There’s no reason to make a change here, so I’d just include the period.

Guzey also misquotes Walker’s corresponding citation:

He does, however, quote the corresponding passage from Encyclopedia of Sleep correctly. He also properly quotes Walker re lack of sleep and its (in)ability to kill people. This is a nice example of a proper quote because it combines a couple of different legitimate changes:

Guzey’s subsequent quote of the Encyclopedia of Sleep is also correct. But then he misquotes a paper by, again, omitting punctuation at the end:

The next quote of Walker is correct, but Guzey misquotes the corresponding endnote (not footnote, as he writes) by omitting emphasis in the title Sleepless in America and turning a URL into a link:

Guzey’s misquotes aren’t limited to text. He misquotes a video, too, where he omits two seconds without indication even though they precede, and are part of, the quoted sentence, and even though he has demonstrated knowledge of how to indicate such omissions:

I’m only about a quarter into Guzey’s article but I’ve already found so many misquotes that I’m going to stop here. Given the error density, he should double check all of his quotes. These are mostly small mistakes, but they add up, and yes, they matter. Guzey points out single-letter mistakes in Walker’s citations, so it’s not like I’m applying a different standard than Guzey’s anyway.

Miscellaneous errors

Guzey claims Walker’s book “was published in September 2017.” It depends. The Wikipedia article Guzey links to (his archive here) and both my editions (see below) say the book was published in October 2017, not September. The UK edition was published in September, though. Guzey should specify which one he used (see below).

Guzey accuses Walker of a miscitation. This is ironic not only given the above misquotes and given the fact that Guzey never provides a full citation of Walker’s book, but also because Guzey miscites the same source he accuses Walker of having miscited. It’s called Encyclopedia of Sleep but, in footnote 5, Guzey cites it as “Encyclopedia of sleep” (with a lowercase ‘s’). (Note: the footnotes disappear on smaller screens; you have to click on the footnote markers to show the corresponding notes inline. That’s not user friendly and I only found a way because I tried harder than most people would, plus I’m a software engineer so I know things to try that most people don’t. Maybe Guzey could either render the footnotes at the bottom of the page or use a pointer for a cursor when hovering over a footnote marker to indicate that it can be clicked.)

Walker’s changes

As I’ve said, Guzey doesn’t give a full citation of Walker’s book. (I checked all 33 instances of the string “Why We Sleep” in Guzey’s article.) Normally, my policy in this case is to pick whatever edition I like and assume it’s the one the other guy used, too. That can go poorly for them but it’s their responsibility to cite properly. As I was checking Guzey’s first quote, about the book being a “scientifically accurate intervention”, I initially thought it was a misquote since my Kindle edition doesn’t contain those words. But then I saw that others (eg this page) have quoted it the same way Guzey did.

So I asked Guzey which edition he used and he replied (in full): “first edition”. My Kindle edition says it’s the first one. The full citation is: Walker, Matthew. Why We Sleep: Unlocking the Power of Sleep and Dreams. Scribner 2017: New York. Kindle Edition. The copyright page says it’s an ebook version of the “[f]irst Scribner hardcover edition [from] October 2017”. Guzey’s Twitter says he lives in New York City, so I’m guessing he also used a US edition (as opposed to the UK one, published by Allen Lane/Penguin Books). You can see a list of all known editions here – the paperback came out a little later than the hardback, as usual.

Due to the in-text discrepancies, I bought the same ebook again on Scribd (not to be confused with Scribner). I see that Walker did respond, on December 19, 2019, to “questions from readers” and error reports. He linked to Guzey’s article. I’ve only skimmed Walker’s response but I do see that he speaks of “corrections” which he “published in a second edition, which is now in print.” Walker deserves credit for making changes. However, there are issues with those changes (see below). In addition, the copyright page of my Scribd edition has the exact same information as the Kindle edition, down to the ISBNs for the ebook and hardcover (the omissions are the same, too, aside from an additional line break):

Both claim to be the first edition, but again, the contents are different. That’s presumably due to the changes Walker speaks of, but then the edition information, ISBN etc should be updated accordingly.

Since it seems that comparisons with the Scribd edition are more charitable for Guzey, I used that edition when checking for misquotes. The full citation is: Walker, Matthew. Why We Sleep: Unlocking the Power of Sleep and Dreams. Scribner 2017: New York. Scribd Edition. https://www.scribd.com/document/533891037/Why-We-Sleep-Unlocking-the-Power-of-Sleep-and-Dreams-PDFDrive-2

In any case, I’ve indicated below some of the changes I found of relevance to Guzey’s article, going from the Scribd edition to the Kindle edition. Guzey wonders whether Walker was responding to him specifically – given that these changes correspond directly to Guzey’s criticisms, I think that’s the case. These changes are fairly easy to find – all I did was go over Guzey’s article and paste quotes into word searches for both the Scribd edition and the Kindle edition. If the former returned results where the latter didn’t, the latter had been changed. The diffs below aren’t technically about quotes but simply compare text.

A significant part of Guzey’s criticisms hinges on Walker’s ambition to make his book “scientifically accurate” – Guzey quotes that part four times and calls it into question, eg alleging Walker’s manipulating of data to fit a narrative.

In addition, Guzey convincingly refutes Walker’s notion that “[r]outinely sleeping less than six or seven hours a night demolishes your immune system, more than doubling your risk of cancer.” In the Kindle edition, this part is pretty different:

Walker seems to have changed things in his favor: now it’s only six hours, not seven. Instead of shorter sleep “demolish[ing]” one’s immune system, it just “weakens” it; instead of “more than doubling your risk of cancer” – without clarifying what types of cancer – it merely “substantially increas[es] your risk of certain forms of cancer”, which is more vague and thus harder to criticize. My Kindle app tells me the updated passage is among the most highlighted in the book, with 6,795 readers having highlighted it.

Next, Guzey criticizes Walker’s claim that “the results of thousands of studies insist that no, there aren’t” “any biological functions that do not benefit by a good night’s sleep.” Guzey cites “an enormous literature” showing that “people with depression frequently benefit by not getting a good night’s sleep.” (Bold emphasis removed.) Maybe lack of depression isn’t itself a biological function. However, in the Kindle edition, the part about “thousands of studies” is missing:

Guzey points out that Walker contradicts himself when he claims that “[a]pproximately 30 to 40 percent of [depressed] patients will feel better after a night without sleep.” (Brackets mine.) In addition, Guzey cites a meta analysis with a result between 45% and 50%. Walker made the corresponding changes in the Kindle edition:

In the next paragraph, Walker writes, as quoted by Guzey, that “the 60 to 70 percent of patients who do not respond to the sleep deprivation will actually feel worse, deepening their depression.” Guzey points out that it’s actually less than 10%. Once again, we find corresponding changes in the Kindle edition:

Guzey also criticizes Walker’s conclusion that “sleep deprivation is not a realistic or comprehensive therapy option.” As you can see above, Walker has relaxed this conclusion by adding the word “alone”.

Originally, Walker claimed that “every species studied to date sleeps.” Guzey points out that Walker’s own source for this claim, the Encyclopedia of Sleep, says the opposite: some species do not sleep at all. Once again, we observe corresponding changes:

Walker’s introduction of the word “carefully” is suspicious – if one were to mention an animal species that does not sleep at all, he could evade the issue by claiming the species had not been studied carefully enough. Note also that the update still contradicts Walker’s original source, which, in the context surrounding the corresponding quote Guzey found, was already about animals (dolphins and humans). Maybe that’s why Walker replaced the endnote citation entirely:

I’m going to stop here. We can plausibly conclude that Walker made edits using Guzey’s notes. If someone wants to continue going through those notes and check the corresponding passages, they’ll probably find more changes.

Guzey’s response

I emailed Guzey a draft of this article to give him an opportunity to comment before publication. His response was disappointing. He only responded to the very first misquote in a way that revealed he doesn’t really understand what a misquote is.

I thought he’d be particularly interested in Walker’s changes. And I figured he’d be grateful someone took the time to take his claims seriously. He complains that “nobody gives a fuck” about Walker’s lack of integrity. I do. Wouldn’t it make sense to take criticisms regarding scholarly integrity seriously? Especially in an article about scholarly integrity?

Retraction

I have previously spoken favorably of Walker. As I recall, I was initially impressed with his appearance on Joe Rogan’s podcast around the time it came out. I never read his book but privately recommended him to others. Years later and armed with better epistemology, I re-listened to parts of the same podcast and noticed some errors, but I didn’t look into it further except to update my contacts about the fact that I had noticed said errors. (It is rare for anyone with a mistaken epistemology to make any significant progress. One glowing exception is Ayn Rand. Walker does not seem to be an exception.) As I’ve said, I haven’t read all of Walker’s response, but given Guzey’s criticisms, for now I retract my previous recommendation and advise caution. However, I’ve shown that Guzey needs to improve his criticisms.