For this count, sentences were segmented using the NLP library, spaCy.

By token we mean an occurrence of a word—a string of characters with no intervening spaces. For this particular count, em-dashes, en-dashes, slashes, and ellipses are also treated as delimiters between tokens but other punctuation such as hyphens or apostrophes are not. If we just split on whitespace, we’d get 148,630 tokens. Using spaCy’s tokenization, which treats punctuation as tokens in their own right, the count is 191,275.