You've heard this rant before

[muckefuck]

On one of my most frequently visited fora, someone is trying yet again to come up with an "objective" definition of "a language". I've seen this discussion so many times at this point I've got my canned arguments all ready to go: "languages" are just high-order abstractions; judgments of "similarity" are hopelessly subjective; lexical comparisons depend on a series of increasingly arbitrary definitions (e.g. which terms you choose to compare, how you measure vocabulary size, what definition of "word" you're using). But it never seems to matter: people seem to feel very strongly that "languages" are concrete things rather than convenient social fictions and, therefore, can be defined scientifically.

Of course, the real problem is that most people don't understand what that word means. Someone linked to this laundry list of intelligibility studies and asked if we thought it was "accurate". How could anybody tell? There's no methodology given at all, so we haven't the faintest idea what the researchers actually measured. It's like asking, "Is it true that when you're embarrassed you're also 27% angry?" Yeah, sure: choose the appropriate criteria for defining "embarrassment" and "anger" and the appropriate method for quantifying the data and you can get that percentage. Does that make it "scientific"? No.

But present your results in numerical form and most people will simply accept them. The first response to that blogpost begins, "I don’t dispute the scientific validity of those findings, but..." Well, then, you're a fool. Worse, you're a cargo cultist who believes the mere presence of numerical data is evidence of "science". For all you know, someone just banged out those percentages on a pub table over a beer or two. Or maybe they abstracted them from a single reading comprehension test. Who cares? One way or another, they don't tell you what you think they're telling you.

Comments

[paulintoronto.livejournal.com]

I discovered many years ago that students will complain about their final mark in a course. "I only got a 63% but I think it should be much higher."

If I say, "No, that is correct. That is the mark you deserve," they are never satisfied.

But if I say: "You got 6.3 out of 10 on four tests, and 12.6 out of 20 on your essay, plus 25.2 out of 40 on the exam: that adds up to 63%," they are almost always satisfied.

If they think I have just made up the final mark, they are angry, but the anger goes away if I present six different marks (each as susceptible to being made up as the final mark they received) and then added them together.

Math is magical! (Heck, even simple arithmetic has mystical powers.)

[snousle.livejournal.com]

Heh, our building contractor tried exactly that trick on us, except he actually was making up the numbers. We were not amused.

[muckefuck.livejournal.com]

That's more than just arithmetic: You've also presented a methodology. Which is, as I rant above, precisely what the dataset in the blogpost lacks.

[snousle.livejournal.com]

Well, at least numbers represent a higher degree of commitment to his position. When someone asks "how did you calculate those numbers?" and he says "uh, um, err...?" then the discussion is over. This is superior to not using numbers and thus not making any commitment at all. At least here there is an opportunity for progress.

[muckefuck.livejournal.com]

The response I'd be expecting would be more like, "I found them somewhere--I don't remember where so don't ask--but who are you to doubt them when they look SO SCIENCY!? Where are your sciency-looking numbers, huh?"

[tortipede.livejournal.com]

I had this once with a professor of Old English. I'd got sidetracked when looking at the poem Daniel: the central section is almost, but not quite, identical to a poem in another manuscript known as Azarias, and the question is always which has been interpolated into or excerpted from the other. The author of the preface to the edition pointed out that certain dialect forms of accusative personal pronouns are more common in the section at issue than in the rest of the poem, suggesting that it may be an interpolation, and I thought it ought to be possible to test that statistically.
At the end of the week I handed in to the somewhat bemused professor a sheet of stats, rather than the essay he was expecting. I got it back with the single comment, "This looks convincing, I think."
The thing was - never mind the fact that the test I'd used wasn't really adequate or that the professor didn't understand the maths - I'd made basic errors in the Old English: half the forms I'd counted were actually datives, something he was well qualified to spot if he hadn't been blinded by the spurious figures.