The crusade against multiple regression analysis

’A huge range of science projects are done with multiple regression analysis. The results are often somewhere between meaningless and quite damaging’, Michigan University professor of psychology Richard Nisbett said in 2016 during a conversation on Egde [1], and added that, should his crusade succeed, ’there’ll be a kind of upfront warning in New York Times articles: These data are based on multiple regression analysis. This would be a sign that you probably shouldn’t read the article because you’re quite likely to get non-information or misinformation’.
Nisbett is so damn right!

When I studied social psychology in the 1970s, I was bewildered by how my colleagues analysed data. I simply couldn’t understand how they could trust in such a high level and precision of data which I knew had been collected rather by chance, episodically, approximatively, and with the bias of the researcher himself. No, my problem was not so much with the data at hand, the best we could get – I had a big problem with the statistical methods that were en vogue at the time.

Maybe it was because I was a dead loss in maths and had to work hard to learn useful and reliable methods for analysing my own survey data. I felt I had invested too much time to simply throw the data into some statistical laundry.

At that time, I came across the book ’Nonparametric Statistics’ by psychologist Sidney Siegel [2], which became my bible and formed the basis for an evaluation programme that I had written myself in Fortran IV, just one year before the famous SSPS came onto the market. When this happened, my programme became useless, but my colleagues stopped making jokes behind my back. A strange kind of success, though.

Decades later, I am truly astonished to learn that the crusade against data analysis on inappropriate scales of measure still needs to be fought. I would have expected that even the so-called exact sciences would have learnt by now to be satisfied with nominal and ordinal scales if their data do not respond to interval or even ratio scale. Instead of analysing the available data, appropriately, there still seems to be a compulsion to fool the scientific community with sugarcoated data analysis. Perhaps this is also the price to pay for the current exaggerated need to publish in order to gain recognition and climb the scientific career ladder…

---

First published on 25.01.2026 on Facebook, edited on 25.01.2026

---

References:
[1] Edge, 21.01.2016: ’A Conversation With Richard Nisbett¨’
[2] https://www.siegel-memorial.org/sid/index.html

---

Discussion on Facebook (2016):

Frank: Yes, its absolutely right… but the first point also needs emphasizing: that the methods of data collection and the validity of the data often does not justify the analysis. I became very aware of this in Africa, very aware.

Billo: Not just in Africa, Frank. When I was working in commercial social research, my hairs (I had still some at that time) would stand on end when I crossed the interviewer department and chanced to hear some internal stories… Africans at least know to tell stories!

Frank: So true, Billo… the African farmers for e.g. might tell stories… the ones they think will please the interviewer… but I also meant the standard of junior and intermediate staff in Africa is very low (maybe in Europe as well). So, some UN expert could sit in his air con office in Nairobi or somewhere. But it is the locally employed staff (who know the local languages) who do the surveys… but these local staff have not been instructed about what the survey isi actually for, so they have no idea of its importance, no interest in it…. sometimes they even just went into a cafe and invented the answers.

Billo: Same here, as far as I learnt, Frank. One of the top interviewers would go to the lido in the summer and gather some people round him, trying to meet the quota he got: 5 under 45, 5 over, half men, half women… if he could not meet it with the people at hand, he just assigned the parts missing, like: you are now a lady under 45, please answer my questions accordingly (to a man over 45, e.g.). Than he read one answer, listened to the chaos of answers, noted anything in his file and proceeded with the next question. But he was the star of the department! And than, these data were multiply regressed (and regression is the right word here, in another sense, is it not?).
I can tell you all this frankly because the institute has been closed many years now…

Simone: Multiple Regressionsmodelle sind fehleranfällig; aber noch mehr Fehlerpotential liegt in unterkomplexen Kausalmodellen (die Sozialpsychologie ist bekannt dafür) oder in überkomplexen Kausalmodellen (z. B. Pädagogik) mit kleinen, kaum interpretierbaren Effektstärken, die dann in den Journals verwertet und überbewertet werden (müssen). Siehe auch z.B. die Kritik von John Hattie.

Billo: Die Sozialpsychologie, die ich studierte, hat eben Aufmerksamkeit darauf gelegt, sich nicht mit Korrelationen zwischen zwei Variablen zufriedenzugeben. Doch das ist ja primär keine Frage der statistischen Methode, sondern gut überlegten, neugierigen und selbstkritischen Forschens. Wer das nicht bringt, kann allerdings versucht sein, sich hinter einem statistischen Overkill zu verstecken…

Frank: Statistics is a branch of mathematics and people can get degrees in statistics. It is a highly rigorous subject. But the problem is that statistics are used as an integral part of many subjects, such as sociology, geography, economics, business studies etc, by people who simply do not understand the ‚bedrock‘ of statistics. And these fairly ignorant people teach their students who then write dissertations etc which are loaded with bogus statistics. It is not amusing, it is dangerous, because these false correlations etc are used as a basis for policy decisions.

Simone: Und es untergräbt leider die Glaubwürdigkeit guter Studien, die es auch gibt.

Frank: Gut gesagt, Simone!

Billo: Frank, I was taught statistics first in high school by a ’backbone’ maths prof and later in the frame of my studies by a maths and IT guy. At least in my case it was not the fault of my teachers if I did something wrong. But the statistics teacher I really listened to was Siegel.

Billo: Simone, was ist eine gute Studie? Heute ist eine Studie vor allem gut, wenn sie publiziert wird und die eigene Karriere befördert, für die man schon fast pausenlos publizieren muss…
Für mich ist eine Studie dann gut, wenn ihre Hypothese klar formuliert ist, wenn ich die angestellte Untersuchung nachvollziehen kann und wenn mir die Daten so präsentiert werden, dass ich deren Auswertung und Interpretation überprüfen kann. In den Disziplinen, in denen ich halbwegs zuhause bin (Sozialpsychologie, Psychologie, Ethologie) werde ich bei grossem statistischem Überbau reflexartig skeptisch. Ich ziehe vergleichsweise «rohe» Auswertungsverfahren vor, sie entsprechen dem rohen Material einfach besser. Am liebsten ist mir ein experimentelles Vorgehen, beim dem der Einfluss von Variablen schrittweise geprüft bzw. ausgeschlossen wird.

Simone: Naja ich kann ja kein Buch schreiben hier… Also, gute deskriptive Studien und Daten (z.B. BFS), Längsschnittstudien und Experimente sind Gemälde. Querschnittstudien sind erste Skizzen, werden aber oft ins Schaufenster gestellt und überinterpretiert.