Why are psychologists and rehabilitationists so unaware that their research, questionnaires and treatment can cause harm?

[Hutan]

Such a good post @Ken Turnbull. I had to do a psychological assessment as part of a recruitment process and some of the questions asked were nonsense and for most, it was obvious what would be a 'wrong' answer, making gaming inevitable. After being offered the job, I mentioned that their recruitment process had problems. The company was paying a contractor to deliver a complicated and expensive screening process that I don't think was achieving much.

The 'I am fascinated by fire' question was part of the battery of questions - I remember saying later to the company that the correct answer for an employee in a company owning hundreds of thousands of hectares of forests is obviously 'no'. But actually it is part of the human condition to be fascinated by fire. We gather around fires - campfires, bonfires, fireplaces, charcoal barbecues. We have candles on tables. Fires have kept us warm, cooked our food, kept the dangerous wild animals away at night, provided light. Obviously, there can be a pathological level of fascination, but it's stupid to think that you are going to assess that by having a yes/no question on a form. If you are a pyromaniac, it's unlikely that you will tick the 'yes, I am fascinated by fire' box.

There's an industry around these surveys, but a lot of them are not much more useful than asking about someone's star sign.

 

[Yann04]

Reminds me of the “are you a terrorist” box you have to say no too when entering the US.

 

[Wyva]

Personal anecdote, not about questionnaires but similar about gaming etc these:

I was abused as a kid and there was this occasion when I was seen by a psychologist. I was quite little, definitely 10 or younger at primary school. I remember that I had to do a couple of different tasks that were fun, I saw them as creative playing and enjoyed doing them.

One such task was to build a town and I could use many different elements to do so: there were buildings, people, trees, etc. There was also one destroyed house and a few soldiers who looked like they are about to shoot. So as a creative kid, my personal goal was to be able to use ALL the available items in a way that they all fit in this nice little town of mine. That was my entire thought process. I had no other thoughts about this, just to somehow be able to use all the available parts. And that also meant using the destroyed building and the soldiers. Not because I was a violent deranged kid (I never was violent) but because I saw this as a creative task and I thought about it more like it was a movie scene, not real life. I saw it as a puzzle.

So I was absolutely about to use the war scene as well when my primary school-aged brain thought: wait a minute. I'm not just playing and having fun here. I'm at a psychologist. Right now she's testing me to see if I'm a psychopath. Oh boy, oh boy! So I put the soldier I grabbed back immediately, with a big "Nope!"

I don't know how often this happens with kids and that creative test but it happened with me.

 

[Peter T]

Such a good post @Ken Turnbull. I had to do a psychological assessment as part of a recruitment process and some of the questions asked were nonsense and for most, it was obvious what would be a 'wrong' answer, making gaming inevitable. After being offered the job, I mentioned that their recruitment process had problems. The company was paying a contractor to deliver a complicated and expensive screening process that I don't think was achieving much.

The 'I am fascinated by fire' question was part of the battery of questions - I remember saying later to the company that the correct answer for an employee in a company owning hundreds of thousands of hectares of forests is obviously 'no'. But actually it is part of the human condition to be fascinated by fire. We gather around fires - campfires, bonfires, fireplaces, charcoal barbecues. We have candles on tables. Fires have kept us warm, cooked our food, kept the dangerous wild animals away at night, provided light. Obviously, there can be a pathological level of fascination, but it's stupid to think that you are going to assess that by having a yes/no question on a form. If you are a pyromaniac, it's unlikely that you will tick the 'yes, I am fascinated by fire' box.

There's an industry around these surveys, but a lot of them are not much more useful than asking about someone's star sign.

In theory the individual questions are irrelevant, what matters is how your answers compare to other people completing the same questionnaire. In theory all the questions could be about flower arranging or building or horoscope usage as long as the results differentiate in relation to the trait being measured.

However this depends on a meaningful validation process which requires standardisation on a relevant population and some independent measure of the trait in question. For example as pointed out regularly here, measures of anxiety or depression contain lots of questions on physical activity, but were standardised on physically able populations so it is meaningless to use them on populations with chronic health issues or disabilities. Also in general there is no independent way to evaluate many of the traits being measured so you end up with circular arguments, as with the use of a questionnaire to measure ‘central sensitisation’ which is defined only in terms of how people answer a particular set of questions.