the lure of surveys

posted in: Blog | 7

I am not a fan of surveys. Their badly worded questions, their nonsensical rating scales, their overall sloppy design upsets me — especially when the results are dumped in my lap and I am expected to “analyse” them and extract meaningful truths from them. “Why didn’t you ask for my help before you sent this out?” I wonder, politely, inside my head…

But. As a student I am on the receiving end of at least a couple of surveys every week, and I have become quite the survey junkie. The trick is that they repeatedly offer me the chance to win cash or book tokens, i.e. things that are desperately needed around here. Unfortunately, I have filled in dozens of the things over the past four years and have yet to win so much as a fiver. The habit has been formed, however, and so yesterday I found myself filling in a survey which offered no reward at all other than the feeling of being a participative citizen in “the ecology of the Visual Arts and the Crafts sectors in Scotland”.

It seems that Creative Scotland (what used to be the Scottish Arts Council) are consulting — so if you are also contributor to that “ecology” then you might be interested in responding here.

I noticed in the twitterverse yesterday that some folk were unhappy about the questions regarding income and felt they were too invasive; however, I can report from experience that it is perfectly fine to skip over those without answering. Personally, I am not particularly shy about income questions in surveys, but I do lack confidence in the ability of researchers to process the information appropriately. Earlier this year there was a big report produced called Craft in an Age of Change (the pdf is here). It was a collaborative effort of the arts councils across the UK and looked at changing patterns of employment in the crafts sector, so it made interesting reading.

But OH DEAR I wish they had asked for help with the numbers. On page 28, for example, they report that 30% of makers have some income from activities not related to craft while 70% do not. The 30% are earning, on average, some £12,000 per annum from their non-craft. The 70%, of course, earn nothing in this category*. According to the report, “This implies that the average non craft-related income across the whole sample of makers with financial information on this issue is £3,658 (=70%*0+30%*£12,284).” No, my dears, it really really doesn’t imply that at all. It implies that your data is very skewed, that using an “average” is way too simplistic for this situation and that you need to stand back from the numbers NOW before you get any further out of your depth. Of course, they don’t do that. They just carry on averaging their way to unjustified conclusions.

So, in the current consultation, the open call might well be a better bet than the survey. Just saying.

Meanwhile, there is exuberance in the garden at the sudden appearance of the sun.

*Funnily enough, as I am typing this, I am listening to my favourite lunchtime programme — The Culture Café on Radio Scotland — and they’re interviewing two people who combine art (acting, photography) and non-art (office work, pharmacy) jobs.

the lure of surveys” was posted by Cally on 22 May 2012 at http://callybooker.co.uk

Creative Commons License

7 Responses

  1. Amy
    | Reply

    Oh my gosh!! That fuzzy belly. I want to reach through the monitor and give a big kiss!

    • Cally
      | Reply

      She’s so plush and inviting, isn’t she? But you’d risk getting a sharp set of teeth wrapped around your nose!

  2. Laura
    | Reply

    Interesting the conclusions that non-participants in craft draw. 🙁

    cheers,
    Laura
    a starving artist for 35+ years….

  3. Meg in Nelson
    | Reply

    I love taking surveys because it lets me know what I think from the very mundane (when was the last time I bought sugary fizzy drink?) to the amusing (multiple questions rephrased intended to ask me basically the same thing, but how I answer differently according to the question preceding and/or the wording of the question… in question…)

    I know what you mean about the average, though, and this is why I tend not to trust survey results where % or average is presented. In very small groups, (the late Nelson Arts Marketing comes to mind,) I liked it when the number of responses are stated.

    Here, there is exuberance among the weeds!

  4. Kerstin
    | Reply

    Well – remember the saying “if you have one hand in boiling water and the other in a bucket of ice, then you are, IN AVERAGE, having a good time”?
    (Possible addendum: and only 50% is getting burns… so it can’t be *that* bad, can it?)

    Kerstin, who has spent far too much time lately trying to convey the idea that one or two “readings” do not a good average make…

    • Cally
      | Reply

      I need a “like” button for comments so I can like yours, Kerstin! Quite so.

  5. Trapunto
    | Reply

    This made me laugh and laugh. It made me wonder if the people shuffling the papers had ever actually observed the elusive Crafter in its native habitat.

Leave a Comment