[Haskell-community] 2018 state of Haskell survey results

[Gershom B]

I also noticed a number of other bizarre statistical anomolies when looking
at the full results. I know this is a bit much to ask — but if you could
rerun the statistics filtering out people that did not give demographic
information (i.e. country of origin or education, etc) I think the results
will change drastically. By all statistical logic, this should _not_ be the
case, and points to a serious problem.

In particular, this drops the results by a huge amount — only 1,200 or so
remain. However, the remaining results tend to make a lot more sense. For
example — of the “no demographics” group, there are 713 users who claim to
develop with notepad++ but all of these say they develop on mac and linux,
and none on windows — which is impossible, as notepad++ is a windows
program. Further if you drop the “no demographics” group, then you find
that almost everyone uses at least ghc 8.0.2, while in the “no
demographics” group,  a stunning number of people claim to be on 7.8.3.
Even more bizarrely, people claim to be using the 7.8 series while only
having used Haskell for less than one year. And people claim to have used
haskell for “one week to one month” and also to be advanced and expert
users!

The differences continue and defy all probability. Of the “no demographics”
group, almost everyone dislikes the new release schedule. Of the
“demographics” group there are answers that like it, were not aware of it,
or are indifferent, but almost nobody dislikes it. There is naturally a
difference in proportions of cabal/stack and hackage/stackage responses as
well.

There are a lot of other things I could point to as well. But, bluntly put,
I think that some disaffected party or parties wrote a crude script and
submitted over 3,000 fake responses. Luckily for us, they were not very
smart, and made some obvious errors, so in this case we can weed out the
bad responses (although, sadly, losing at least a few real ones as well).

However, assuming  this party isn’t entirely stupid, it doesn’t bode well
for future surveys as they may get at least slightly less dumb in the
future if they decide to keep it up :-/

—Gershom


On November 18, 2018 at 1:10:31 AM, Gershom B (gershomb at gmail.com) wrote:

This is interesting, but I’m thoroughly confused. Over 2500 people said
they took last year’s survey, but it only had roughly 1,300 respondants?


On Sat, Nov 17, 2018 at 9:56 PM Taylor Fausak <taylor at fausak.me> wrote:

> Hello! It took a little longer than I expected, but I am nearly ready to
> announce the 2018 state of Haskell survey results. Some community members
> have expressed interest in seeing the announcement post before it's
> published. If you are one of those people, you can see the results here:
> https://github.com/tfausak/tfausak.github.io/blob/7e4937e284a3068add9e9af6b585c8d0215ff360/_posts/2018-11-16-2018-state-of-haskell-survey-results.markdown
>
> If you would like to suggest changes to the announcement post, please
> respond to this email, send me an email directly, or reply to this pull
> request on GitHub: https://github.com/tfausak/tfausak.github.io/pull/148
>
> I plan on publishing the results tomorrow. Once the results are published,
> the post is by no means set in stone. I will happily accept suggestions
> from anyone at any time.
>
> Thank you!
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-community/attachments/20181118/5f7245e8/attachment.html>