[Haskell-community] 2018 state of Haskell survey results

[Chris Smith]

> For example, just reading this thread, it sounds like the bogus responses
also really don't like the new release schedule. Maybe the troll wants the
old release schedule back and was just lazy about programming the tool to
vary the stack/cabal question answers adequately.

There is another scenario, though, which should caution against making
official statements about motivation.  There was a set of people who worked
very hard while the survey was open to preemptively cast doubt on its
motivation and goals.  It may be that someone was mainly attempting to
sabotage the survey results themselves, rather than taking a side in any
specific dispute.  Of course, had the results been published claiming that
a mere 12% of Haskellers use Cabal, it would have been immediately
dismissed by many people as obviously biased, which would have achieved
that goal, too.

I think Taylor's post handled this well, saying what we know to be true,
that the attack targeted divisive issues, but without drawing unnecessary
conclusions.

On Sun, Nov 18, 2018 at 11:21 PM Richard Eisenberg <rae at cs.brynmawr.edu>
wrote:

> I have not analyzed the data myself, but I wonder how we jumped to the
> conclusion that the troll was trying to promote Stack. Is there statistical
> data that supports that conclusion? For example, just reading this thread,
> it sounds like the bogus responses also really don't like the new release
> schedule. Maybe the troll wants the old release schedule back and was just
> lazy about programming the tool to vary the stack/cabal question answers
> adequately.
>
> Given the contention around cabal vs stack, I agree that sociological
> concerns suggest that the troll meant to tilt those scales. But I wouldn't
> want a public accusation without at least some statistical analysis that
> independently supports that conclusion.
>
> In any case, thanks to all for putting this together!
>
> Richard
>
> On Nov 18, 2018, at 4:31 PM, Taylor Fausak <taylor at fausak.me> wrote:
>
> Oops, the ordering of the answer choices is manual because some questions
> have a natural order while others should just be most to least popular.
> I've made another run through to make sure everything is sorted properly.
> I'll probably hit publish in the next half hour or so unless there are any
> objections.
>
>
> https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>
>
> On Sun, Nov 18, 2018, at 3:07 PM, Gershom B wrote:
>
> The language extensions section doesn’t appear to be sorted properly.
> Outside of that, I think that these results are looking much better and any
> effort to find any additional outliers is probably not worth it for the
> moment. Thanks for your work on this, and I appreciate you being responsive
> and attentive when problems with the data were pointed out. There’s
> certainly some interesting and helpful information to be gleaned from this
> data.
>
> Cheers,
> Gershom
>
>
>
>
> On November 18, 2018 at 2:55:10 PM, Taylor Fausak (taylor at fausak.me)
> wrote:
>
>
>
>
> Ok, I updated the function that checks for bad responses, re-ran the
> script, and updated the announcement along with all the assets (charts,
> tables, and CSV). Hopefully it's the last time, as I can't justify spending
> much more time on this.
>
>
> https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown
>
>
> On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:
>
> Just wanted to add in: good catch Gershom on identifying the problem, and
> thank you Taylor for working to remove them from the report.
>
> On 18 Nov 2018, at 21:17, Taylor Fausak <taylor at fausak.me> wrote:
>
> Great catch, Gershom! There are indeed about 300 responses that tick all
> the boxes except for disliking the new GHC release schedule. The main thing
> the attacker seemed to be interested in was over-representing Stack and
> Stackage. Also, bizarrely, Java.
>
> That brings the number of bogus responses up to 3,735, which puts the
> number of legitimate responses at 1,361. For context, last year's survey
> asked far fewer questions and had 1,335 responses.
>
>
> On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:
>
> What if the announcement mentioned a large number of potentially bogus
> responses, explained the grounds for this conclusion, with a new survey
> conducted early next year?
>
> The next survey would then need to be done differently from this one
> somehow. To improve the reliability, some authentication may be necessary.
>
>
> Maybe Stack, Cabal questions could be grouped as separate distinct
> surveys, conducted by their maintainers through own channels?
>
> Not sure how much value is in exact numbers of users of Stack or Cabal.
> Both groups are large enough. The maintainers of both groups are aware
> about usage stats.
>
> Is either library likely to be influenced by this survey?
> *_______________________________________________*
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-community/attachments/20181118/6fa703cf/attachment.html>