[Haskell-community] 2018 state of Haskell survey

[Gershom B]

Hi Taylor.

We're discussing this in the committee. I agree that to the extent
they can accurately reflect something, language surveys are useful,
and appreciate that you want to run a useful survey, and certainly
want to encourage and help you in making it as broad and useful as
possible. That said, I don't know if slapping a "haskell.org" label on
the survey will help manage the biggest drivers of selection bias --
which is not only about who chooses to respond, but about who is
reached through what mechanisms. (I don't know the relative importance
of response-bias vs. outreach-bias in general, and would be curious if
somebody has some good research on that to point to). I honestly don't
know if we have enough channels _in general_ to do a good survey, no
matter who runs it or how at all! Regardless of the decision we come
to, here are a few of my personal thoughts on the questions you have
thus far, and what could be added:

1) A question "how did you hear about this survey" -- this could at
least help to disentangle outreach-bias, or notice it, depending on if
it induces any correlations.

2) A question on how long after a new GHC release users upgrade --
both personally, and at work.

3) Distinguishing between personal and work build-systems in the
relevant question.

4) I think the sorts of questions that make sense to ask in this early
part can resemble those in the first part of the Go survey:
https://blog.golang.org/survey2017-results (I especially like the
questions about the area of development and server vs cli apps vs
libraries). I also like their questions about what environments teams
deploy programs to. It would also be worth asking if the apps
developed are customer-facing or internal.

5) I think an interesting question would be what preferred js
solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
raw js, etc.

6)  for the "why did you stop" question, there are a good range of
potential multi-choice answers that can be drawn from with the rust
user survey: https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html

Cheers,
Gershom

On Sun, Oct 14, 2018 at 3:32 PM Taylor Fausak <taylor at fausak.me> wrote:
>
> I'm not entirely sure what official support would look like. A few things come to mind:
>
> 1. Simply putting "official" somewhere in the title, such as: Official 2018 state of Haskell survey.
>
> 2. Putting something about Haskell.org in the description of the survey, such as: Sponsored by Haskell.org.
>
> 3. Announcing the survey through channels that I may not be aware of. Or helping me announce the survey through various channels (mailing lists, Reddit, and so on) by mentioning Haskell.org.
>
> I'm sure there are more ways that I'm not thinking of.
>
> Perhaps it would be better to state my goal and see what, if anything, can be to achieve that goal? My goal is for this survey to be *the* authoritative Haskell survey and for the community to broadly accept it results. In particular I would like to avoid reactions like these to the recent FP Complete survey:
>
> > I browse r/haskell all the time and follow FPco on Twitter, and I wasn't aware of this survey. [1]
>
> > It should not be surprising to think that fp complete has much better outreach to Stackage users than to non Stackage users. [2]
>
> > Any survey hosted by FPComplete is biased towards users of stack, for reasons that should be self evident. [3]
>
> Similar sentiments have been expressed about last year's Haskell Weekly survey:
>
> > Haskell Weekly's reputation is tainted as it appears to be seen as partisan (and I tend to agree). [4]
>
> > one of those surveys is from FP Complete and one of them is from someone who I would consider very partisan in these kind of discussions. [5]
>
> > For the love of god stop posting those surveys. They're not convincing and obviously flawed. [6]
>
> I don't expect to be able to make everyone happy, but I think that presenting this year's survey as sponsored by both Haskell Weekly and Haskell.org would go a long way toward making it acceptable to a broad range of the Haskell community.
>
> I hope that helps!
>
> [1]: https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gplya/
> [2]: https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e188ftv/?context=3
> [3]: https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7gpfwe/
> [4]: https://np.reddit.com/r/haskell/comments/9mm05d/2018_haskell_survey_results/e7ka8xn/
> [5]: https://np.reddit.com/r/haskell/comments/8tc8pr/fp_complete_launches_new_blockchain_auditing/e187pzq/?context=1
> [6]: https://np.reddit.com/r/haskell/comments/8uw9hw/psa_cabal_breaks_with_yaml0831_on_ghc_710_and/e1lfzgr/?context=1
>
>
> On Sun, Oct 14, 2018, at 2:21 PM, Neil Mitchell wrote:
>
> Hi Taylor,
>
> What does official support look like? I don't think there's anything above and beyond what you're already doing.
>
> Thanks, Neil
>
> On Sun, 14 Oct 2018 at 3:49 pm, Taylor Fausak <taylor at fausak.me> wrote:
>
> (I have CCed <committee at haskell.org> because the haskell-community mailing list seems relatively quiet and I want to make sure this is seen.)
>
> Hello! My name is Taylor Fausak. I run the Haskell Weekly newsletter. Last year I published a survey [1] for the Haskell community. I collected and reported [2] on about 1,335 responses. I plan on publishing another survey this year on the same date, November 1st. I am developing it in the open again [3] and would love to hear from any interested parties. Please let me know if you have any ideas about the survey!
>
> This year I am interested in making the survey official by seeking support from Haskell.org. Is such a thing possible and desirable?
>
> Thanks for your consideration! I hope to hear from you soon.
>
> [1]: https://haskellweekly.news/surveys/2017.html
> [2]: https://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/
> [3]: https://github.com/haskellweekly/haskellweekly.github.io/issues/206
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
>
>
> _______________________________________________
> Haskell-community mailing list
> Haskell-community at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community