[Haskell-community] 2018 state of Haskell survey

Mon Oct 15 23:42:43 UTC 2018

Thanks for the kind words, Simon! They mean a lot :) 

I would be happy to include questions that would benefit the GHC team, including:

- Simon's question
- Reaction to the new pace of GHC releases
- Target of the GHC team's focus: performance, features, ergonomics, etc.
- Average wait time before upgrading GHC

Are there any other questions the GHC team would be interested in asking? Perhaps I should ask on a different mailing list.

In response to Gershom's comments:

1. Asking how people heard about the survey is a great idea. Not only would it let me identify the best ways to reach people, it could also be useful in dealing with selection bias. 

2. Addressed above.

3. In general, distinguishing between work and home is something I would love to do for basically every question. Unfortunately I think that would balloon the size of the survey. Maybe identifying a few key questions for the work/home split would be the best way to go? Build systems, as you identified, are certainly one of those key questions. Maybe GHC versions used is another? 

4. I also like Go's survey and have been trying to crib as much as I can from it. Questions worth asking:
- Area of development (web, embedded, etc.)
- Type of development (server, CLI, desktop, library, etc.)
- Deploy environments / infrastructure
- Internal versus external

5. Asking about JS solutions for web developers is a great idea! I like the choices you've given, and there are a whole slew of JS libraries to include as well, such as React or Vue. 

6. Giving multiple choice answers to the "why did you stop" question (and, in fact, as many questions as possible) is awesome and would make the results much easier to digest. It also makes things easier to compare across time, which could be used to gauge the effectiveness of various endeavors. 

Thank you all for your feedback so far! I am very excited about this year's survey. I want to include as many useful questions as I can without overwhelming respondents. As I continue to develop the survey, I constantly ask myself this question: "How would I act on responses to this question?" For example, last year's survey asked if people had contributed to an open source Haskell project. I suspect I will exclude that question because it's not really actionable. 

With regards to Haskell.org sponsorship, I still think that throwing around the words "official" and "Haskell.org" would do a lot in terms of credibility. I don't expect that to remove selection bias, but it will let me (us, really) say: We're doing this together for the benefit of all sides. And if people have problems with the survey, I want them to feel comfortable trying to fix those problems, even if they're not on my "side". 

On Mon, Oct 15, 2018, at 3:55 AM, Simon Peyton Jones wrote:
> The GHC team (coincidentally) is cooking up a very short survey
> that is intended to help guide our strategic priorities.
> 
> There is only one substantial question:
> 
> 	Imagine that you had one developer working on GHC for 
> 	six months full-time, that you were paying for yourself.
> 	What would you ask that person to do?   Keep in mind that
> 	the project should be within GHC itself and tractable
> 	in a six-month time-frame.
> 
> It has a rather different purpose to Taylor's proposed survey
> and FP Complete's, because it is not comprehensive, but instead
> focuses on a single question about a single artefact.
> 
> Returning to Taylor's question list, yes, it would be interesting to 
> know whether the increased release tempo is perceived as helpful or 
> unhelpful.  (It's helpful for the development /process/ because it makes 
> reduces the pressure to squeeze "just one more thing" into a release: 
> the next bus will be along in only 6 months.)
> 
> Incidentally, I for one think that it's a Good Thing that you included 
> FP Complete's survey in the HWN, Taylor.  First, I think it's a 
> substantial and interesting piece of work -- and /any/ survey is 
> vulnerable to response bias.  Second, I don’t think anyone should expect 
> you as HWN editor to play a role as community censor. Third, 
> deliberately excluding it would in itself be a divisive act in a 
> community that needs less division and more love.
> 
> You do a fantastic job with HWN.  Please keep doing it!
> 
> Thanks
> 
> Simon
> 
> | -----Original Message-----
> | From: Haskell-community <haskell-community-bounces at haskell.org> On
> | Behalf Of Gershom B
> | Sent: 15 October 2018 01:56
> | To: taylor at fausak.me; Mathieu Boespflug <mboes at tweag.net>; Ben Gamari
> | <bgamari at gmail.com>
> | Cc: haskell-community at haskell.org
> | Subject: Re: [Haskell-community] 2018 state of Haskell survey
> | 
> | (cc mathieu boespflug, ben gamari)
> | 
> | One more thought:
> | 
> | mathieu, ben -- do you think you would be interested in any questions
> | on the frequency of ghc releases -- if people appreciate more
> | frequent, smaller releases, or not, etc?
> | 
> | I wonder if there are any other questions as well about core libraries
> | vs. performance vs. "big features" (like type system things) vs. small
> | ergonomic features etc. that the core ghc team might be interested in
> | sounding out people on, bearing in mind the necessary limitations of
> | survey derived data.
> | 
> | --g
> | On Sun, Oct 14, 2018 at 8:36 PM Gershom B <gershomb at gmail.com> wrote:
> | >
> | > Hi Taylor.
> | >
> | > We're discussing this in the committee. I agree that to the extent
> | > they can accurately reflect something, language surveys are useful,
> | > and appreciate that you want to run a useful survey, and certainly
> | > want to encourage and help you in making it as broad and useful as
> | > possible. That said, I don't know if slapping a "haskell.org" label on
> | > the survey will help manage the biggest drivers of selection bias --
> | > which is not only about who chooses to respond, but about who is
> | > reached through what mechanisms. (I don't know the relative importance
> | > of response-bias vs. outreach-bias in general, and would be curious if
> | > somebody has some good research on that to point to). I honestly don't
> | > know if we have enough channels _in general_ to do a good survey, no
> | > matter who runs it or how at all! Regardless of the decision we come
> | > to, here are a few of my personal thoughts on the questions you have
> | > thus far, and what could be added:
> | >
> | > 1) A question "how did you hear about this survey" -- this could at
> | > least help to disentangle outreach-bias, or notice it, depending on if
> | > it induces any correlations.
> | >
> | > 2) A question on how long after a new GHC release users upgrade --
> | > both personally, and at work.
> | >
> | > 3) Distinguishing between personal and work build-systems in the
> | > relevant question.
> | >
> | > 4) I think the sorts of questions that make sense to ask in this early
> | > part can resemble those in the first part of the Go survey:
> | >
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.go
> | lang.org%2Fsurvey2017-
> | results&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bbf2
> | 008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6367516175958
> | 02229&sdata=2wB5Ph5%2Be6wZm0CWP7Yzx%2Fxe4dOlKHqNPKZReiJmE50%3D&r
> | eserved=0 (I especially like the
> | > questions about the area of development and server vs cli apps vs
> | > libraries). I also like their questions about what environments teams
> | > deploy programs to. It would also be worth asking if the apps
> | > developed are customer-facing or internal.
> | >
> | > 5) I think an interesting question would be what preferred js
> | > solution, if any, teams adopt -- i.e. ghcjs, typescript, purescript,
> | > raw js, etc.
> | >
> | > 6)  for the "why did you stop" question, there are a good range of
> | > potential multi-choice answers that can be drawn from with the rust
> | > user survey:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.ru
> | st-lang.org%2F2017%2F09%2F05%2FRust-2017-Survey-
> | Results.html&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e42
> | 0bbf2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63675161
> | 7595802229&sdata=MpnYeOui%2FfTPfpYRLlzzpStJHqYWJ4ENZd77qdrQ1BY%3D&am
> | p;reserved=0
> | >
> | > Cheers,
> | > Gershom
> | >
> | > On Sun, Oct 14, 2018 at 3:32 PM Taylor Fausak <taylor at fausak.me>
> | wrote:
> | > >
> | > > I'm not entirely sure what official support would look like. A few
> | things come to mind:
> | > >
> | > > 1. Simply putting "official" somewhere in the title, such as:
> | Official 2018 state of Haskell survey.
> | > >
> | > > 2. Putting something about Haskell.org in the description of the
> | survey, such as: Sponsored by Haskell.org.
> | > >
> | > > 3. Announcing the survey through channels that I may not be aware
> | of. Or helping me announce the survey through various channels (mailing
> | lists, Reddit, and so on) by mentioning Haskell.org.
> | > >
> | > > I'm sure there are more ways that I'm not thinking of.
> | > >
> | > > Perhaps it would be better to state my goal and see what, if
> | anything, can be to achieve that goal? My goal is for this survey to be
> | *the* authoritative Haskell survey and for the community to broadly
> | accept it results. In particular I would like to avoid reactions like
> | these to the recent FP Complete survey:
> | > >
> | > > > I browse r/haskell all the time and follow FPco on Twitter, and I
> | wasn't aware of this survey. [1]
> | > >
> | > > > It should not be surprising to think that fp complete has much
> | better outreach to Stackage users than to non Stackage users. [2]
> | > >
> | > > > Any survey hosted by FPComplete is biased towards users of stack,
> | for reasons that should be self evident. [3]
> | > >
> | > > Similar sentiments have been expressed about last year's Haskell
> | Weekly survey:
> | > >
> | > > > Haskell Weekly's reputation is tainted as it appears to be seen as
> | partisan (and I tend to agree). [4]
> | > >
> | > > > one of those surveys is from FP Complete and one of them is from
> | someone who I would consider very partisan in these kind of discussions.
> | [5]
> | > >
> | > > > For the love of god stop posting those surveys. They're not
> | convincing and obviously flawed. [6]
> | > >
> | > > I don't expect to be able to make everyone happy, but I think that
> | presenting this year's survey as sponsored by both Haskell Weekly and
> | Haskell.org would go a long way toward making it acceptable to a broad
> | range of the Haskell community.
> | > >
> | > > I hope that helps!
> | > >
> | > > [1]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnp.redd
> | it.com%2Fr%2Fhaskell%2Fcomments%2F9mm05d%2F2018_haskell_survey_results%2
> | Fe7gplya%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420
> | bbf2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636751617
> | 595802229&sdata=X518Wt%2BsB5yepxQO7iKxYzbcOBqNTwH4QvesKNVccBg%3D&amp
> | ;reserved=0
> | > > [2]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnp.redd
> | it.com%2Fr%2Fhaskell%2Fcomments%2F8tc8pr%2Ffp_complete_launches_new_bloc
> | kchain_auditing%2Fe188ftv%2F%3Fcontext%3D3&data=02%7C01%7Csimonpj%40
> | microsoft.com%7C9fbbfac1fd9e420bbf2008d63238f7c4%7C72f988bf86f141af91ab2
> | d7cd011db47%7C1%7C0%7C636751617595802229&sdata=4oPTVet8hrgE2lTzjt%2B
> | 02f13o%2Fq%2BOrnGoITlsBsdKgw%3D&reserved=0
> | > > [3]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnp.redd
> | it.com%2Fr%2Fhaskell%2Fcomments%2F9mm05d%2F2018_haskell_survey_results%2
> | Fe7gpfwe%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420
> | bbf2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636751617
> | 595802229&sdata=9L8ztYN3kOArpdAhOiydFI6gfuz8Evu5%2FmtHZ5j3vyY%3D&amp
> | ;reserved=0
> | > > [4]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnp.redd
> | it.com%2Fr%2Fhaskell%2Fcomments%2F9mm05d%2F2018_haskell_survey_results%2
> | Fe7ka8xn%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420
> | bbf2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636751617
> | 595802229&sdata=tpyLQ%2BA3Lj4GLfxjMUgglfRyl46jH6go1ZE3OhVLwLo%3D&amp
> | ;reserved=0
> | > > [5]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnp.redd
> | it.com%2Fr%2Fhaskell%2Fcomments%2F8tc8pr%2Ffp_complete_launches_new_bloc
> | kchain_auditing%2Fe187pzq%2F%3Fcontext%3D1&data=02%7C01%7Csimonpj%40
> | microsoft.com%7C9fbbfac1fd9e420bbf2008d63238f7c4%7C72f988bf86f141af91ab2
> | d7cd011db47%7C1%7C0%7C636751617595812242&sdata=ZbjRYFCXl4rraHE%2Bno7
> | kefm5N3%2FvtNpaI%2FXTz3N6i7w%3D&reserved=0
> | > > [6]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnp.redd
> | it.com%2Fr%2Fhaskell%2Fcomments%2F8uw9hw%2Fpsa_cabal_breaks_with_yaml083
> | 1_on_ghc_710_and%2Fe1lfzgr%2F%3Fcontext%3D1&data=02%7C01%7Csimonpj%4
> | 0microsoft.com%7C9fbbfac1fd9e420bbf2008d63238f7c4%7C72f988bf86f141af91ab
> | 2d7cd011db47%7C1%7C0%7C636751617595812242&sdata=oyy8raidob%2Bjf8Evw0
> | Kb6aCul0knqYlOlxN8zLjzl3I%3D&reserved=0
> | > >
> | > >
> | > > On Sun, Oct 14, 2018, at 2:21 PM, Neil Mitchell wrote:
> | > >
> | > > Hi Taylor,
> | > >
> | > > What does official support look like? I don't think there's anything
> | above and beyond what you're already doing.
> | > >
> | > > Thanks, Neil
> | > >
> | > > On Sun, 14 Oct 2018 at 3:49 pm, Taylor Fausak <taylor at fausak.me>
> | wrote:
> | > >
> | > > (I have CCed <committee at haskell.org> because the haskell-community
> | mailing list seems relatively quiet and I want to make sure this is
> | seen.)
> | > >
> | > > Hello! My name is Taylor Fausak. I run the Haskell Weekly
> | newsletter. Last year I published a survey [1] for the Haskell
> | community. I collected and reported [2] on about 1,335 responses. I plan
> | on publishing another survey this year on the same date, November 1st. I
> | am developing it in the open again [3] and would love to hear from any
> | interested parties. Please let me know if you have any ideas about the
> | survey!
> | > >
> | > > This year I am interested in making the survey official by seeking
> | support from Haskell.org. Is such a thing possible and desirable?
> | > >
> | > > Thanks for your consideration! I hope to hear from you soon.
> | > >
> | > > [1]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhaskell
> | weekly.news%2Fsurveys%2F2017.html&data=02%7C01%7Csimonpj%40microsoft
> | .com%7C9fbbfac1fd9e420bbf2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db
> | 47%7C1%7C0%7C636751617595812242&sdata=0eGdCpPFIyxgNp5YNXvP6TZQMAvTJ%
> | 2BIck4UvSV7wWSU%3D&reserved=0
> | > > [2]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftaylor.
> | fausak.me%2F2017%2F11%2F15%2F2017-state-of-haskell-survey-
> | results%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420b
> | bf2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6367516175
> | 95812242&sdata=QZjow3PKFG7BitCQCdTe7xW%2BXl4YA4ICVmA%2B87GXOCs%3D&am
> | p;reserved=0
> | > > [3]:
> | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.
> | com%2Fhaskellweekly%2Fhaskellweekly.github.io%2Fissues%2F206&data=02
> | %7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bbf2008d63238f7c4%7C72f
> | 988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636751617595812242&sdata=v2b
> | W4fjDtWrJnKACnQaoLRoaYw0kADGucwasn2X0wtI%3D&reserved=0
> | > > _______________________________________________
> | > > Haskell-community mailing list
> | > > Haskell-community at haskell.org
> | > >
> | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
> | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fhaskell-
> | community&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bb
> | f2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63675161759
> | 5812242&sdata=2SB8gyaEQI2zxvUE%2FEdw%2B%2B2OLcJW10hpH8A1McoP0MY%3D&a
> | mp;reserved=0
> | > >
> | > >
> | > > _______________________________________________
> | > > Haskell-community mailing list
> | > > Haskell-community at haskell.org
> | > >
> | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
> | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fhaskell-
> | community&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bb
> | f2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63675161759
> | 5812242&sdata=2SB8gyaEQI2zxvUE%2FEdw%2B%2B2OLcJW10hpH8A1McoP0MY%3D&a
> | mp;reserved=0
> | _______________________________________________
> | Haskell-community mailing list
> | Haskell-community at haskell.org
> | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
> | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fhaskell-
> | community&data=02%7C01%7Csimonpj%40microsoft.com%7C9fbbfac1fd9e420bb
> | f2008d63238f7c4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63675161759
> | 5812242&sdata=2SB8gyaEQI2zxvUE%2FEdw%2B%2B2OLcJW10hpH8A1McoP0MY%3D&a
> | mp;reserved=0