<div dir="ltr">> For example, just reading this thread, it sounds like the bogus responses also really don't like the new release schedule. Maybe the troll wants the old release schedule back and was just lazy about programming the tool to vary the stack/cabal question answers adequately.<div><br></div><div>There is another scenario, though, which should caution against making official statements about motivation.  There was a set of people who worked very hard while the survey was open to preemptively cast doubt on its motivation and goals.  It may be that someone was mainly attempting to sabotage the survey results themselves, rather than taking a side in any specific dispute.  Of course, had the results been published claiming that a mere 12% of Haskellers use Cabal, it would have been immediately dismissed by many people as obviously biased, which would have achieved that goal, too.<br></div><div><br></div><div>I think Taylor's post handled this well, saying what we know to be true, that the attack targeted divisive issues, but without drawing unnecessary conclusions.</div></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Nov 18, 2018 at 11:21 PM Richard Eisenberg <<a href="mailto:rae@cs.brynmawr.edu" target="_blank">rae@cs.brynmawr.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">I have not analyzed the data myself, but I wonder how we jumped to the conclusion that the troll was trying to promote Stack. Is there statistical data that supports that conclusion? For example, just reading this thread, it sounds like the bogus responses also really don't like the new release schedule. Maybe the troll wants the old release schedule back and was just lazy about programming the tool to vary the stack/cabal question answers adequately.<div><br></div><div>Given the contention around cabal vs stack, I agree that sociological concerns suggest that the troll meant to tilt those scales. But I wouldn't want a public accusation without at least some statistical analysis that independently supports that conclusion.</div><div><br></div><div>In any case, thanks to all for putting this together!</div><div><br></div><div>Richard<br><div><br><blockquote type="cite"><div>On Nov 18, 2018, at 4:31 PM, Taylor Fausak <<a href="mailto:taylor@fausak.me" target="_blank">taylor@fausak.me</a>> wrote:</div><br class="m_5405127461866784255m_-4930649622469628289m_8264192315836989908m_2280566610556847867Apple-interchange-newline"><div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">Oops, the ordering of the answer choices is manual because some questions have a natural order while others should just be most to least popular. I've made another run through to make sure everything is sorted properly. I'll probably hit publish in the next half hour or so unless there are any objections.<br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><a href="https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown" target="_blank">https://github.com/tfausak/tfausak.github.io/blob/fce97d07c369856d4c05b756c492eb6229a1b5c7/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown</a><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">On Sun, Nov 18, 2018, at 3:07 PM, Gershom B wrote:<br></div><blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div style="font-family:Helvetica,Arial;font-size:13px;margin:0px">The language extensions section doesn’t appear to be sorted properly. Outside of that, I think that these results are looking much better and any effort to find any additional outliers is probably not worth it for the moment. Thanks for your work on this, and I appreciate you being responsive and attentive when problems with the data were pointed out. There’s certainly some interesting and helpful information to be gleaned from this data.<br></div><div style="font-family:Helvetica,Arial;font-size:13px;margin:0px"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;margin:0px">Cheers,<br></div><div style="font-family:Helvetica,Arial;font-size:13px;margin:0px">Gershom<br></div><div style="font-family:Helvetica,Arial;font-size:13px;margin:0px"><br></div><div><br></div><div><br></div><div><br></div><p>On November 18, 2018 at 2:55:10 PM, Taylor Fausak (<a href="mailto:taylor@fausak.me" target="_blank">taylor@fausak.me</a>) wrote:<br></p><blockquote type="cite"><div><span></span><br></div><div><div><br></div><div><div><span></span><br></div><div><span>Ok, I updated the function that checks for bad responses, re-ran the script, and updated the announcement along with all the assets (charts, tables, and CSV). Hopefully it's the last time, as I can't justify spending much more time on this.</span><br></div><div><span></span><br></div><div><span><a href="https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown" target="_blank">https://github.com/tfausak/tfausak.github.io/blob/6f9991758ffeed085c45dd97e4ce6a82a8b1a73f/_posts/2018-11-18-2018-state-of-haskell-survey-results.markdown</a></span><br></div><div><span></span><br></div><div><span></span><br></div><div><span>On Sun, Nov 18, 2018, at 2:32 PM, Michael Snoyman wrote:</span><br></div><blockquote type="cite"><div><span>Just wanted to add in: good catch Gershom on identifying the problem, and thank you Taylor for working to remove them from the report.</span><br></div><div><div><span></span><br></div><blockquote type="cite"><div><span>On 18 Nov 2018, at 21:17, Taylor Fausak <<a href="mailto:taylor@fausak.me" target="_blank">taylor@fausak.me</a>> wrote:</span><br></div><div><span></span><br></div><div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span>Great catch, Gershom! There are indeed about 300 responses that tick all the boxes except for disliking the new GHC release schedule. The main thing the attacker seemed to be interested in was over-representing Stack and Stackage. Also, bizarrely, Java.<span> </span></span><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span></span><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span>That brings the number of bogus responses up to 3,735, which puts the number of legitimate responses at 1,361. For context, last year's survey asked far fewer questions and had 1,335 responses.</span><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span></span><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span></span><br></div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span>On Sun, Nov 18, 2018, at 1:26 PM, Imants Cekusins wrote:</span><br></div><blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div><div><span>What if the announcement mentioned a large number of potentially bogus responses, explained the grounds for this conclusion, with a new survey conducted early next year?</span><br></div><div><span></span><br></div><div><span>The next survey would then need to be done differently from this one somehow. To improve the reliability, some authentication may be necessary.</span><br></div><div><span></span><br></div><div><span></span><br></div><div><span>Maybe Stack, Cabal questions could be grouped as separate distinct surveys, conducted by their maintainers through own channels?</span><br></div><div><span></span><br></div><div><span>Not sure how much value is in exact numbers of users of Stack or Cabal. Both groups are large enough. The maintainers of both groups are aware about usage stats.</span><br></div><div><span></span><br></div><div><span>Is either library likely to be influenced by this survey?</span><br></div></div><div><span><u>_______________________________________________</u></span><br></div><div><span>Haskell-community mailing list</span><br></div><div><span><a href="mailto:Haskell-community@haskell.org" target="_blank">Haskell-community@haskell.org</a></span><br></div><div><span><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community</a></span><br></div></blockquote><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span></span><br></div><div><span><span class="m_5405127461866784255m_-4930649622469628289m_8264192315836989908m_2280566610556847867font" style="font-family:Helvetica"><span class="m_5405127461866784255m_-4930649622469628289m_8264192315836989908m_2280566610556847867size" style="font-size:12px">_______________________________________________</span></span></span><br></div><div><span><span class="m_5405127461866784255m_-4930649622469628289m_8264192315836989908m_2280566610556847867font" style="font-family:Helvetica"><span class="m_5405127461866784255m_-4930649622469628289m_8264192315836989908m_2280566610556847867size" style="font-size:12px">Haskell-community mailing list</span></span></span><br></div><div><span><a href="mailto:Haskell-community@haskell.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">Haskell-community@haskell.org</a></span><br></div><div><span><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community</a></span><br></div></div></blockquote></div></blockquote><div><span></span><br></div><div><span>_______________________________________________<br>Haskell-community mailing list<br><a href="mailto:Haskell-community@haskell.org" target="_blank">Haskell-community@haskell.org</a><br><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community</a></span></div></div></div><div><span></span><br></div></blockquote></blockquote><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><br></div><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">Haskell-community mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important"><a href="mailto:Haskell-community@haskell.org" target="_blank">Haskell-community@haskell.org</a></span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important"><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community</a></span></div></blockquote></div><br></div></div>_______________________________________________<br>
Haskell-community mailing list<br>
<a href="mailto:Haskell-community@haskell.org" target="_blank">Haskell-community@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community</a><br>
</blockquote></div>