[Haskell-cafe] Package Takeover: `toml`

Fri Mar 12 13:22:19 UTC 2021

hey dimtrii, i'm sure theres other pieces, thats just the one i'm spec

robust engineering is for robustness. not for assuming sane users or
nonadversarial inputs

relatedly/on that topic, you should probably consider using the float
parser in the scientific package rather than
https://hackage.haskell.org/package/tomland-1.3.2.0/docs/src/Toml.Parser.Value.html#floatP,
theres some easy to design malicious inputs otherwise (talking about
robustness made me look to see if you were using what for parsing floats! )

On Fri, Mar 12, 2021 at 8:11 AM Dmitrii Kovanikov <kovanikov at gmail.com>
wrote:

> The TOML format is optimized for human-readability, not space efficiency.
> And it has some data redundancy, which makes it great for the application
> configuration use-case but not so great as a serialization format. If you
> are using TOML to serialise data or you need to parse 3-10 GB of
> application configuration, there's a chance that something can be improved
> without streaming TOML parsing.
>
> So streaming TOML parser looks like a very specific use-case that doesn't
> justify taking someone's package and making it the official TOML parser.
>
> Best regards,
> Dmitrii
>
> On Fri, 12 Mar 2021 at 12:40, Carter Schonwald <carter.schonwald at gmail.com>
> wrote:
>
>> Hey Dmitrii,
>> I believe emily has in mind fully incremental streaming support. Which
>> requires a wildly different internal architecture than all the stuff in the
>> aeson inspired design family
>>
>> https://github.com/cartazio/streaming-machine-json
>> Is an example of a constant space json parser with fully incremental
>> consumption and emissions.
>>
>> An predecessor  was used in a work prject 5 years ago and it could handle
>> multi gig json monsters like a champ.  I never released it to hackage
>> because I want to have a better / more reusable design.  Parsing the same
>> 3-10gb json with aeson was impossible on a machine that had hundreds of
>> gigs of ram. :)
>>
>> I believe emily has in mind similar levels of robustness
>>
>> On Fri, Mar 12, 2021 at 7:08 AM Dmitrii Kovanikov <kovanikov at gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I feel extremely sad about this discussion for multiple reasons. But
>>> regarding the technical agenda:
>>>
>>> > I'm going to look at `toml-parser` in the meantime, but no toml
>>> library does what I have in mind (namely a full fledged implementation of
>>> the spec, streaming, deriving etc.), nor do many of them provide
>>> bidirectional serialization save `tomland`.
>>>
>>> This does sound very disappointing to me and I don't fully understand
>>> the needs. Because:
>>>
>>> * tomland is the official TOML parsing library in Haskell according to
>>> the TOML spec wiki <https://github.com/toml-lang/toml/wiki>
>>> * tomland fully supports the spec version 0.5.0, and the latest spec
>>> 1.0.0 was published relatively recently. And to my knowledge, it is the
>>> only Haskell library that supports the latest spec.
>>> * tomland is the most popular TOML parsing library according to reverse
>>> dependencies <https://packdeps.haskellers.com/reverse> on Hackage
>>> * tomland is based on explicit values, but nevertheless it provides
>>> deriving via Generics
>>>
>>> I feel very confused about this situation. And again I feel like people
>>> the Haskell committees members are not willing to recognise other's
>>> people work and would rather rewrite everything from scratch instead of
>>> collaborating with existing projects created by people outside of
>>> committees. Even outside the Haskell community (the TOML org), tomland was
>>> acknowledged as the official TOML library, but not in the Haskell community
>>> itself.
>>>
>>> At least, the following steps could be taken first:
>>>
>>> * Why not open issues to tomland (or other libraries) and discuss the
>>> features you want? We maintain tomland for multiple years. The latest
>>> release was Feb 12 2021 (a month ago!). We constantly improve the
>>> implementation, fix parsing issues, improve interface and error-handling.
>>> Attempting to rewrite all this from scratch instead of collaborating with
>>> existing maintainers feels very unfriendly.
>>> * If you want to have the official TOML parsing library under the `toml`
>>> namespace on Hackage, again, why not ask the maintainers if they consider
>>> moving the library? And only after this discussion act accordingly.
>>> * If you are concerned about the lack of people working on the `tomland`
>>> library (which I don't fully understand, because in Kowainik we always have
>>> at least two people maintaining packages), then why not ask to add as a
>>> maintainer, instead of rewriting another library? Or even ask to move to
>>> the official `haskell` organization on GitHub, if you want to have more
>>> people maintaining the official package.
>>>
>>> I mean, how am I supposed to feel motivated working on Haskell
>>> open-source projects, if my work can be just discarded at any time, the
>>> official library will be appointed without even communicating this desire?
>>> If I weren't subscribed to this thread, I probably wouldn't even know that
>>> something is going behind backs. We've put a lot of effort into tomland. We
>>> literally spent years of maintenance, UX improvements, bug fixes, writing
>>> tutorials and blog posts about the library and its implementation. And it
>>> is still not enough just to be respected and even give the chance to reply
>>> to the users needs?
>>>
>>> That sounds very concerning to me. I don' feel like Haskell tech can
>>> move forward if people's (specifically if they are not associated with any
>>> Haskell leaders) work is disrespected.
>>>
>>> Best regards,
>>> Dmitrii
>>>
>>> On Fri, 12 Mar 2021 at 07:02, amindfv--- via Haskell-Cafe <
>>> haskell-cafe at haskell.org> wrote:
>>>
>>>> On Fri, Mar 12, 2021 at 06:28:44AM +0000, Emily Pillmore wrote:
>>>> > Tom,
>>>> >
>>>> > Look, I don't want to debate syntax and semantics here, but "burning
>>>> need/desire/ambition" etc (
>>>> https://idioms.thefreedictionary.com/burning+desire ) is an extremely
>>>> common colloquialism that doesn't imply an emergency, just a strongly felt
>>>> urge.  I can't apologize for my wording, but I'm sorry for the situation
>>>> nonetheless.
>>>>
>>>> I also don't want to debate semantics but I can tell you "I have a
>>>> burning need" on a Hackage takeover has a different connotation of urgency
>>>> than "I have a burning desire to write a toml parsing library and to have
>>>> it be named 'toml'". I still feel duped and now condescended to as well. I
>>>> do nonetheless appreciate your apology for the situation.
>>>>
>>>> > >
>>>> > > It's now seeming more just like a desire for the package name.
>>>> > >
>>>> > >
>>>> >
>>>> > I'm going to look at `toml-parser` in the meantime, but no toml
>>>> library does what I have in mind (namely a full fledged implementation of
>>>> the spec, streaming, deriving etc.), nor do many of them provide
>>>> bidirectional serialization save `tomland`. To reiterate Carter's point,
>>>> hackage names are a community resource, and they deserve to be thought
>>>> through carefully, so yes, the package name is part of the request. I do
>>>> alot of community service to make sure things that take up precious Hackage
>>>> real-estate are treated well, which is why `toml` posed an opportunity.
>>>>
>>>> I'm actually open to the idea of using a simple name like "toml" for a
>>>> best-in-class Haskell library, but I'd want to see proof that it is clearly
>>>> the best in terms of implementation and adoption. I of course think my
>>>> plans for toml parsing are the most wonderful, but if a consensus favorite
>>>> package emerges and it's not mine I will step aside.
>>>>
>>>> >
>>>> > To that point, anything you put out is something I am interested in
>>>> investing time and effort into making a standard. Do you have any code
>>>> currently, or is this a TODO on your list? Going through your hackage
>>>> libraries, I see no source repository listings, issue trackers, or even an
>>>> email to reach you by.
>>>>
>>>> I do have code, yes. As mentioned earlier I'm in the middle of a
>>>> rewrite. If there's more to discuss maybe we should move this conversation
>>>> off-list as it's no longer about a package takeover?
>>>>
>>>> Tom
>>>>
>>>> _______________________________________________
>>>> Haskell-Cafe mailing list
>>>> To (un)subscribe, modify options or view archives go to:
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>>> Only members subscribed via the mailman list are allowed to post.
>>>
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> To (un)subscribe, modify options or view archives go to:
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>> Only members subscribed via the mailman list are allowed to post.
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20210312/27da053f/attachment.html>