[Haskell-cafe] Package Takeover: `toml`

Fri Mar 12 13:10:57 UTC 2021

The TOML format is optimized for human-readability, not space efficiency.
And it has some data redundancy, which makes it great for the application
configuration use-case but not so great as a serialization format. If you
are using TOML to serialise data or you need to parse 3-10 GB of
application configuration, there's a chance that something can be improved
without streaming TOML parsing.

So streaming TOML parser looks like a very specific use-case that doesn't
justify taking someone's package and making it the official TOML parser.

Best regards,
Dmitrii

On Fri, 12 Mar 2021 at 12:40, Carter Schonwald <carter.schonwald at gmail.com>
wrote:

> Hey Dmitrii,
> I believe emily has in mind fully incremental streaming support. Which
> requires a wildly different internal architecture than all the stuff in the
> aeson inspired design family
>
> https://github.com/cartazio/streaming-machine-json
> Is an example of a constant space json parser with fully incremental
> consumption and emissions.
>
> An predecessor  was used in a work prject 5 years ago and it could handle
> multi gig json monsters like a champ.  I never released it to hackage
> because I want to have a better / more reusable design.  Parsing the same
> 3-10gb json with aeson was impossible on a machine that had hundreds of
> gigs of ram. :)
>
> I believe emily has in mind similar levels of robustness
>
> On Fri, Mar 12, 2021 at 7:08 AM Dmitrii Kovanikov <kovanikov at gmail.com>
> wrote:
>
>> Hi everyone,
>>
>> I feel extremely sad about this discussion for multiple reasons. But
>> regarding the technical agenda:
>>
>> > I'm going to look at `toml-parser` in the meantime, but no toml library
>> does what I have in mind (namely a full fledged implementation of the spec,
>> streaming, deriving etc.), nor do many of them provide bidirectional
>> serialization save `tomland`.
>>
>> This does sound very disappointing to me and I don't fully understand
>> the needs. Because:
>>
>> * tomland is the official TOML parsing library in Haskell according to
>> the TOML spec wiki <https://github.com/toml-lang/toml/wiki>
>> * tomland fully supports the spec version 0.5.0, and the latest spec
>> 1.0.0 was published relatively recently. And to my knowledge, it is the
>> only Haskell library that supports the latest spec.
>> * tomland is the most popular TOML parsing library according to reverse
>> dependencies <https://packdeps.haskellers.com/reverse> on Hackage
>> * tomland is based on explicit values, but nevertheless it provides
>> deriving via Generics
>>
>> I feel very confused about this situation. And again I feel like people
>> the Haskell committees members are not willing to recognise other's
>> people work and would rather rewrite everything from scratch instead of
>> collaborating with existing projects created by people outside of
>> committees. Even outside the Haskell community (the TOML org), tomland was
>> acknowledged as the official TOML library, but not in the Haskell community
>> itself.
>>
>> At least, the following steps could be taken first:
>>
>> * Why not open issues to tomland (or other libraries) and discuss the
>> features you want? We maintain tomland for multiple years. The latest
>> release was Feb 12 2021 (a month ago!). We constantly improve the
>> implementation, fix parsing issues, improve interface and error-handling.
>> Attempting to rewrite all this from scratch instead of collaborating with
>> existing maintainers feels very unfriendly.
>> * If you want to have the official TOML parsing library under the `toml`
>> namespace on Hackage, again, why not ask the maintainers if they consider
>> moving the library? And only after this discussion act accordingly.
>> * If you are concerned about the lack of people working on the `tomland`
>> library (which I don't fully understand, because in Kowainik we always have
>> at least two people maintaining packages), then why not ask to add as a
>> maintainer, instead of rewriting another library? Or even ask to move to
>> the official `haskell` organization on GitHub, if you want to have more
>> people maintaining the official package.
>>
>> I mean, how am I supposed to feel motivated working on Haskell
>> open-source projects, if my work can be just discarded at any time, the
>> official library will be appointed without even communicating this desire?
>> If I weren't subscribed to this thread, I probably wouldn't even know that
>> something is going behind backs. We've put a lot of effort into tomland. We
>> literally spent years of maintenance, UX improvements, bug fixes, writing
>> tutorials and blog posts about the library and its implementation. And it
>> is still not enough just to be respected and even give the chance to reply
>> to the users needs?
>>
>> That sounds very concerning to me. I don' feel like Haskell tech can move
>> forward if people's (specifically if they are not associated with any
>> Haskell leaders) work is disrespected.
>>
>> Best regards,
>> Dmitrii
>>
>> On Fri, 12 Mar 2021 at 07:02, amindfv--- via Haskell-Cafe <
>> haskell-cafe at haskell.org> wrote:
>>
>>> On Fri, Mar 12, 2021 at 06:28:44AM +0000, Emily Pillmore wrote:
>>> > Tom,
>>> >
>>> > Look, I don't want to debate syntax and semantics here, but "burning
>>> need/desire/ambition" etc (
>>> https://idioms.thefreedictionary.com/burning+desire ) is an extremely
>>> common colloquialism that doesn't imply an emergency, just a strongly felt
>>> urge.  I can't apologize for my wording, but I'm sorry for the situation
>>> nonetheless.
>>>
>>> I also don't want to debate semantics but I can tell you "I have a
>>> burning need" on a Hackage takeover has a different connotation of urgency
>>> than "I have a burning desire to write a toml parsing library and to have
>>> it be named 'toml'". I still feel duped and now condescended to as well. I
>>> do nonetheless appreciate your apology for the situation.
>>>
>>> > >
>>> > > It's now seeming more just like a desire for the package name.
>>> > >
>>> > >
>>> >
>>> > I'm going to look at `toml-parser` in the meantime, but no toml
>>> library does what I have in mind (namely a full fledged implementation of
>>> the spec, streaming, deriving etc.), nor do many of them provide
>>> bidirectional serialization save `tomland`. To reiterate Carter's point,
>>> hackage names are a community resource, and they deserve to be thought
>>> through carefully, so yes, the package name is part of the request. I do
>>> alot of community service to make sure things that take up precious Hackage
>>> real-estate are treated well, which is why `toml` posed an opportunity.
>>>
>>> I'm actually open to the idea of using a simple name like "toml" for a
>>> best-in-class Haskell library, but I'd want to see proof that it is clearly
>>> the best in terms of implementation and adoption. I of course think my
>>> plans for toml parsing are the most wonderful, but if a consensus favorite
>>> package emerges and it's not mine I will step aside.
>>>
>>> >
>>> > To that point, anything you put out is something I am interested in
>>> investing time and effort into making a standard. Do you have any code
>>> currently, or is this a TODO on your list? Going through your hackage
>>> libraries, I see no source repository listings, issue trackers, or even an
>>> email to reach you by.
>>>
>>> I do have code, yes. As mentioned earlier I'm in the middle of a
>>> rewrite. If there's more to discuss maybe we should move this conversation
>>> off-list as it's no longer about a package takeover?
>>>
>>> Tom
>>>
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> To (un)subscribe, modify options or view archives go to:
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>> Only members subscribed via the mailman list are allowed to post.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20210312/175adb88/attachment.html>