[Haskell-cafe] Re: Why binding to existing widget toolkits doesn't make any sense

[Antony Courtney]

On Fri, Jan 30, 2009 at 2:21 AM, Evan Laforge <qdunkan at gmail.com> wrote:
>> Conal> As Meister Eckhart said, "Only the hand that erases can write the
>> Conal> true thing."
>>
>> Nicely said...
>>
>> I'm sure you're not the only one desiring to write GUI in "genuinely
>> functional" toolkit, but, being realistic and considering how many people
>> are working on bindings for those "legacy libraries", I doubt we'll see
>> something written from the scratch and usable for "Real World Haskell"
>> soon ;)
>
> I'm vaguely aware that there has been a line of functional-style
> widget libraries, from fudgets down to developments like arrows and
> FRP and the various libraries built on that.  The fact that the
> libraries have continued to be research projects and beget further
> research projects implies that they've had some critical flaws that
> needed further research.  Is there a description somewhere of what the
> critical flaws have been and are, and what the current problems are to
> solve before we can finally have a practical declarative and
> compositional UI library?

Here is a message I sent to Conal and the yampa-users list about a
year ago with my thoughts on this very topic.  Hope this helps.

Kind Regards,

     -Antony

Date: Fri, 22 Feb 2008 09:35:48 -0500
From: "Antony Courtney" <antony.courtney at gmail.com>
To: "Conal Elliott" <conal at conal.net>
Subject: Re: Fruit
Cc: Yampa-Users <yampa-users at cs.yale.edu>

Hi Conal (and Paul and Henrik and everyone else!),

Apologies for the delay in replying.  Paul is quite right about my day
job keeping me quite busy, although I've been enjoying watching the
discussion.

I think there are several reasons that Fruit never really caught on.

First and foremost, Fruit never provided anything close to the full
suite of GUI components found in modern UI toolkits (buttons, sliders,
scrollbars, check-boxes, text fields, combo boxes, split panes, tabs,
menus, toolbars, etc. etc.).  I never got much further than
implementing labels, buttons, text fields and maybe one or two other
components.   This wasn't quite as limiting as one might think as
Fruit subsumed everything that was possible to do in Fran and had a
very powerful vector graphics library underneath (Haven) so one could
use Fruit to implement quite sophisticated interactive animations and
combine them with the library of other GUI components.  But one
couldn't realistically expect to download and install Fruit and just
start using it to write the next great full-fledged GUI app. in
Haskell.

This is related to the second reason Fruit didn't catch on, which is
that Fruit was built directly on an interactive graphics library, not
a standard GUI toolkit.  I think that this was the right first step
for our research goals of giving a rigorous answer to the question of
"What is a formal model of a GUI?".  But from both a pragmatic and
research point of view, the next interesting research problem to solve
would be:  "How can we present the Fruit programming interface (which
offers a clean, simple, pure semantic model of a GUI), but implement
the individual GUI components using a standard GUI toolkit?".  This is
actually an interesting implementation challenge since Yampa's
switching and dynamics collections mean that absolutely anything can
happen to the widget hierarchy from one time step to the next.  You
really, really don't want to destroy and rebuild the entire widget
hierarchy of the application at every time step, so figuring out how
to do some kind of sample-to-sample differencing of the widget
hierarchy would be essential.  It's not obvious how to do this, which
is probably why FranTk and wxFruit took a bit more of a pragmatic
approach and exposed bits of the imperative programming model of the
underlying toolkit for widget creation and destruction.

Another important point is that even the bits of fruit that were
implemented involved a number of different dependent parts, so it was
always a bit more work to download and install than I would have
liked.  I was hell-bent on using a vector graphics library as the
foundation. Unfortunately there was no widely-available cross-platform
vector graphics library implemented in C.  So I chose to use Java2D,
which meant that I had to build my own system for Haskell/Java
interoperability (GCJNI, later hsjni) and a Haskell vector graphics
library (Haven) on top of that.  So if one wanted to play with fruit,
it was necessary to download and install Java, gcjni, haven, yampa and
fruit.  And in the early days one also had to grab greencard and the
arrows preprocessor!  I think we did a reasonable job of packaging
these given the severe limits of Haskell packaging and build tools at
the time, but it was a ton of work and still never made it easy enough
for others to experiment casually.

Apart from the practicalities of the implementation, I also think
there are legitimate pragmatic concerns about using the FRP model to
write a full-fledged application.  The benefit of using FRP is that it
forces the programmer to be explicit about input and output
connections, which enables precise reasoning about exactly what a
particular signal function may or may not do.  But there is an
inevitable tension between reasoning power and easy extensibility.
If, for example, we wanted to add a "File...Open" menu item to a Fruit
application that didn't have this originally, we would have to very
explicitly thread signals that provides access to the filesystem
through the entire application to make it available in the menu.  This
is always going to be more work than in a conventional imperative
toolkit in which every component always always has implicit access to
the World.  A related point here is that if you look closely at the
dynamic collections work (pSwitch and friends), you will see that they
operate on collections of signal functions with homogenous types of
input and output signals.  This means that a collection of signal
functions that have heterogenous input and output signals will all
have to be "widened" to some input and output type that is the union
of all the individual signal types in the collection, and then the
individual members of the collection will just ignore the signals they
don't need.  Taken to it's logical extreme we end up with a model
where we have a time-varying collection of SF World World, which seems
disturbingly close to the overly-general semantic model we were trying
to escape from in the original Fruit paper.

But I agree that Fruit does offer some nice practical benefits as a
programming model, and I think it would be great to focus on fruit's
ability to provide interesting and novel animated vector graphics UIs
in a modular way.  Have you played with an iPhone?  It is quite simply
one of the best UIs I have ever used, precisely because of the
pervasive use of vector graphics (spatially continuous zooming) and
pervasive, uniform and subtle animation effects.  See:
      http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=00036T=1
for a video of the iPhone interface and the veritable Ed Tufte
offering some critique and design suggestions.

I think it would be quite interesting and relatively straightforward
to express a number of the standardized UI effects from the iPhone as
a library of GUI transformers in Fruit.  I also suggest checking out
Piccolo, a 2-D structured graphics framework for .Net and Java:
    http://www.cs.umd.edu/hcil/jazz/
Piccolo is based on the idea of using a scenegraph for 2-D graphics,
and is able to leverage that to provides a nice standard library of
animations and effects.  Again, I think many of the ideas would fit
very naturally into a Fruit-like programming model.

One issue I think you may encounter, though, is that last time I
looked, there was still no high-quality, widely available
cross-platform 2-D vector graphics library in C or C++!  So that means
either building on something like Java2D like I did and ending up with
lots of moving parts and a fat runtime (an entire JVM), giving up on
cross-platform compatibility (and losing important Linux-only folks
like Henrik!), or rolling up your sleeves and writing your own vector
graphics library in C before you even start (a lot of work!).  But
maybe something has changed on this front since I was actively working
in the area.

Hope that helps.  Good luck with making another foray into purely
functional GUIs.  Please keep me and the rest of the list informed of
your progress!

Kind Regards,

       -Antony

P.S. Paul:  I have not sold my soul to Wall Street....I'm merely
renting it out for a few years. ;-)

P.P.S.: A sincere "Thank You" to George and others who have been
working to make Yampa easily available using modern Haskell packaging
tools!

On Thu, Feb 21, 2008 at 10:06 PM, Conal Elliott <conal at conal.net> wrote:
> Here's a possible answer to my own question.  By clear intention, Fruit gave
> a full semantic model for the GUI and the GUI-wrapped behavior.  That model
> includes GUI input (mouse & keyboard) and output (imagery).  For this
> reason, we didn't punt the graphics to a semantically inscrutable GUI
> toolkit.  As a result, Fruit could do some pretty neat, non-standard stuff,
> such as continuously zoomable UIs and replicated views (passive and/or
> active) -- all without app-specific effort.  Also as a result, however, the
> visual appearance was not up to par with standard GUI toolkits.
>
> This perspective leads me to an idea.  Maybe a way to breathe life into
> Fruit is to focus on its strength rather than its weakness.  Target
> radically new kinds of GUIs, rather than ones with conventional appearance
> and limitations.
>
> Cheers,  - Conal
>
>
>
>
> On Tue, Feb 19, 2008 at 10:01 PM, Conal Elliott <conal at conal.net> wrote:
> > Antony & others,
> >
> > I've been playing with FRP ideas again, including GUIs.  Some of the
> thought trails lead me back to Fruit.  I really like that model, with its
> clear & simple functional semantics.
> >
> > So, I'm wondering: why didn't Fruit catch on for GUI programming in
> Haskell?
> >
> > Any thoughts?
> >
> > Cheers,  - Conal
> >
>
>