[Haskell-cafe] Re: Threads with high CPU usage

[Duncan Coutts]

On Mon, 2008-12-22 at 08:15 -0600, John Goerzen wrote:
> Duncan Coutts wrote:
> > On Mon, 2008-12-22 at 10:30 +0000, Malcolm Wallace wrote:
> > 
> >> The terminology seems counter-intuitive, but in other other words, a
> >> "safe" call is slower but more flexible, an "unsafe" call is fast and
> >> dangerous.  Therefore it is always OK to convert an "unsafe" declaration
> >> into a "safe" one, but never OK to convert from "safe" to "unsafe"
> >> without looking at what the foreign side actually does.
> > 
> > And in general we would not even bother with considering using "unsafe"
> > for calls that are already expensive. It's only worth considering when
> > the length of the call is always very short.
> > 
> > For example in a database library it might make sense to use 'unsafe' on
> > the data-access functions that extract data from a local query result
> > but we should always use 'safe' on any DB function that might want to
> > talk to the network (eg to get more query results).
> 
> It's difficult to anticipate the needs here.  For instance, some people
> may be using a few very-long-running queries measured in minutes, such
> as the original poster.  Other people, such as web app developers, may
> be issuing literally millions of queries, right after another, where the
> difference matters.

The cost of a safe call is not really that high. In comparison to
something a C function that just extracts a member from a structure it's
high, but compared to executing even a simple SQL query I expect it's
not even measurable.

> I had initially used "unsafe" because of the documented performance
> benefit, plus I certainly am not expecting Sqlite to call back into the
> Haskell runtime.
> 
> It seems to me strange that using "unsafe" instead of "safe" would have
> negative implications for threading.  After all, as Malcolm said above,
> "it is always OK to convert an unsafe declaration into a safe one".  So
> could the compiler be made to be smart enough to do so when it is
> advantageous for threading purposes?

I think it could only do it by turning all "unsafe" functions into
"safe" ones. As I understand it, the ability to switch rts capability is
the more expensive of the properties that "safe" provides (compared to
allowing the function to be re-entrant) because it involves pthread
mutexes.

> What's the best way to make this suitable for both people with many
> queries and those with long-running queries?

If the squlite API distinguishes executing a query (potentially long
running) from extracting data from the result set (always quick) then it
would be possible to mark the former safe and the latter unsafe and
still not impose any great overhead on any user.

Duncan