[Haskell-cafe] Instrumentation without requiring code-level changes?

Fri Jan 27 08:17:09 UTC 2017

On 27 Jan 2017 07:16, "Saurabh Nanda" <saurabhnanda at gmail.com> wrote:

(copying the mailing list -- I don't think your intention was to keep this
off the mailing list)

Apologies, yes, I failed to CC the mailing list. Thanks. Double checked
that this one is addressed properly.

I've looked at middleware for Scotty (or any WAI application), but it has
limitations. I want deeper insights about how much time is being spent
where, when serving a web request -- which is why the need to instrument
SQL, Redis, and HTML rendering.

One example that doesn't necessarily need to be in the IO monad is HTML
rendering using Lucid, where the data is being passed separately to it
(standard Controller <> View separation).

Ok, this is definitely something that's hard to define, let alone measure:
Lucid's rendering uses Bytestring.Builder to interleave the rendering and
the transmission of traffic over the network, so a simple
endTime-minus-startTime calculation doesn't work. You'd have to do
something within ByteString.Builder itself to count the time spent filling
each buffer vs the time spent sending it. I don't know of an existing
implementation, but I can see how you'd do it (and it's worth noting that
the relevant code is in IO, hidden behind a call to `unsafePerformIO` or
similar):

https://hackage.haskell.org/package/bytestring-0.10.8.1/docs/Data-ByteString-Builder-Extra.html#v:runBuilder

Also, if everything needs to be run in MonadIO, what does that do to
fine-grained effect tracking? I wanted to have HasDb, HasRedis, HasEmail
kind of monads (or type-classes) that give me some guarantees about the
**type** of IO interactions a particular function call can do. If I
instrument them and add an additional MonadIO constraint, does it mean any
IO can be lifted into such monads?

Your type classes can avoid mentioning IO, it's the instance definitions
that would want to contain the instrumentation code. But they need IO
anyway so that's no big deal.

It's unfortunate to know that the RTS doesn't have these kind of
instrumentation capabilities built-in. Is it absolutely not possible to do
this at an RTS level?

Ah, actually, there's the RTS event log:
https://ghc.haskell.org/trac/ghc/wiki/EventLog and
https://hackage.haskell.org/package/base-4.9.1.0/docs/Debug-Trace.html#g:2

But it emits a _lot_ of events from the RTS if you switch it on.

Cheers,

David

-- Saurabh.

On Fri, Jan 27, 2017 at 12:32 PM, David Turner <
dct25-561bs at mythic-beasts.com> wrote:

> Hi Saurabh,
>
> There's no direct equivalent to monkey-patching in Haskell (AFAIK (*)).
> Why not instrument things always? The performance impact is usually quite
> negligible, and the ability to switch instrumentation on at runtime in
> production is sometimes invaluable.
>
> For instrumenting Scotty (in particular) or WAI-based apps (in general)
> you can instrument things simply by inserting an appropriate `Middleware`,
> a decision that can be deferred until service-startup time, and which
> involves no change to the Scotty-side code.
>
> There's no equivalent to WAI for database access so I don't have a general
> answer in that case, but typically database APIs have only a few core
> `execute` or `query` functions that are quite straightforward to wrap with
> instrumentation if needs be.
>
> This all kinda _has_ to be in IO anyway, as you're trying to measure a
> real-world activity. Certainly there's no problem with either Scotty or
> databases in this regard, as they're running in IO. Do you have an example
> where you need instrumentation of a pure (i.e. non-IO) computation? In such
> a case you could use the `unsafePerformIO` escape hatch (like `Debug.Trace`
> does) but you'll have to be very careful to measure what you think you're
> measuring: the language is quite free to reorder computations and avoid
> repeating duplicated computations via referential transparency etc.
>
> (*) I mean, you could do something with dynamic libs and LD_PRELOAD (the
> original monkey-patching) but please don't!
>
> Hope that helps,
>
> David
>
> On 27 Jan 2017 05:21, "Saurabh Nanda" <saurabhnanda at gmail.com> wrote:
>
>> (another cross post from Reddit - https://www.reddit.com/r/haske
>> ll/comments/5qfd36/instrumentation_without_required_application/)
>>
>> Is it possible to collect instrumentation data from a Hakell application
>> without requiring the core application to change code? Here are some
>> examples of instrumentation data:
>>
>> * Time taken to serve an HTTP request (not average time, but individual
>> times, for individual requests) along with the incoming request
>> params+headers and outgoing response length.
>> * Time taken to make an SQL query along with the actual query (again, not
>> average times, but individual times, for individual queries)
>> * Time taken to fetch data from Redis along with the Redis command
>> * Time taken to render an HTML page along with path to the HTML file
>>
>> Most dynamic languages (like Ruby) allow monkey-patching OR "decorating"
>> existing functions during runtime. Most instrumentation agents (like
>> Skylight, Newrelic), simply monkey-patch the API surfaces of known
>> libraries to collect this data. Therefore, using these instrumentation
>> agents is a one line code change (just include the gem, or drop the JAR, or
>> whatever).
>>
>> What is the equivalent of this in Haskell?
>>
>> Here's what I've tried so far [1] , but it has the following
>> disadvantages:
>>
>> * Requires changes to the application's cabal file to replace the
>> non-instrumented versions of the libraries with the instrumented version
>> (replace scotty with instrumentedscotty, in my example)
>> * Requires the application to explicitly import the instrumented versions
>> of the libraries (replace import Scotty with import InstrumentedScotty, in
>> my example)
>> * Forces all the code interacting with instrumented APIs to be run in a
>> MonadIO environment - because of the underlying calls to getCurrentTime
>>
>> I'm sure there's a better way, but I couldn't get my hands on it!
>>
>> [1] https://gist.github.com/saurabhnanda/27592da0269bc35569ec6239e1a91b75
>>
>> -- Saurabh.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>>
>

-- 
http://www.saurabhnanda.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20170127/3fa514b6/attachment-0001.html>