[Haskell-beginners] Nonsensical behavior with http-conduit, httpLbs

[Michael Snoyman]

http-conduit does not perform any kind of automatic concurrency. httpLbs
will only return after it has fully read the response body from the server.
So unless there's a bug somewhere in http-conduit, it seems that the server
has not fully updated its index by the time it gives back a response.


On Thu, Apr 10, 2014 at 9:25 AM, Christopher Allen <cma at bitemyapp.com>wrote:

> the function actually running the requests is:
>
> dispatch :: String -> Method -> Maybe L.ByteString
>             -> IO Reply
> dispatch url method body = do
>   initReq <- parseUrl url
>   let reqBody = RequestBodyLBS $ fromMaybe emptyBody body
>   let req = initReq { method = method
>                     , requestBody = reqBody
>                     , checkStatus = \_ _ _ -> Nothing}
>   withManager $ httpLbs req
>
> rest of the code is here: https://github.com/bitemyapp/bloodhound/
>
>
> On Thu, Apr 10, 2014 at 1:25 AM, Christopher Allen <cma at bitemyapp.com>wrote:
>
>> It's a step removed from that, and `seq`'ing insertData isn't fixing it,
>> only a threadDelay is.
>>
>> The operations insertData is performing are:
>> delete index, create index, add document to index
>>
>> the operation queryTweet is performing is:
>> search index
>>
>> The behavior seems to indicate "search index" is happening before "add
>> document to index" and after "delete index", as the document will not exist
>> when the query is performed.
>>
>> Only adding a threadDelay as reliably worked. Even `seq`'ing insertData
>> doesn't work.
>>
>> Are the requests somehow being performed concurrently?
>>
>>
>>
>> On Thu, Apr 10, 2014 at 1:15 AM, Michael Snoyman <michael at snoyman.com>wrote:
>>
>>> I can't say I fully understand your code, but it seems like it's doing
>>> the following:
>>>
>>> 1. Send request to server to delete data.
>>> 2. Send request to server to add data.
>>> 3. Request data from server.
>>>
>>> You're saying that with a long enough delay between steps 2 and 3, then
>>> (3) works, otherwise it fails. It sounds to me like there's some kind of a
>>> race condition. I don't know anything about createExampleIndex, but are you
>>> certain that it only returns after the data is completely ready for
>>> querying?
>>>
>>>
>>> On Thu, Apr 10, 2014 at 8:43 AM, Christopher Allen <cma at bitemyapp.com>wrote:
>>>
>>>> to add to my question adding a delay made it work:
>>>>
>>>> main :: IO ()
>>>> main = do
>>>>   _ <- insertData
>>>>   threadDelay 1000000
>>>>   myTweet <- queryTweet
>>>>   print myTweet
>>>>
>>>>
>>>>
>>>> On Thu, Apr 10, 2014 at 12:41 AM, Christopher Allen <cma at bitemyapp.com>wrote:
>>>>
>>>>> Repro'd on GHC 7.8.1 and 7.6.3
>>>>>
>>>>> Original code is: https://github.com/bitemyapp/bloodhound/
>>>>>
>>>>> The code only works if I manually (out of band) run insertData (in a
>>>>> REPL) and then run main with the insertData invocation stripped out. If I
>>>>> run main with the insertData invocation included, it throws an exception
>>>>> (head) because the search results are empty.
>>>>>
>>>>> The behavior is as if queryTweet was executing after insertData
>>>>> deleted the index, but before it inserted the new data.
>>>>>
>>>>> The following is a stripped down example:
>>>>>
>>>>> insertData :: IO ()
>>>>> insertData = do
>>>>>   let encoded = encode exampleTweet
>>>>>   _ <- deleteExampleIndex
>>>>>   created <- createExampleIndex
>>>>>   docCreated <- indexDocument (Server "http://localhost:9200")
>>>>> "twitter" "tweet" exampleTweet "1"
>>>>>   print "test"
>>>>>   return ()
>>>>> el
>>>>> queryTweet :: IO (Either String Tweet)
>>>>> queryTweet = do
>>>>>   let queryFilter = BoolFilter (MustMatch (Term "user" "bitemyapp")
>>>>> False)
>>>>>                     <||> IdentityFilter
>>>>>   let search = Search Nothing (Just queryFilter)
>>>>>   reply <- searchByIndex testServer "twitter" search
>>>>>   let result = eitherDecode (responseBody reply) :: Either String
>>>>> (SearchResult Tweet)
>>>>>   let myTweet = fmap (hitSource . head . hits . searchHits) result
>>>>>   return myTweet
>>>>>
>>>>> main :: IO ()
>>>>> main = do
>>>>>   _ <- insertData
>>>>>   myTweet <- queryTweet
>>>>>   print myTweet
>>>>>
>>>>> further up the call chain, the http call is getting dispatched with
>>>>> http-conduit's gear, the result returned with the expression:
>>>>>
>>>>> withManager $ httpLbs req
>>>>>
>>>>> Included this in case it affects the semantics.
>>>>>
>>>>> Can anybody help? This has stumped me for a couple hours and I
>>>>> couldn't get anything clueful on IRC.
>>>>>
>>>>> Thanks to any that help.
>>>>>
>>>>> --- Chris
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Beginners mailing list
>>>> Beginners at haskell.org
>>>> http://www.haskell.org/mailman/listinfo/beginners
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Beginners mailing list
>>> Beginners at haskell.org
>>> http://www.haskell.org/mailman/listinfo/beginners
>>>
>>>
>>
>
> _______________________________________________
> Beginners mailing list
> Beginners at haskell.org
> http://www.haskell.org/mailman/listinfo/beginners
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/beginners/attachments/20140410/7765f946/attachment-0001.html>