[Haskell-cafe] Strange encoding issue with Text and Servant
David Turner
dct25-561bs at mythic-beasts.com
Sat Jan 14 20:00:21 UTC 2017
Ok, so there's definitely a difference of some kind client-side. It'd be
useful to see the network traffic nonetheless, as the source of the
'!D(MISSING)' is still a bit of a mystery and you may see something
suggestive in the raw bytes.
I can't find the string 'MISSING' anywhere in servant or its dependencies,
even looking in libs and binaries. Any chance of tracking its source down
using `find` and `grep` and `strings`?
On 14 Jan 2017 19:48, "Jan von Löwenstein" <jan.loewenstein at gmail.com>
wrote:
Actually the code I pointed at _is_ the swagger-codegen generated client.
I have got a very small wrapper around it:
```
listService :: (MonadIO m, MonadCatch m, MonadReader Config m, MonadLog
Text m) =>
Text
-> m ServiceList.ServiceList
listService labelSelector = namespacedF $ \namespace ->
Kube.listNamespacedService namespace Nothing (Just labelSelector) Nothing
Nothing Nothing Nothing
listSecret :: (MonadIO m, MonadThrow m, MonadReader Config m, MonadLog Text
m) =>
Text
-> m SecretList.SecretList
listSecret labelSelector = namespacedF $ \namespace ->
Kube.listNamespacedSecret namespace Nothing (Just labelSelector) Nothing
Nothing Nothing Nothing
namespacedF :: (MonadIO m, MonadThrow m, MonadReader Config m, MonadLog
Text m) =>
NamespacedF model
-> m model
namespacedF f = do
config <- ask
result <- let
baseUrl = apiEndpoint config
tlsSettings = Base.tlsSettings baseUrl (credentials config)
kubeNamespace = namespace config
in do
manager <- liftIO $ newManager $ (mkManagerSettings tlsSettings
Nothing)
{
managerModifyRequest = \req ->
return req {
checkResponse = \req res -> do
Text.IO.hPutStrLn stderr
(Text.pack (show req))
responseMessage <-
showResponse res
Text.IO.hPutStrLn stderr
responseMessage
return ()
}
}
liftIO $ runExceptT $ f kubeNamespace manager baseUrl
either throwM return result
```
I call them like:
```
do
let Just agentId = ...
selector = "agentId" <> "=" <> agentId
logMessage $ "listSecret selector: '" <> selector <> "'"
secretList <- listSecret selector
```
`logMessage` gives a literal equals sign in both cases.
`Text.IO.hPutStrLn stderr (Text.pack (show req))` from namespacedF gives
`%3D` and `%!D(MISSING)` depending on which function I call.
>From what I understand this is definitely happening on the client side. Do
you still think a tcpdump might be worth it?
```
listSecret selector: 'agentId=b49d4406-8020-44e6-7788-6f92d5c0e732'
Request {
host = "192.168.99.100"
port = 8443
secure = True
requestHeaders = [("Accept","application/json")]
path = "/api/v1/namespaces/default/secrets"
queryString = "?labelSelector=agentId%!D(
MISSING)b49d4406-8020-44e6-7788-6f92d5c0e732"
method = "GET"
proxy = Nothing
rawBody = False
redirectCount = 10
responseTimeout = ResponseTimeoutDefault
requestVersion = HTTP/1.1
}
```
```
listService selector: 'agentId=24f99a4b-1682-44da-7ba4-dba935d107d2'
Request {
host = "192.168.99.100"
port = 8443
secure = True
requestHeaders = [("Accept","application/json")]
path = "/api/v1/namespaces/default/services"
queryString = "?labelSelector=agentId%3D24f99a4b-1682-44da-7ba4-
dba935d107d2"
method = "GET"
proxy = Nothing
rawBody = False
redirectCount = 10
responseTimeout = ResponseTimeoutDefault
requestVersion = HTTP/1.1
}
```
Thanks for the time you already invested. I am completely puzzled because I
see no chance how the code could behave as it does. (From my experience
that points out I am looking at the wrong place ;) )
Best
Jan
David Turner <dct25-561bs at mythic-beasts.com> schrieb am Sa., 14. Jan. 2017
um 20:08 Uhr:
> Grasping at straws a little bit here, but can you (a) do a packet capture
> (e.g. `tcpdump -X`) to see what's going back and forth on the wire, just to
> make absolutely sure all of the oddness is on the client's end?
>
> A string like `!D(MISSING)` sorta looks like a decoding error so perhaps
> it will be instructive to look at the bytes on the wire it's trying to
> decode, although I can't think what could be confused with an equals sign.
> It's not like that blooming Greek question mark (
> http://www.fileformat.info/info/unicode/char/037e/index.htm).
>
> Cheers,
>
>
>
>
> On 14 January 2017 at 18:47, Nigel Rantor <wiggly at wiggly.org> wrote:
>
> On 14/01/17 17:37, Jan von Löwenstein wrote:
>
> Oh, it happens with servant on the client side.
>
> I use a swagger-codegen generated servant based client to access
> kubernetes.
> Calling https://github.com/soundcloud/haskell-kubernetes/blob/
> master/lib/Kubernetes/Api/ApivApi.hs#L475 works
> Calling https://github.com/soundcloud/haskell-kubernetes/blob/
> master/lib/Kubernetes/Api/ApivApi.hs#L473 doesn't.
>
> From what I see the `QueryParam"labelSelector" Text`is the same in both.
>
> Logging the value gives the literal `=` as expected. Logging the
> (http-client) requests shows the difference.
>
>
> So I went ahead and wrote a simple client by hand and when I call the two
> functions on a local endpoint that shows me the request URL I get the
> following.
>
> Same arguments for 'pretty' and 'labelSelector' in both cases.
>
> Code:
>
> runListSecret :: Manager -> BaseUrl -> ClientM SecretList
>
> runListSecret manager baseUrl = listSecret (Just "prettyArg") (Just "=")
> Nothing Nothing Nothing Nothing manager baseUrl
>
>
>
> runListService :: Manager -> BaseUrl -> ClientM ServiceList
>
> runListService manager baseUrl = listService (Just "prettyArg") (Just "=")
> Nothing Nothing Nothing Nothing manager baseUrl
>
> Server log:
>
> 127.0.0.1 - - [14/Jan/2017:18:39:49 +0000] "GET /api/v1/secrets?pretty=prettyArg&labelSelector=%3D
> HTTP/1.1" 404 - 0.0003
> 127.0.0.1 - - [14/Jan/2017:18:39:49 +0000] "GET /api/v1/services?pretty=prettyArg&labelSelector=%3D
> HTTP/1.1" 404 - 0.0003
>
> I haven't used swagger-codegen but the servant-client code works for me so
> I would look more into that.
>
> Swagger codegen looks like it might take me a while to get my head into so
> maybe you could look at the generated code, or sling it to me here or
> directly.
>
> The two calls are so similar that I can't see why the generated client
> would behave differently in those cases.
>
> Regards,
>
> n
>
> Best
> Jan
>
> Nigel Rantor <wiggly at wiggly.org <mailto:wiggly at wiggly.org>> schrieb am
>
> Sa., 14. Jan. 2017 um 18:11 Uhr:
>
> On 13/01/17 19:28, Jan von Löwenstein wrote:
> > Hi,
> >
> > I have got a two places with a `QueryParam "q" Text` and call it
> with a
> > Text that contains a `=` literal. In one place the `=` is correctly
> > encoded as %3D, in the other I see a `!D(MISSING)`.
> >
> > This has to happen somewhere in Servant or the lower layers. Both
> Texts
> > print out nicely with an `=` sign if I just print them to stdout.
> >
> > Google does not find `!D(MISSING)` anywhere.
> >
> > Any idea what could possibly be the problem here?
>
> It would be really useful to see exactly how you are 'calling' this.
>
> Do you have a curl command line or are you using some other client?
>
> n
>
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20170114/903aab75/attachment.html>
More information about the Haskell-Cafe
mailing list