If you don’t want to read my rant below: use HttpChunkinModule
In my case i used a Readbility API to parse news articles’ content. The API receives requests and POST it back to URL specified in the request.
However no matter how hard i tried, the API’s log always told me there was error. I POST’ed many requests manually to test it myself; my server received and processed all of them, but not the API’s ?
So I turned on unicorn‘s log to see what’s wrong. Well, there was absolutely no hit from any IP except from my computer. Strange, it means API’s requests must be blocked from higher (lower?) level, either nginx or iptables. Firewall is unlikely the cause since it’s normal HTTP, it must be nginx:
129.206.*.* - - [20/Nov/2010:15:37:08 -0800] "POST / HTTP/1.1" 200 106 "-" "Ruby" "-"
77.234.*.* - - [20/Nov/2010:15:37:45 -0800] "POST / HTTP/1.1" 411 174 "-" "-" "-"
- client sent invalid “Content-Length” header
- client sent … method without “Content-Length” header
- client sent “Transfer-Encoding: chunked” header
The last reason is nginx feature: it still does not support chunked body.
Oh, it’s not a bug, it’s a feature ! I couldn’t stop swearing, this “feature” takes me almost one day to find out. And Google shows a lot people who are like me.
nginx’s error log (you must turn debug on):
2010/11/20 18:19:05 [info] 16755#0: *80135 client sent "Transfer-Encoding: chunked" header while reading client request headers, client: 22.214.171.124, server: lab.lamnk.com, request: "POST / HTTP/1.1", host: "lab.lamnk.com"
Once you know causes of the problem, then it’s (supposed to be) easy to fix it: recompile nginx with HttpChunkinModule, added configuration directives in nginx.conf and we’re good !
PS: kudos to the guys at Preona for the nice API