HttpWebRequest and the Expect: 100-continue Header Problem

asp.net, code 0 comments suggest edit

Apparently I’m not the only one to run into this annoying problem. When using the HttpWebRequest to POST form data using HTTP 1.1, it ALWAYS adds the following HTTP header “Expect: 100-Continue”. Fixing the problem has proved to be quite elusive.

According to the HTTP 1.1 protocol, when this header is sent, the form data is not sent with the initial request. Instead, this header is sent to the web server which responds with 100 (Continue) if implemented correctly. However, not all web servers handle this correctly, including the server to which I am attempting to post data. I sniffed the headers that Internet Explorer sends and noticed that it does not send this header, but my code does.

Looking through the newsgroups, several people have had problems with this, but nobody with a solution (apart from going back to HTTP 1.0 which doesn’t work for me).

At this point, I thought I would fire up Lutz Roeder’s Reflector 4.0 and look through the source code for the System.Net.HttpWebRequest class. Aha! There it is. Within a private method named MakeRequest() are the following lines of code:

if (this._ExpectContinue && ((this._HttpWriteMode == HttpWriteMode.Chunked) || (this._ContentLength > ((long) 0))))
{
    this._HttpRequestHeaders.AddInternal("Expect", "100-continue");
}

So even if you try to remove the Expect header from the _HttpRequestHeaders collection, the header will get added back when the request is actually made.

Unfortunately, fixing this is not easy since MakeRequest is a private method. Walking up the callee graph, I found that the method I would have to override is BeginGetRequestStream (there are other ancestors aside from this one). Unfortunately this method relies on several internal and private objects to which I do not have access. I hoped to re-use the existing code base and only make a slight tweak.

I even started down the path of building my own HttpWebRequest class using the Rotor source code but ran into several problems there as well.

In any case, I think the easiest way to get this fixed is to find the right person at Microsoft and ask them very nicely to try to get this in SP2. Pretty Please?

UPDATE: Lance Olson points me to the solution in my comments section. The System.Net.ServicePointManager class has a static property named Expect100Continue. Setting this to false will stop the header “Expect: 100-Continue” from being sent.

Thanks Lance!!!

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

73 responses

  1. Avatar for dk
    dk May 15th, 2004

    dude that happens to me all.....the time.

  2. Avatar for nospamplease75@yahoo.com (Haac
    nospamplease75@yahoo.com (Haac May 15th, 2004

    DK, this post wasn't for you. ;) I need some real answers!

  3. Avatar for Koba
    Koba May 15th, 2004

    Happiness is not getting what you want, it's wanting what you have.

  4. Avatar for Lance Olson
    Lance Olson June 1st, 2004

    You can turn off the HttpWebRequest 100 continue behavior using the System.Net.ServicePointManager.Expect100Continue property.



    -Lance

  5. Avatar for S
    S June 27th, 2004

    Lance, after struggling with .NET SOAP POST'ing through an ISA proxy (which seems to be blocking the 100 Continue response, possibly that's configurable somewhere, but I haven't got access), this was exactly what I needed. Thanks!

  6. Avatar for Michael
    Michael January 19th, 2005

    This worked well for me. Thanks for the help.

  7. Avatar for tonyg
    tonyg February 23rd, 2005

    Sadly, setting a global variable is a real non-solution if you're writing library code. What if the library's client application needs to use 100-continue for something? They're basically completely out of luck. (The situation gets worse if the client is multithreaded.)



    So of the two possible options, (1) set a global variable and (2) downgrade to HTTP/1.0, neither are really very clever ways of doing things.



    Clearly the correct thing is for the .NET libraries to allow setting Expect100Continue for *each* *request* in a thread-safe and library-writer-friendly manner, not using a global variable.

  8. Avatar for Haacked
    Haacked February 23rd, 2005

    That's a very good point TonyG.

  9. Avatar for Miracle
    Miracle March 30th, 2005

    We have the same issue also, glad to see this thread... however we don't know where should we place the code (ServicePointManager.Expect100Continue = false;) in our ASP.NET application.



    Can someone help? Thanks.

  10. Avatar for Haacked
    Haacked March 30th, 2005

    That's a static property, so place the call somewhere that you know will execute before your http request code does. For example you could set it just before your http request. Or you could put it in a static constructor.

  11. Avatar for ben
    ben April 19th, 2005

    thanks mate, exactly our problem too and worked perfectly for us

  12. Avatar for Mirronelli
    Mirronelli June 1st, 2005

    Hi,



    I think I found a bit better solution. Thanks to this thread I started reading all around the internet and have found out this:



    The servicepointmanager is something like a http connection pool. It creates new servicepoints whenever needed and sets their Expect100Continue property based on its own static Expect100Continue value.



    This means if a servicepoint was created before changing servicepointmanagers Expect100Continue property it preserves its original value.



    The point is you can always change the servicepoints Expect100Continue instance property, thus changing the behaviour of each individual request.



    The WebRequest class provides its underlying servicepoint so you can change it right before the request is being made.



    Example:

    // create web request

    HttpWebRequest webRequest = (HttpWebRequest) WebRequest.Create("http://something");

    webRequest.Method = "POST";

    webRequest.ServicePoint.Expect100Continue = false;



    // post data

    requestStream = webRequest.GetRequestStream();

    StreamWriter requestWriter = new StreamWriter(requestStream);

    requestWriter.Write(dataToPost);

    requestWriter.Close();



    //wait for server response

    HttpWebResponse response = (HttpWebResponse) webRequest.GetResponse();

  13. Avatar for IvanX
    IvanX April 2nd, 2006

    Thanks for writing this. I thought it was just me! I was excited to see there was an easy solution. Frustratingly, however, the Expect100Continue property of WebRequest.ServicePoint is absent in .NET Compact Framework, which is what I am working with. I might just have to go with raw sockets for posting if I can't figure something else out...

  14. Avatar for REvol
    REvol May 17th, 2006

    This thread is a great help! I encountered the 417: Expectation Failed problem too, and the description from the message is just that much. There was no way for me to derive the actual solution without the help from this thread.
    Can't they provide a more descriptive message to help pin point the issue?
    Anyway, thanks.

  15. Avatar for aaron
    aaron July 25th, 2006

    didnt help YET but its getting me somewhere, thanks for the post

  16. Avatar for Michael Freidgeim
    Michael Freidgeim September 12th, 2006

    Thanks for the post. It helped me to fight with Squid proxy server(see my post http://geekswithblogs.net/m....
    Also could you change your link Expect100Continue to point to MSDN2(http://msdn2.microsoft.com/....
    MSDN Link Expect100Continue has notice "supported only in version 1.1", which can be interpreted as obsolete.

  17. Avatar for XCRYX
    XCRYX October 2nd, 2006

    Hello guys. I have a site that drives me crazy. I want to make a post in there from a WinApp in C# but everytime I tried, had no luck.
    The website is: http://patrick.net/housing/... but you can see that it redirects you at http://patrick.net/housing/... which requires an email and a password first.
    When you try a request with ie browser is something like this: email=xcrysoft@hotmail.com&pw=xxxxx&submit=Log+In Everytime I post with my app I receive as response the same login.php and no news.php that I want. This are the header from the request from my app:
    Content-Type: application/x-www-form-urlencoded
    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
    Cache-Control: no-cache
    Accept-Encoding: gzip, deflate
    Content-Length: 59
    Connection: Keep-Alive
    Host: patrick.net
    So, I managed to get rid of the 100-continue thing thanks to this thread... but still have this problem. Can anybody help me? PLEASE!!! I can tell you in details if someone wants to help me

  18. Avatar for eeichinger
    eeichinger October 16th, 2006

    Hi,
    There's also an undocumented configuration setting for this:

    <configuration>
    <system.net>
    <settings>
    <servicePointManager
    checkCertificateName="true"
    checkCertificateRevocationList="false"
    useNagleAlgorithm="true"
    expect100Continue="false"
    />

    <httpWebRequest
    maximumResponseHeadersLength="64"
    useUnsafeHeaderParsing="false"
    />
    </settings>
    </system.net>
    </configuration>

  19. Avatar for John Mencias
    John Mencias November 21st, 2006

    You helped me too! Thanks.

  20. Avatar for Pavel
    Pavel November 23rd, 2006

    Very nice describtion, anyways, I've set the property (Expect100Continue) to false and I'm still getting the time out exception...
    Could this be just because of the connection? I mean, that a simple time out occures due to the slow connection?
    Increasing the time out property seems to help...
    Any ideas?

  21. Avatar for Kin
    Kin March 7th, 2007

    Whoah...
    thanks alot for writing this topic...
    its problem nearly make me hang my self... :))
    now my uploader work fine...
    Thanks again!
    Regards
    Kin

  22. Avatar for Chris Martin
    Chris Martin March 16th, 2007

    Holy shit!
    I found this post whilst searching for POST timeout and thought it was a long shot for my problem. Well, it worked for me!
    Thank you so much!!!

  23. Avatar for Latest Newsgroup Posts
    Latest Newsgroup Posts May 8th, 2007

    At [link] , the suggested fix is: [System.Net.ServicePointManage r]::Expect100Continue = $false

  24. Avatar for AJB
    AJB May 15th, 2007

    Has anyone been able to resolve this issue using this fix? I have the exact same problem and it appears as though the fix did nothing.
    Before initializing my httpwebrequest objects, I did:
    ServicePointManager.Expect100Continue = False;
    Then, after my variable declaration, I did:
    myRequest.ServicePoint.Expect100Continue = false;
    Still, it times out on getresponse().
    Ideas?

  25. Avatar for coollil7
    coollil7 May 16th, 2007

    Hi all,
    Thanks for this helpful forum. I've been facing the problem of POST for a long time. Now i have tried disabling the Expect100Continue. But I'm having another error now. It says :
    Server error in '/melatWSite' application: the state information is invalid for this page and might be corrupted.
    melatWSite is my ASP.NET Web application.
    I have a Proxy build in C# and i'm trying to access my own created web application from this developed proxy. When it receives GET messages it works but with POST it does.
    Can anyone help me in this please, cause i'm really stuck.
    Tnks in advance

  26. Avatar for Sunil Rashinkar
    Sunil Rashinkar May 31st, 2007

    I have done the following....but i havent tested it yet...cause i was not sure if this is the right place to do the change...
    A Snip of my exisiting entries in "web.config" file in
    directory "C:\WINNT\Microsoft.NET\Framework\v2.0.50727\CONFIG" for <system.net>
    <system.net>
    <defaultProxy>
    <proxy usesystemdefault="true" />
    </defaultProxy>
    </system.net>
    I updated it with the following in the "web.config" file and it looks like below:
    <system.net>
    <defaultProxy>
    <proxy usesystemdefault="true" />
    </defaultProxy>
    <settings>
    <servicePointManager checkCertificateName="true" checkCertificateRevocationList="false" useNagleAlgorithm="true" expect100Continue="false"/>
    <httpWebRequest maximumResponseHeadersLength="64" useUnsafeHeaderParsing="false"/>
    </settings>
    </system.net>
    Is this correct?
    Sunil Rashinkar

  27. Avatar for Sunil Rashinkar
    Sunil Rashinkar May 31st, 2007

    I have done the following....but i havent tested it yet...cause i was not sure if this is the right place to do the change...
    A Snip of my exisiting entries in "web.config" file in
    directory "C:\WINNT\Microsoft.NET\Framework\v2.0.50727\CONFIG" for <system.net>
    <system.net>
    <defaultProxy>
    <proxy usesystemdefault="true" />
    </defaultProxy>
    </system.net>
    I updated it with the following in the "web.config" file and it looks like below:
    <system.net>
    <defaultProxy>
    <proxy usesystemdefault="true" />
    </defaultProxy>
    <settings>
    <servicePointManager checkCertificateName="true" checkCertificateRevocationList="false" useNagleAlgorithm="true" expect100Continue="false"/>
    <httpWebRequest maximumResponseHeadersLength="64" useUnsafeHeaderParsing="false"/>
    </settings>
    </system.net>
    Is this correct?
    Sunil Rashinkar

  28. Avatar for David Grant
    David Grant June 8th, 2007

    Jesus Christ, thanks for the solution.

  29. Avatar for henk de bakker
    henk de bakker August 21st, 2007

    Thanks, saved me a lot of time.

  30. Avatar for sg
    sg August 22nd, 2007

    System.Net.ServicePointManager.Expect100Continue
    Worked for us too!

  31. Avatar for IBR
    IBR September 15th, 2007

    Thanks man the error 417 error no longer appear, but when i post the login data to the form the form didn't login?????
    HELP PLEASE

  32. Avatar for SC
    SC January 23rd, 2008

    According to the HTTP 1.1. RFC...
    http://www.w3.org/Protocols...
    ... it is not recommended the client wait long before sending the the request body if it does not receive a suitable response (eg. if the server is HTTP 1.0).
    ---------
    Because of the presence of older implementations, the protocol allows ambiguous situations in which a client may send "Expect: 100- continue" without receiving either a 417 (Expectation Failed) status or a 100 (Continue) status. Therefore, when a client sends this header field to an origin server (possibly via a proxy) from which it has never seen a 100 (Continue) status, the client SHOULD NOT wait for an indefinite period before sending the request body.

  33. Avatar for kotopyos
    kotopyos July 1st, 2008

    Thank U very much! Very useful article

  34. Avatar for a5med
    a5med September 6th, 2008

    Thank you all! I had such a problem but now I've coped with it! =)

  35. Avatar for amrhassan
    amrhassan November 9th, 2008

    i just spent a whole day trying to figure out the problem with my code. after firing up wireshark and sniffing the calls, comparing them with the same requests made in python successfully, i found out about this stupid expect header value. after googling the words "httpwebrequest expect continue", i found this blog post. thank you for sharing your rant, you saved my work.

  36. Avatar for stagionedelpane
    stagionedelpane January 1st, 2009

    Thankyou Phil and Lance!
    Your solution solved my problem too!

  37. Avatar for dan
    dan January 8th, 2009

    Wow this post saved me a lot of time, I'm sure. This was one of the first hits I got on Google - sometimes it is your lucky day and sometimes you burn hours. Sorry you had to go through the pain though. I ran into this issue hacking together an automated login for a website where they are using ssl for the auth page.

  38. Avatar for Michael Freidgeim
    Michael Freidgeim January 13th, 2009

    Thanks to Mirronelli I was able to Specify Expect100Continue=false in Web Service client. See my post geekswithblogs.net/.../...-web-service-client.aspx

  39. Avatar for Andreas
    Andreas January 18th, 2009

    Thank you all for the solution and diffent ways how to fix. This problem really made me nervous :-)

  40. Avatar for MT from SD
    MT from SD January 22nd, 2009

    I just spent 8 hours on this error. After comparing the Fiddler2 sniffer requests of the headers when accessing the site using I.E. vs. those headers when accessing the site via my code, I saw that "Expect: 100-continue" header. After doing a Google search, I stumbled accross your blog, implemented "myHttpWebRequest.ServicePoint.Expect100Continue = false;" in my code, and bingo... it now works! Thanks for saving me from another 8 hours of frustration!!!

  41. Avatar for Biju Alapatt
    Biju Alapatt January 26th, 2009

    The problem solved by adding "ignore_expect_100 on" at the end of squid.conf file.

  42. Avatar for Chris Parsons
    Chris Parsons January 10th, 2010

    Reverse proxying via Apache and a client installation (that is proprietary) to an IIS 6.0 server that is also managed and am getting this problem - I have tried to strip the 'Expect' header with Apache config (using 'RequestHeader unset Expect' to no avail and even tried the IIS config changes but still can't get any joy.
    This was caused by an upgrade to the latest version of Apache, but I don't really want to roll it back...

  43. Avatar for Sridhar
    Sridhar January 13th, 2010

    I am having an annoying problem, I am making a https request to www2.swift.com/.../bicdownloader it returns a 401 unauthorized error. I have tried setting setting the authorization header and also set PreAuthenticate to true. I have set most of the settings specified in this site and still not able to solve this problem. The service uses a basic authentication.
    Uri uri = new Uri("www2.swift.com/.../bicdownloader);
    HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(uri);
    try
    {
    System.Net.ServicePointManager.CertificatePolicy = new TrustAllCertificatePolicy();
    CredentialCache cache = new CredentialCache();
    cache.Add(uri, "Basic", new NetworkCredential("me@example.com", "Password1$","www2.swift.com"));
    myReq.Credentials = cache;
    myReq.UnsafeAuthenticatedConnectionSharing = true;
    myReq.PreAuthenticate = true;
    myReq.ServicePoint.Expect100Continue = false;
    myReq.MaximumAutomaticRedirections = 10;
    myReq.Method = "Post";
    myReq.ContentType = "application/x-www-form-urlencoded";
    myReq.KeepAlive = true;
    myReq.Pipelined = true;
    string usernamePassword = "me@example.com" + ":" + "Password1$";
    byte[] authBytes = Encoding.UTF8.GetBytes(usernamePassword.ToCharArray());
    myReq.Headers["Authorization"] = "Basic " + Convert.ToBase64String(authBytes);

    HttpWebResponse wr = (HttpWebResponse)myReq.GetResponse();
    }
    Please help I have already wasted 2 days on this.
    Kind Regards,
    Sridhar

  44. Avatar for Guissepp88
    Guissepp88 January 14th, 2010

    Wasted more than 2 days on this...Thanks dude!!!

  45. Avatar for hobbes
    hobbes January 18th, 2010

    看得不是很清楚:)

  46. Avatar for Gang
    Gang February 3rd, 2010

    good discussion. I met the same question when our SOAP calling program meets Squid proxy.
    I think the solution of disabling ServicePoint.Expect100Continue would done well.
    Besides, I noticed that set ServicePoint.Expect100Continue to false still affects following calls in .net programs, even if I restart debugging another inststance. Who has idea ?

  47. Avatar for Vaibhav
    Vaibhav March 1st, 2010

    Great help thanks

  48. Avatar for Mel B
    Mel B March 21st, 2010

    just set the header with the "Expect: " value set to null/empty.

  49. Avatar for Raji
    Raji March 23rd, 2010

    Thank you for giving me the answer I needed. Really appreciate it.

  50. Avatar for Raj
    Raj April 28th, 2010

    Brilliant! wasted 2 days trying to figure out, Why it won't work..

  51. Avatar for antihacker101
    antihacker101 May 16th, 2010

    just in case its part of my issue. i been fighting the origin of the botnet and creator since aug 2008. they used fake hardware for proxys such as HTTP:proxy, i found a way to turn that off, but also they use NULL as a hardware device. somehow fiddler2.com1 lead me here. any ideas. i do know that all info is intercepted and im blocked from alot of security sites. they alter the hubs and even use udp to send to my phone company which is used as part of the system and a satalite also used linking to asia and http:proxy, and also the firmware/hardware bios and such to overide anything afterwards affecting reformats(even low level).
    any ideas? send to responder@deepandcrazy.com a 10percent chance of getting it.

  52. Avatar for denis
    denis May 21st, 2010

    Thanks a lot ! I was getting mad, trying to post data to a simple web server on a Wd Hd :-)

  53. Avatar for a
    a May 25th, 2010

    Thank you so much for this thread. I would have never figured this out.

  54. Avatar for Jack Walker
    Jack Walker June 7th, 2010

    Thank You Thank You Thank You... This solve a problem that I have been wrestling with for 5 days. Microsoft please update your MSDN examples and explain the importance of setting this object System.Net.ServicePointManager.Expect100Continue on all related articles for HttpWebRequest/Response, WebRequest/Response, WebClient and TCPClient. This information could have saved me 4 days of frustration. This definitely was the 'needle in my haystack' that I had been trying to find to solve my problem.
    Thanks again Phil Haack! Hope the family is doing well.

  55. Avatar for grace
    grace July 31st, 2010

    haizzz very headache abot how to post picture file using httpwebrequest from server to client ...i knw i knw ned to use ContentType = "multipart/form-data;........thnx if any body helping me ...

  56. Avatar for Jamie
    Jamie August 8th, 2010

    Thanks man! This post was exactly what I was looking for. Spent forever just trying to set the "Expect" header and it never stuck!! Thanks again.
    - Jamie

  57. Avatar for Kaly
    Kaly October 14th, 2010

    thanks for the help, had exactly the same issue with error 417 :)

  58. Avatar for Kris Krause
    Kris Krause December 14th, 2010

    Thanks guys. The following code removed the header for me:
    webRequest.ServicePoint.Expect100Continue = false;
    One can verify the raw HTTP using Fiddler (if you want).

  59. Avatar for Darren Thurman
    Darren Thurman April 14th, 2011

    I have tried both System.Net.ServicePointManager.Expect100Continue = false and (HttpWebRequest)myRequest.ServicePoint.Expect100Continue =false, just before myRequest.GetResponse()
    Fiddler still shows Expect: 100-continue in my request header.
    I'm working with .Net 4.0.
    Any thoughts on why setting the property to false is not working?
    Thanks!

  60. Avatar for DeWayne Lane
    DeWayne Lane April 25th, 2011

    Using .Net 4.0
    added the following before my request fixed the issue through a reverse proxy.
    System.Net.ServicePointManager.Expect100Continue = false;
    //make the request
    myClient.createPayment(myOperation);
    Thanks a ton for the post!!!

  61. Avatar for arkadius
    arkadius May 29th, 2011

    Works well when using BoxSync.Net 0.4.1.0 library with a proxy!

  62. Avatar for wiadran
    wiadran June 8th, 2011

    thanks! very nice advice... it did not help with my problem, but at least eliminated one potential fail.

  63. Avatar for Franck
    Franck February 15th, 2013

    Thanks man!

  64. Avatar for Jack
    Jack March 27th, 2013

    Oh my god you just saved my life.

  65. Avatar for bear
    bear June 12th, 2013

    Have anyone here experience with the message "HTTP/1.1 100 Continue" PERIODICALLY not being received in the .net framework?

  66. Avatar for Rafael Nicoletti
    Rafael Nicoletti December 16th, 2013

    This is not needed. When you create a http request, you can use this:

    var uri = new Uri("http://contoso.com");
    var p = ServicePointManager.FindServicePoint(uri);
    p.Expect100Continue = false;
    var r = HttpWebRequest.Create(uri) as HttpWebRequest;

    // do whatever you want with this request.

  67. Avatar for Mike Ober
    Mike Ober February 24th, 2014

    This just eliminated a 301 page has moved response that pointed to the original page. Thanks.

  68. Avatar for Sajjad Hossain
    Sajjad Hossain May 4th, 2014

    Thanks. The information helped us improve the performance of our script.

  69. Avatar for sally
    sally March 6th, 2015

    I have done the following....but i havent tested it yet...cause i was not sure if this is the right place to do the change...
    A Snip of my exisiting entries in "web.config" file in
    directory "C:\WINNT\Microsoft.NET\Framework\v2.0.50727\CONFIG" for <system.net>
    <system.net>
    <defaultproxy>
    <proxy usesystemdefault="true"/>
    </defaultproxy>
    </system.net>
    I updated it with the following in the "web.config" file and it looks like below:
    <system.net>
    <defaultproxy>
    <proxy usesystemdefault="true"/>
    </defaultproxy>
    <settings>
    <servicepointmanager checkcertificatename="true" checkcertificaterevocationlist="false" usenaglealgorithm="true" expect100continue="false"/>
    <httpwebrequest maximumresponseheaderslength="64" useunsafeheaderparsing="false"/>
    </settings>
    </system.net>
    Is this correct?
    Taken from http://www.filmrally.com/

  70. Avatar for ak
    ak March 8th, 2015

    This worked well for me. Thanks for the help.

  71. Avatar for adamo
    adamo April 18th, 2016

    Hi! I have had similar issue. Thanks to Mirronellis comment I figured out that there can be some connections in pool while static is set. I've move setting Expect100Continue = false; code to application start and it's working fine.

  72. Avatar for Carlos Bustamante
    Carlos Bustamante March 31st, 2017

    It Works, THANKS A LOT

  73. Avatar for Kotena L
    Kotena L September 21st, 2017

    I had the same problem and it looks like the property cannot be set multiple times but only once before first request is made. I did not investigate frankly but found that if property is set in web.config or before any other web-request is made then it works.