2013-07-23 19:37:14,015 [11] ERROR TradingSession [(null)] - The operation has timed out, Description: URI: https://testapi.13.com/cancel, Exception: System.Net.WebException: The operation has timed out at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context) at System.Net.HttpWebRequest.GetRequestStream()
This exception message did not appear in our main code base, so where do you begin? We eliminated network latency, resource constraints (the user's server had 4GB RAM instead of the recommended 8GB by the third party), the vendor system, and our main code base; leaving only the API.
After I wrote an aggressive (and very naive) trading bot we managed to replicate the issue consistently. Whilst traversing the .NET API I found a suspicious piece of code:
When a HttpWebRequest (see stack trace) is created, it is provided with a ServicePoint object by the ServicePoiuntManager, which instantiates the ServicePoint with whatever values it currently has.
ServicePointManager.DefaultConnectionLimit = 5 meant no more than five simultaneous web connections could be made, and and all further requests would be queued once that limit is met. The user was making so many web requests that they these requests were being queued up and timing out before they could be executed.
So we have found our problem, now how do we resolve it? Just hard coding a higher number for DefaultConnectionLimit is not the correct solution as you're just mitigating the problem, so making this value configurable is the ultimate goal.
The obvious place to do this is in the constructor as to guarantee that no web requests have been made before the value is set, and also because the HttpInvoker (which HttpWebRequest belongs to) to instantiated in the API constructor. However, when I found our API constructor I saw the following:
Making an API-breaking change is not an option (just adding the property onto one of constructors) is not an option, and there is no way that I am creating a further two instructors to just deal with this optional property. What if we want to make another property configurable next week, do we create another four constructors to deal with all the properties? I think not.
The most obvious solution to me was to use a builder pattern for our API instantiation. Using a builder pattern doesn't feel "right" for an API, especially for a mature one. So we opted for mystery option three (which had been established elsewhere in the internals of our API), and created the following:
By creating a new Options object we were able to avoid adding two new constructors to accommodate the different property options available, didn't break the API, and produced a solution that is expandable for the future.
Hopefully you have found this interesting for both potential contention issues within HttpWebRequest, and some basic API design.
For more information on ServicePointManager and HttpWebRequest have a look at MSDN blogs.
