Only one usage of each socket address (protocol/network address/port) is normally permitted - azure-cognitive-search

Question

The last few weeks we have been experiencing this error message while using the Azure Search SDK (1.1.1 - 1.1.2) and performing searches.
We consume the Search SDK from internal APIs (deployed as Azure Web Apps) that scale up-down based on traffic (so there could be more than 1 instance of the APIs doing the searches).
Our API queries 5 different indexes and maintains an in-memory copy of the SearchIndexClient object that corresponds to each index, a very simple implementation would look like:
public class AzureSearchService
{
private readonly SearchServiceClient _serviceClient;
private Dictionary<string, SearchIndexClient> _clientDictionary;
public AzureSearchService()
{
_serviceClient = new SearchServiceClient("myservicename", new SearchCredentials("myservicekey"));
_clientDictionary = new Dictionary<string, SearchIndexClient>();
}
public SearchIndexClient GetClient(string indexName)
{
try
{
if (!_clientDictionary.ContainsKey(indexName))
{
_clientDictionary.Add(indexName, _serviceClient.Indexes.GetClient(indexName));
}
return _clientDictionary[indexName];
}
catch
{
return null;
}
}
public async Task<SearchResults> SearchIndex(SearchIndexClient client, string text)
{
var parameters = new SearchParameters();
parameters.Top = 10;
parameters.IncludeTotalResultCount = true;
var response = await client.Documents.SearchWithHttpMessagesAsync(text, parameters, null, null);
return response.Body;
}
}
And the API would invoke the service by:
public class SearchController : ApiController
{
private readonly AzureSearchService service;
public SearchController()
{
service = new AzureSearchService();
}
public async Task<HttpResponseMessage> Post(string indexName, [FromBody] string text)
{
var indexClient = service.GetClient(indexName);
var results = await service.SearchIndex(indexClient, text);
return Request.CreateResponse(HttpStatusCode.OK, results, Configuration.Formatters.JsonFormatter);
}
}
We are using SearchWithHttpMessagesAsync due to a requirement to receive custom HTTP headers instead of the SearchAsync method.
This way we avoid opening/closing the client under traffic bursts. Before using this memory cache (and wrapping each client on a using clause) we would get port exhaustion alerts on Azure App Services.
Is this a good pattern? Could we be receiving this error because of the multiple instances running in parallel?
In case it is needed, the stack trace shows:
System.Net.Http.HttpRequestException: Only one usage of each socket address (protocol/network address/port) is normally permitted service.ip.address.hidden:443
[SocketException:Only one usage of each socket address (protocol/network address/port)is normally permitted service.ip.address.hidden:443]
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure,Socket s4,Socket s6,Socket& socket,IPAddress& address,ConnectSocketState state,IAsyncResult asyncResult,Exception& exception)
[WebException:Unable to connect to the remote server]
at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult,TransportContext& context)
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
EDIT: We are also receiving this error A connection attempt failed because the connected party did not properly respond after a period of time:
System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond service.ip.address.hidden:443
[SocketException:A connection attempt failed because the connected party did not properly respond after a period of time,or established connection failed because connected host has failed to respond service.ip.address.hidden:443]
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure,Socket s4,Socket s6,Socket& socket,IPAddress& address,ConnectSocketState state,IAsyncResult asyncResult,Exception& exception)
[WebException:Unable to connect to the remote server]
at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult,TransportContext& context)
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)

Answer 1

As implemented in the code in your question, the cache will not prevent port exhaustion. This is because you're instantiating it as a field of the ApiController, which is created once per request. If you want to avoid port exhaustion, the cache must be shared across all requests. To make it concurrency-safe, you should use something like ConcurrentDictionary instead of Dictionary.
The "connection attempt failed" error is likely unrelated.