Throttling Webservice Requests

Julio Bello 221 Reputation points
2022-10-14T13:12:59.75+00:00

Hi, Everybody...

I have an "internal" API that tracks packages (plural). This "internal" API accepts a collection of tracking numbers, iterates through the collection and invokes the "external" (i.e., third-party vendor) track package (singular) API. This effectively "blasts" a number of requests at the "external" (i.e., third-party vendor) track package (singular) API. Apparently, the vendor API cannot handle multiple concurrent requests. I need to somehow "throttle" the requests one at a time (i.e., invoke the "external" (i.e., third-party vendor) track package (singular) API, wait for the response, process it and then invoke the same "external" (i.e., third-party vendor) track package (singular) API with the next tracking number, until there are no more tracking numbers).

I imagine instantiating a singleton and have it instantiate a queue upon starting the webservice. The singleton and the queue will persist until the webservice is shutdown. The "internal" API that tracks packages (plural) will simply enqueue the tracking numbers. A worker thread would dequeue a tracking number, invoke the "external" (i.e., third-party vendor) track package (singular) API, wait for the response, process it and repeat until all the tracking numbers have been dequeued.

Also, is it possible to make a queue "eventful" (i.e., raise an event when the queue count becomes non-zero and another event when the queue count becomes zero)? Thus, the former event would create a worker thread and the latter event would release/dispose the worker thread.

Also, will I need to perform some housekeeping when the webservice is shutdown?

Is this a viable approach?... Or is there a better way?

Note: I viewed the following. None were applicable specifically to my situation.

.NET
.NET
Microsoft Technologies based on the .NET software framework.
3,877 questions
C#
C#
An object-oriented and type-safe programming language that has its roots in the C family of languages and includes support for component-oriented programming.
10,962 questions
.NET Runtime
.NET Runtime
.NET: Microsoft Technologies based on the .NET software framework.Runtime: An environment required to run apps that aren't compiled to machine language.
1,163 questions
ASP.NET API
ASP.NET API
ASP.NET: A set of technologies in the .NET Framework for building web applications and XML web services.API: A software intermediary that allows two applications to interact with each other.
338 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Bruce (SqlWork.com) 65,576 Reputation points
    2022-10-14T16:30:52.94+00:00

    you need to better understand the api limits. is max concurrent requests (denial of service attack detection), max requests per second, max requests per minute, max requests per hour, max requests per day?

    you only need as many threads a con-current requests are allowed. to implement the queue use:

    https://zcusa.951200.xyz/en-us/dotnet/api/system.collections.concurrent.concurrentqueue-1?view=net-7.0

    you can use semaphores to wakeup the background queue processes when adding to the queue, or the they can use a timer if the queue is empty.

    0 comments No comments

  2. Lex Li (Microsoft) 5,582 Reputation points Microsoft Employee
    2022-10-19T06:28:47.187+00:00

    ASP.NET Core 7 already includes built-in rate limiting support,

    https://github.com/dotnet/aspnetcore/issues/29933

    https://zcusa.951200.xyz/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-7.0

    If you are using ASP.NET 4.x, then you need to find an alternative for that legacy framework.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.