Concurrent PHP With curl-multi

Disclaimer: This was just little experiment I put together during a long train ride. This is probably not something you would want to implement in a production setting, for reasons I’ll go over at the end; but, I was having fun.

In this post, I’ll show how I used PHP’s curl_multi_* functions to achieve concurrent behavior with PHP. If you’re interested in testing it out for yourself, you can see/clone a full functional example here.

Ok, ok, here’s the code…

By exploiting the server’s ability to run HTTP requests in parallel, the script can achieve concurrency by having it make asynchronous requests to itself using curl_multi_exec .

As I mentioned at the start, this is probably not something you would want to implement in a production setting, for multiple reasons that I’m sure you can imagine; but, mainly, this will exponentially increase the traffic to your server(s).

To see how the traffic would increase, let’s math…
f(n, m) = n + n * m , where:
- n is the number of external/parent requests
- m is the number of internal/child requests

In the example above, there are 2 internal/child requests for every 1 external/parent request. So, if there is 1 external/parent request, the server will have to handle it plus the 2 internal/child requests at the same time; thus, f(1, 2) = 1 + 1 * 2 = 3 requests. Not bad; but, let’s bump up n to a cool 100, and m to a reasonable 4. The server, which just needed to handle 100 requests, now has to handle 100 + 100 * 4 = 500 requests. Yuck.

TL/DR: It’s very easy to overload a server with this implementation.

You could mitigate the traffic cost by adding dedicated servers for the internal/child requests, or dream up some other ‘duct tape and bubble gum’ solutions; but, why bother? There are many better alternatives:
- queues
- pcntl_fork if it’s a cli script
- pthreads
- coroutines
- or just not using PHP

Again, you can see/clone a fully functional example of the experiment here.

That’s all I got. It was a good time killer for me; so, I hope you found it time-killing also. Thanks for reading, and feedback is always welcomed 🙇