Multithreaded Network application design questions

Multithreaded Network application design questions

Post by shiygoogle » Thu, 04 Oct 2007 22:48:54

I am designing and implementing the clientside of a multi-threaded
network application. The client interfaces need to be asynchronous and
work on Unix/Windows. I have decided on the following at a high level

1) have a monitor thread that polls/selects on the sockets
2) a queue to hold the I/O completed requests awaiting further
3) a "background" threadpool that picks up requests from the queue and
processes it

I have the following questions :-

1) How will the foreground/background threads communicate with the
monitor thread to add sockets of "interest" ? I can think of only
or Unix datagram sockets.
2) Which is the best data structure and concurrency control for the
3) I am planning to provide the following style of interface for the
application :-

token1 = ReadFromDb();/* asynchronous - processed by the monitor
and background thread(s) */
token2 = ReadFromSNMP();/* asynchronous */

... //application processing


I plan to use pthread condition variables in each token. How do I
implement a interface that provides to wait for multiple tokens :-

WaitForMultiple(token1, token2); //returns when any of the 2
requests is complete

Thanks in advance for all responses. How can I design/implement
something similar to Windows I/O Completion ports on Unix?


Multithreaded Network application design questions

Post by Gianni Mar » Fri, 05 Oct 2007 06:26:40

The Austria C++ library has a "netcabletv" alpha threaded "network"

The Austria C++ "Net" interface provides the functionality that usually
surrounds the poll/select

The Austria C++ library uses the "Twin" interface. The responsibility
for queuing is up to the clients. There is a "thread pool" or
ActivityList object that provides most of the functionality for queing

Austria C++ has a generic threadpool system in such a way that a singe
object can register it's own thread pool with the main thread pool so
that in a single object, only one thread will ever be in it at any point
in time and the "dispatching" of events is generic - i.e. no switch

Asynchronous calls must never return a value.

You have to use the same condition variable. However, you're mixing
thread paradigms. If you're using a thread pool, why not simply enqueue
an action ?

Austria C++'s Net interface works the same with Linux epoll as well as
Windows completion ports (although I didn't implement those).

The Austria C++ network library is not production code yet, although it
does pass unit tests, it needs a few changes to the interface IMHO.


Multithreaded Network application design questions

Post by Arnold Hen » Sat, 06 Oct 2007 02:56:55

I'd suggest a shared data area to communicate events such as 'add a socket
to your set' back to your main select/poll loop, and a
pipe/socketpair/whichever to signal the availability of events in your data
area. Sending a signalling byte on the pipe allow you to break out of the
poll loop (sending the event data through the pipe itself would require you
to handle hitting the maximum pipe buffer size).

Use a single pthread condition variable for both tokens - signal it whenever
either the state of token1 or token2 changes.