Hi, Le mardi 05 janvier 2021 à 17:13 -0800, Joe Doe a écrit :

One of the architectures that Linux applications use, is to run in a loop that watches a number of FDs (e.g. epoll()) and react to the events from those FDs.

Now let's say I am writing a communication library that users can use to send and receive messages to other processes over the network. And let's assume that for reasons, my library is using multiple connections internally.

I have no way to construct my library in such a way that it will only export a single FD to the user, and hide the rest of the internal connections. I will either have to expose all the FDs, making it more difficult for the user, or I would have to run the library code in a different thread, and use eventfd to make the interface simpler.

Having a "UnionFD" that would get triggered when one of my other FDs is ready for I/O would solve the problem.

It's already here, epollfd_create1() returns a file descriptor that can be polled.

For example an API of the hypothetical library with the tools we have today might look something like this:

int comm_lib_init(void); // Returns the "master" FD int comm_lib_accept(int master_fd); // Returns an FD when a new connection is accepted int comm_lib_send_message(int master_fd, const char *dest, void *data, size_t data_len); // If a new connection was required to send this message, then it will return the FD int comm_lib_recv_message(int master_fd, ...);

The user of the library would have to do the bookkeeping for all the FDs that are returned by the above APIs themself.

Instead with a "UnionFD", the API would only ever return a single FD and the user would call `comm_lib_send_message` and `comm_lib_recv_message` with this FD.

E.g. the application's code would look something like that:

comm_fd = comm_lib_init(); epollfd = poll_create1(0);

ev.events = EPOLLIN; ev.data.fd = comm_fd; epoll_ctl(epollfd, EPOLL_CTL_ADD, comm_fd, &ev); for (;;) {      fds = epoll_wait(epollfd, events, MAX_EVENTS, -1); }

Once in the loop, the user doesn't have to worry about FD bookkeeping anymore.

Another example would be a much simpler HTTP library. The library would just return a single "master_fd" that the user would plug in their epoll() and just call something like `get_next_request()` when the FD gets triggered. That way we could easily add this HTTP server to an epoll() loop that monitors other FDs as well. No more worrying about accept(), and per-connection FD bookkeeping.

I believe a construct like this would help some popular existing libraries as well, like zeromq and the grpc library.

Does the above make sense? Or is there already a way to achieve that in Linux that I am not aware of?

Thanks,

Regards. -- Yann Droneaud OPTEYA