Does moving a GPU server's request loop into the kernel save CPU? (eunomia.dev)

The recent development of bpf-gpu-server explores the feasibility of moving a GPU server's request loop into the Linux kernel to optimize CPU usage and reduce latency. By leveraging eBPF and io_uring, this approach allows the TCP path of a GPU server to function without the involvement of a user thread for each request, theoretically saving CPU resources and streamlining communication. However, extensive testing revealed that this strategy did not yield significant improvements in CPU efficiency or latency when compared to a standard C++ epoll server managing the same GPU workloads. This project highlights the challenges of optimizing server performance through architectural changes. Despite removing user-level overhead such as system calls and memory copies, the increased system time due to kernel processing, TCP stack overhead, and lack of batching mechanisms led to similar CPU costs for both approaches. The insights gained from this study suggest that future optimizations should focus on processing packets before entering the TCP stack or employing direct NIC-to-GPU paths to truly capitalize on potential CPU savings. The project remains an open research prototype, encouraging further exploration of these concepts in GPU networking.