CPU Usage on 10GB test

[andrea.m]

Hi all, i’m testing the driver performance over a 10GB network.

The drivers works in pass-mode (g_dwStartupMode = 0) so it doesn’t make operations on packets flow.

When Jumbo packets are used i get about 9.5+ GBps with only one core used by the driver. The used core in far from being full, so the test complete succesfully.

But when JP are disabled there are ~6x more request that cause a 100% core usage and the throughput collpase to 5Gbps. That behaviour happens when the filter is active, when i disable it al works well.

I’m pretty sure that the bottleneck is the CPU load, so the question is:
Is there a way to take advantage by using multithread solutions?

Maybe is a silly question, thanks in advance

 

 

[Vadim Smirnov]

It’s great to see that you’re testing the driver performance over a 10GB network.

Exploring multithreading is a good approach, as it can help distribute the load across multiple CPU cores and potentially improve performance. For example, in a project I encountered, packets were read by one thread, processed by two additional threads (one for incoming and another for outgoing), and injected into the stack by a fourth thread.

To provide more specific suggestions and help optimize your code, it would be helpful to see a code snippet demonstrating how you’re reading and re-injecting packets per operation. The more data you read from the driver in a single call, the better performance you can achieve. For instance, if you’re currently reading 256 packets per operation, you might try increasing this value to 1024 and see if it improves the results.

[andrea.m]

Hi Vadim, thanks for the reply. Due to the hurry i didn’t explain well the problem, let me add some detail.

To test network bandwith i use a custom application (such iperf) that creates several connection, every connection on a different thread. This app can be sever or client and can be binded on a NIC, but it doesn’t know anything about the filter driver on the card and it doesn’t make any operation directly with driver (read packets etc), only a multiple socket based connection.

So if i start the test over the network with the filter disabled i achieve about 9,5+ Gbps and from task manager i see that the application threads are spreaded well on processor cores. If i use 10 different connections i see almost every core busy ( in my enviroment machines had 8 cores ).
More specifically form resource monitor i see that the load is split by my app ad System Interrupts.

By only enabling the filter from adapter properties in control panel ( any of two peers ), test result collapse to 2,5-3Gbps. My app run with same arguments the only difference is the filter active on the NIC.
Watching task manager and resource monitor only one core is busy ( in fact saturated ) by handle system interrupts. So the bandwith drop is certainly an effect of the use of the cores. One core can’t serve all interrupts and becomes the bottleneck.

Do you have any idea of the cause of this behavior?

Many thanks

[andrea.m]

This is preliminary test, after that by enabling the filter i have to do some operations on the packet flow.
In case some particular modification is needed, I have the sources.

Thanks again

[Vadim Smirnov]

Have you confirmed whether your system utilizes jumbo frames when the Windows Packet Filter option is unchecked or the driver was uninstalled?

I have conducted tests on the Windows Packet Filter over a 10 Gbit network and did not observe any significant performance degradation solely from enabling the driver. It’s important to note that the test system was quite powerful, equipped with an E-2378G processor and Broadcom P210tep NetXtreme network interfaces. Furthermore, even when running a test packet filtering application with minimal output (such as dnstrace), the bandwidth was not substantially affected.

Would you mind sharing your hardware configuration, so we can make a rough comparison?

[andrea.m]

Many thanks for the help.

Jumbo packet are off by requirements.

I’m adding some screenshot to understand better, and there are also the HW spec.

When i start the test with the filter of i have that scenario ( idk why i can’t post links, add https: before to create link ) :

//ibb.co/3R7vd2b

//ibb.co/FH5pkNL

As you can see many cores are involved and network interface in almost full. Now i enable the filter:

and the result is

//ibb.co/q0Xspks

//ibb.co/93pBqLK

The NIC is a VLAN in this instance but with a physiscal one i have no change.

The bandwith drop is certainly because is the maximum amount of packets that a single core can handle. With npcap i dont’ have the same beahviour.

Hope it helps

Thanks again

[Vadim Smirnov]

Thank you for providing the details. I will conduct some research and keep you informed.

[andrea.m]

Thank you Vadim, i will make some test also bymyself.

Please keep me informed if you have some news.

[Vadim Smirnov]

Could you please test this experimental build?

P.S. Please note to download v3.4.2, I had to fix the link.

[andrea.m]

Hi Vadim, your build works perfectly! thank a lot.

as i told you my company bought source code and one year of assistance, and now we would need the sources updated with this fix.

I have sent to you an e-mail at support@ntkernel.com with all the details of the order.

Keep in touch.

[Vadim Smirnov]

Hi Andrea,

Thank you for the update. I recently responded to your email. As mentioned, I want to further test several modifications, including the new driver registry parameter designed to expand the internal driver packet pool without rebuilding the driver.

I am also working on a project that necessitates the efficient transfer of packets over high-speed 10 Gbps interfaces. Consequently, I plan to rigorously assess this build in the coming days, confirming its reliability and compatibility before proceeding with an official release.

If you require the updated version urgently, I can supply you with the latest code snapshot from the repository.

[andrea.m]

Yes, this CPU usage behaviour blocks a series of test and more releases that we have in program, so if you can update us with the sources patched in this way as soon as possible, we appreciate.

For the rest of the changes you have planned, that are really interesting, we can update more calmly.

Thanks again.

 

[Vadim Smirnov]

I’ve sent you a download link via email for the recent source code corresponding to version 3.4.2. If you have any additional questions or require further clarification, please don’t hesitate to ask.

[andrea.m]

I’ve checked my business email <andrea.montanari@aubay.it> that’s the one that i used this morning but i can’t find your aswer, even in the spam folder.

Maybe with a link inside it was rejected by company server, idk. Please try to resend also in my personal account <andrea.monti.89@gmail.com>

Thanks

 

 

[Vadim Smirnov]

I’ve forwarded an original e-mail. Please confirm.

[Author]

Posts