Member Login | Register
Navigation
  • Home
  • Blog
  • Forums
  • FAQ
Home : All Forums : Hardware : APU
Recent Posts My Posts My Subscriptions

APU2 performance is insufficient for Gigabit if WAN is PPPoE

Subscribe New Topic Reply
    Next
    1 2
  • QuoteRate
    My Posts

    Posts: 18
    ricsip - Posted Jul 20th 2018

    Hello dear PCENGINES team.

    Just wanted to highlight, that your APU2 board cannot handle full 1 Gigabit WAN traffic inbound (from Internet), if the ISP provides the WAN connection through  PPPoE protocol.
    Regardless of APU2C0,APU2C2, or APU2C4 (the issue present in all models)

    PPPoE connection type cannot use Receive Side Scaling loadbalancing, and the Intel i210 / 211 NIC multi-queue feature cannot be used (the NIC will use only queue 0, and not the others), because all the WAN traffic will be encapsulated
    into 1 stream, and by definition that cannot be load-balanced into multiple receive queues.Same issue affects any non-IP traffic, like VPN tunneling so all of the IPSEC, PPTP. L2TP, GRE protocols.

    So only 1 CPU core will receive the full PPPoE network traffic, regardless of how many different flows are inside the encapsulated stream. Because a single core running @1Ghz is insufficient to process the full Gigabit network traffic, you will never be able to reach 1 Gbit routing traffic using an APU2 board (1 CPU core is 100% utilised, while the other 3 CPU core is IDLE 90+%). If you add firewall / IDS rules, performance will be even more lower. So APU2 is NOT RECOMMENDED for Gigabit over PPPoE firewall purpose.


    Unless the RSS standard is updated by MSFT or Intel improves the i210/211 NIC firmware to properly load-balance even PPPoE frames (totally unlikely as I see this issue).

    I would like to ask you to put this info clear warning on the datasheet of the APU2, not to mislead your future customers, who will buy your product believing that multi-queue NIC and quad-core CPU can easily handle
    Gigabit traffic, when their ISP provider is using PPPoE protocol instead of simple IP connection.

    Proof from FreeBSD kernel mail list:
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203856

  • QuoteRate
    My Posts

    Posts: 118
    mikeboss - Posted Jul 21st 2018

    what was the max. throughput possible? have you tried Linux based firewalls (VyOS or Sophos UTM)?

  • QuoteRate
    My Posts

    Posts: 18
    ricsip - Posted Jul 23rd 2018
    Actually I did only iperf test: 1 iperf endpoint connected to LAN interface of the APU2 opnsense, 1 iperf endpoint connected to the WAN interface.

    Max. throughput between LAN and WAN was 700mbit for 1-flow iperf, and 930 mbit for 2-flow iperf. I started troubleshooting of the poor 1-flow iperf performance, when I found that PPPoE performance will be similarly poor, as the PPP session will also work like a 1-flow iperf session.

    I am planning to test real-life PPPoE session, but its not that easy, as simulating plain simple IP WAN testing with 2 iperf endpoint connecting to 2 interface on the APU2.
  • QuoteRate
    My Posts

    Posts: 67
    VAMike - Posted Jul 24th 2018

    ricsip - Posted 19 Hours Ago

    Actually I did only iperf test: 1 iperf endpoint connected to LAN interface of the APU2 opnsense, 1 iperf endpoint connected to the WAN interface.

    Max. throughput between LAN and WAN was 700mbit for 1-flow iperf, and 930 mbit for 2-flow iperf. I started troubleshooting of the poor 1-flow iperf performance, when I found that PPPoE performance will be similarly poor, as the PPP session will also work like a 1-flow iperf session.

    I am planning to test real-life PPPoE session, but its not that easy, as simulating plain simple IP WAN testing with 2 iperf endpoint connecting to 2 interface on the APU2.


    Out of the box on linux one of these boards can do  over 900Mbps with a single iperf session. You need to distinguish between what the hardware can do and what the OS is doing.

  • QuoteRate
    My Posts

    Posts: 18
    ricsip - Posted Jul 24th 2018

    VAMike - Posted 14 Minutes Ago

     


    ricsip - Posted 19 Hours Ago

    Actually I did only iperf test: 1 iperf endpoint connected to LAN interface of the APU2 opnsense, 1 iperf endpoint connected to the WAN interface.

    Max. throughput between LAN and WAN was 700mbit for 1-flow iperf, and 930 mbit for 2-flow iperf. I started troubleshooting of the poor 1-flow iperf performance, when I found that PPPoE performance will be similarly poor, as the PPP session will also work like a 1-flow iperf session.

    I am planning to test real-life PPPoE session, but its not that easy, as simulating plain simple IP WAN testing with 2 iperf endpoint connecting to 2 interface on the APU2.


    Out of the box on linux one of these boards can do  over 900Mbps with a single iperf session. You need to distinguish between what the hardware can do and what the OS is doing.

     


    I will try to repeat the tests with OpenWRT to see if its really the FreeBSD OS / drivers are the bottleneck, or under Linux it performs similar. In the meantime, are you aware of any REAL test proving 1 Gbit performance under PPPoE?

  • QuoteRate
    My Posts

    Posts: 67
    VAMike - Posted Jul 24th 2018

    ricsip - Posted 4 Hours Ago

    I will try to repeat the tests with OpenWRT to see if its really the FreeBSD OS / drivers are the bottleneck, or under Linux it performs similar. In the meantime, are you aware of any REAL test proving 1 Gbit performance under PPPoE?


    With linux I ran a quick test using iperf through an APU2, got around 880Mbps down & 770Mbps up via pppoe, with faster hardware hitting about 930Mbps both ways. Without pppoe it's about 930Mbps both ways on the APU2. I didn't do any performance tuning, YMMV.


    Post last edited Jul 24th 2018
  • QuoteRate
    My Posts

    Posts: 18
    ricsip - Posted Jul 27th 2018

    VAMike - Posted 11 Hours Ago

     


    ricsip - Posted 4 Hours Ago

    I will try to repeat the tests with OpenWRT to see if its really the FreeBSD OS / drivers are the bottleneck, or under Linux it performs similar. In the meantime, are you aware of any REAL test proving 1 Gbit performance under PPPoE?


    With linux I ran a quick test using iperf through an APU2, got around 880Mbps down & 770Mbps up via pppoe, with faster hardware hitting about 930Mbps both ways. Without pppoe it's about 930Mbps both ways on the APU2. I didn't do any performance tuning, YMMV.

     


    Would you mind trying Opnsense 18.1 on your APU box (running on a pendrive) to see if you get the same results as your linux test or instead same results as mine (the bad one)?

  • QuoteRate
    My Posts

    Posts: 67
    VAMike - Posted Jul 27th 2018

    ricsip - Posted 6 Hours Ago

     


    VAMike - Posted 11 Hours Ago

     


    ricsip - Posted 4 Hours Ago

    I will try to repeat the tests with OpenWRT to see if its really the FreeBSD OS / drivers are the bottleneck, or under Linux it performs similar. In the meantime, are you aware of any REAL test proving 1 Gbit performance under PPPoE?


    With linux I ran a quick test using iperf through an APU2, got around 880Mbps down & 770Mbps up via pppoe, with faster hardware hitting about 930Mbps both ways. Without pppoe it's about 930Mbps both ways on the APU2. I didn't do any performance tuning, YMMV.

     


    Would you mind trying Opnsense 18.1 on your APU box (running on a pendrive) to see if you get the same results as your linux test or instead same results as mine (the bad one)?

     


    OPNsense gives about the same or a little faster with the firewall off, but only about 530Mbps up and 620Mbps down with the firewall on and rules to allow everything, no NAT. NAT may have been an additional hit, but I didn't want to deal with it. :) Without pppoe it's hitting about 810Mbps up and 820Mbps down with the firewall enabled (maxes out with firewall off). On linux turning the firewall on has basically no impact on a direct connection, and much lower impact (30-50Mbps) on pppoe.

  • QuoteRate
    My Posts

    Posts: 18
    ricsip - Posted Jul 31st 2018

    VAMike - Posted 4 Days Ago

     


    ricsip - Posted 6 Hours Ago

     


    VAMike - Posted 11 Hours Ago

     


    ricsip - Posted 4 Hours Ago

    I will try to repeat the tests with OpenWRT to see if its really the FreeBSD OS / drivers are the bottleneck, or under Linux it performs similar. In the meantime, are you aware of any REAL test proving 1 Gbit performance under PPPoE?


    With linux I ran a quick test using iperf through an APU2, got around 880Mbps down & 770Mbps up via pppoe, with faster hardware hitting about 930Mbps both ways. Without pppoe it's about 930Mbps both ways on the APU2. I didn't do any performance tuning, YMMV.

     


    Would you mind trying Opnsense 18.1 on your APU box (running on a pendrive) to see if you get the same results as your linux test or instead same results as mine (the bad one)?

     


    OPNsense gives about the same or a little faster with the firewall off, but only about 530Mbps up and 620Mbps down with the firewall on and rules to allow everything, no NAT. NAT may have been an additional hit, but I didn't want to deal with it. :) Without pppoe it's hitting about 810Mbps up and 820Mbps down with the firewall enabled (maxes out with firewall off). On linux turning the firewall on has basically no impact on a direct connection, and much lower impact (30-50Mbps) on pppoe.

     


    Greatly appreciate your feedback!

    Would you mind sharing the details of your testbed config (if it wasnt your production PPPoE internet provider but a demo built), so I can simulate the same PPPoE server setup for the througput testing? I have a spare laptop to install a FreeBSD-based PPPoE server if that was the same that you used.

  • QuoteRate
    My Posts

    Posts: 38
    sparkie - Posted Jul 31st 2018

    generally speaking BSD based distributions appear to perform poorly on APUs. Don't know why.

    Just use Debian and enjoy.  Massive 111MB/s being forwarded between two APU2C4 ethernet ports (result gained by iftop) on Debian (stretch aka 9.0) based systems.

    For further reading I recommend:

    http://www.pcengines.info/forums/?page=post&id=0B90812A-37D8-457C-99E6-1E15B66E529E&fid=DF5ACB70-99C4-4C61-AFA6-4C0E0DB05B2A


    Post last edited Jul 31st 2018
Next
1 2
Subscribe

Rules: