Xenial - OpenvSwitch with DPDK binding to 10G NIC, not working...

Tue Mar 8 01:07:24 UTC 2016

On 7 March 2016 at 21:45, Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote:

> On 7 March 2016 at 20:16, Martinx - ジェームズ <thiagocmartinsc at gmail.com>
> wrote:
>
>> On 7 March 2016 at 17:28, Martinx - ジェームズ <thiagocmartinsc at gmail.com>
>> wrote:
>>
>>> On 7 March 2016 at 16:40, Martinx - ジェームズ <thiagocmartinsc at gmail.com>
>>> wrote:
>>>
>>>> Hey guys,
>>>>
>>>>  I'm trying to create two OVS bridges powered by DPDK, on Xenial, but
>>>> I'm facing problems...
>>>>
>>>>  First, there is no documentation about how to do this... So far, I
>>>> found the following:
>>>>
>>>>
>>>>  - This link doesn't how to bind real NIC:
>>>>
>>>> https://software.intel.com/en-us/articles/using-open-vswitch-with-dpdk-on-ubuntu
>>>>
>>>>  - This one isn't specific to Ubuntu OVS + DPDK packages:
>>>>  https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md
>>>>
>>>>  - Similar with above but, very old... Also not related to Ubuntu
>>>> packages:
>>>>
>>>> https://feiskyer.github.io/2015/03/01/open-vswitch-over-dpdk-on-ubuntu/
>>>>
>>>>
>>>>  So, to begin with, I start with Intel.com guide, it worked! But
>>>> without real NIC.
>>>>
>>>>  Then, I tried to bind it to real NIC, like this:
>>>>
>>>>  Added two NIC PCI id to "/etc/dpdk/interfaces" file:
>>>>
>>>> ---
>>>> pci     0000:01:00.0    uio-pci-generic
>>>> pci     0000:01:00.1    uio-pci-generic
>>>> ---
>>>>
>>>>  Then, reboot, I can see both NICs using "DPDK-compatible driver" by
>>>> running: "dpdk_nic_bind --status".
>>>>
>>>>  Trying to create new OVS Bridges with DPDK devices, failing:
>>>>
>>>> ---
>>>> root at xenial-1:~# ovs-vsctl add-br ovsbr0
>>>> root at xenial-1:~# ovs-vsctl set bridge ovsbr0 datapath_type=netdev
>>>> root at xenial-1:~# ovs-vsctl add-port ovsbr0 dpdk0 -- set Interface
>>>> dpdk0 type=dpdk
>>>> ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd
>>>> log for details.
>>>> ---
>>>>
>>>> Log shows:
>>>>
>>>> ---
>>>> 2016-03-07T19:34:07.532Z|00136|bridge|WARN|could not open network
>>>> device dpdk0 (No such device)
>>>> ---
>>>>
>>>> ---
>>>> root at xenial-1:~# ovs-vsctl show
>>>> 2b84f2f7-6d3f-4d51-94a5-fe6d04f0a062
>>>>     Bridge "ovsbr0"
>>>>         Port "ovsbr0"
>>>>             Interface "ovsbr0"
>>>>                 type: internal
>>>>         Port "dpdk0"
>>>>             Interface "dpdk0"
>>>>                 type: dpdk
>>>>                 error: "could not open network device dpdk0 (No such
>>>> device)"
>>>>     ovs_version: "2.5.0"
>>>> ---
>>>>
>>>>  What am I missing?
>>>>
>>>>  I tried to use real NIC device name "em1", instead of "dpdk0", same
>>>> results... Since "em1" does not even shows up on "ip link" list... DPDK
>>>> toke it over...
>>>>
>>>> Thanks!
>>>> Thiago
>>>>
>>>
>>> Guys,
>>>
>>> I am also trying this:
>>>
>>>  Added two NIC PCI id to "/etc/dpdk/interfaces" file:
>>>
>>> ---
>>> pci     0000:01:00.0    vfio-pci
>>> pci     0000:01:00.1    vfio-pci
>>> ---
>>>
>>> According to:
>>> https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md - but,
>>> it also doesn't work...
>>>
>>> When using "vfio-pci", the NIC doesn't even show up as a DPDK-compatible
>>> driver after running: "dpdk_nic_bind --status"...
>>>
>>> So, how to use "vfio-pci" with Ubuntu DPDK?
>>>
>>> Cheers!
>>> Thiago
>>>
>>
>> After reading INSTALL.DPDK.md again, I realized that it only works with
>> VFIO or IGB (UIO), Ubuntu DPDK package does not provides the "igb_uio"
>> module, neither via DKMS. So, the only option is VFIO, right?
>>
>> And, I am unable to use "vfio-pci" configured inside of
>> "/etc/dpdk/interfaces" but, I managed to make it work manually.
>>
>> I just filled a bug report against DPDK Ubuntu package related to this
>> "vfio-pci" problem:
>>
>> ---
>> Usege of vfio-pci module via /etc/dpdk/interfaces doesn't work:
>> https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1554214
>> ---
>>
>> So, after workaround the bug #1554214 (bug description have the
>> instructions), I tried OpenvSwitch with DPDK + 10G NIC, again, still no
>> success.
>>
>> Same problem...
>>
>> * DPDK using VFIO:
>>
>> ---
>> root at xenial-1:~# dpdk_nic_bind --status
>>
>> Network devices using DPDK-compatible driver
>> ============================================
>> 0000:01:00.0 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' drv=vfio-pci
>> unused=bnx2x
>> 0000:01:00.1 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' drv=vfio-pci
>> unused=bnx2x
>> ---
>>
>> * OpenvSwitch with DPDK still failing:
>>
>> ---
>> service openvswitch-switch stop
>> service openvswitch-switch start
>>
>> ps xfa | grep dpdk  # I can see "ovs-vswit... --dpdk -c 0x3 -n 4 -- ...."
>> running...
>>
>>
>> root at xenial-1:~# ovs-vsctl add-br br0 -- set bridge br0
>> datapath_type=netdev
>> root at xenial-1:~# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0
>> type=dpdk
>> ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd log
>> for details.
>> ---
>>
>> No luck... I can't wait to see it working in my lab!
>>
>> Ubuntu rocks! Almost there...   :-D
>>
>> Best,
>> Thiago
>>
>
> Hey guys,
>
>  I just found a minor issue with openvswitch-switch Init Script:
>
>  * Script "/etc/init.d/openvswitch-switch restart" ignores "DPDK_OPTS",
> while "service openvswitch-switch restart" works!
>
>  https://bugs.launchpad.net/ubuntu/+source/openvswitch-dpdk/+bug/1554275
>
> -
> Thiago
>

The interest about this subject is growing!

--
Using Open vSwitch* with DPDK on Ubuntu is great! But what about NIC
binding?
https://software.intel.com/en-us/forums/networking/topic/611450

-

Unable to restart ovs-vswitchd with dpdk on physical NICs under Ubuntu
16.04 - Address family not supported by protocol:
https://software.intel.com/en-us/forums/networking/topic/611366
--

I can't wait for an answer! Exciting times!   :-D

Best!
Thiago
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel-discuss/attachments/20160307/55aa8a80/attachment.html>