Changes between Version 2 and Version 3 of DPDKNotes


Ignore:
Timestamp:
02/13/14 15:29:39 (7 years ago)
Author:
rjs51
Comment:

Update to reflect that DPDK v1.5 or newer is now required and the changes made to the build system.

Legend:

Unmodified
Added
Removed
Modified

  • DPDKNotes

    v2 v3  
    44
    55The Intel Data Plane Development Kit format allows packets to be captured in a truly zero copy manner and provides direct access to every packet with almost zero overhead. This means more CPU is left for your application to process the packet. Libtraces Intel DPDK capture format works in a very similar way to the DAG capture format.
    6 The format supports most Intel NIC's see the release notes pdf.
     6The format supports most Intel NIC's see the DPDK release notes pdf.
    77
    88Documentation and source code for the Intel DPDK can be downloaded from [http://www.intel.com/go/DPDK] the links are in a box at the bottom of the page.
    99
    1010= System Requirements =
    11 * Gettimeofday() and/or clock_gettime() must be implemented as virtual system calls for your linux kernel, these are called for every packet received so the advantage of using DPDK will be lost if a system call still has to be made.
     11* Gettimeofday() and/or clock_gettime() must be implemented as virtual system calls for your Linux kernel, these are called for every packet received so the advantage of using DPDK will be lost if a system call still has to be made.
    1212* DPDK is a polling format hence it is highly recommended to use a multicore system so other processes can be run on the remaining cores.
    1313* For better performance the CPU core that has DPDK bound to it should only be running DPDK as such interrupts could be disabled on this core.
    1414
    1515= Libtrace application requirements =
     16* DPDK v1.5 or newer is required
    1617* The same thread must be used to create, start and read/write packets and all other calls to libtrace format dependent functions.
    1718* Minimal processing should be done on the thread interacting with libtrace and the DPDK format, for two main reasons:
    … …  
    2627
    27281. Read the DPDK Getting Started Guide and make sure the prerequisites are met such as hugepages.
    28 2. Download DPDK from the Intel website [http://www.intel.com/go/DPDK]
     292. Download DPDK from the Intel website [http://www.intel.com/go/DPDK] or dpdk.org [http://www.dpdk.org/]
    29303. Extract the archive:
    3031
    3132{{{
    32     ~unzip IntelDPDK.L.1.3.1_7.zip -d IntelDPDK
    33     ~cd IntelDPDK
     33    ~unzip DPDK-1.6.0-18 -d IntelDPDK
     34    ~cd IntelDPDK/DPDK-1.6.0
    3435}}}
    3536
    36 4. Apply the patch named DPDK_libtrace.patch included within libtrace/Intel DPDK Patches, assuming we have copied this into the newly created IntelDPDK folder. This is '''required''' to allow libtrace to create shared libraries otherwise building libtrace will fail.
     374. Apply optional patches (For a specific card HW timestamping etc.. make sure changes are also made to libtrace defines where needed)[[BR]]
     38
     395. Make the DPDK library with the CONFIG_RTE_BUILD_COMBINE_LIBS=y and EXTRA_CFLAGS="-fPIC" added. This should create a the static library x86_64-default-linuxapp-gcc/libs/libintel_dpdk.a required by libtrace, note prior to DPDK v1.5 CONFIG_RTE_BUILD_COMBINE_LIBS is not supported and this lirbary will not be created.
    3740
    3841{{{
    39     ~patch -p0 < DPDKLibtracePatch.diff
     42    ~make install T=x86_64-default-linuxapp-gcc CONFIG_RTE_BUILD_COMBINE_LIBS=y EXTRA_CFLAGS="-fPIC"
    4043}}}
    4144
    42 5. Apply optional patches (For a specific card HW timestamping etc.. make sure changes are also made to libtrace defines where needed)[[BR]]
    43 
    44 6. Make the DPDK library
    45 
    46 {{{
    47     ~cd DPDK
    48     ~make install T=x86_64-default-linuxapp-gcc
    49 }}}
    50 
    51 7. Export RTE_SDK and RTE_TARGET
     456. Export RTE_SDK and RTE_TARGET
    5246
    5347{{{
    … …  
    5650}}}
    5751
    58 8. Load the DPDK modules
    59 
    60 {{{
    61     ~cd $RTE_TARGET/kmod
    62     ~sudo modprobe uio
    63     ~sudo insmod ./igb_uio.ko
    64 }}}
    65 
    66 9. Set any advance options within libtrace if required (defines at top of ./lib/format_dpdk.c)
    67 10. Configure and build - RTE_SDK and RTE_TARGET must be set in the environment for Intel DPDK to be detected
     527. Set any advance options within libtrace if required if you have applied patches (defines at top of ./lib/format_dpdk.c)
     538. Configure and build - RTE_SDK and RTE_TARGET must be set in the environment for Intel DPDK to be detected
    6854
    6955{{{
    … …  
    7460}}}
    7561
     629. Load the DPDK modules
     63
     64{{{
     65    ~cd $RTE_TARGET/kmod
     66    ~sudo modprobe uio
     67    ~sudo insmod ./igb_uio.ko
     68}}}
     69
     7010. Use the pci_unbind.py tool (found in IntelDPDK/tools/) to bind the port you want to use to the igb_uio driver
     71
     72{{{
     73    ~cd ../IntelDPDK/DPDK-1.6.0
     74    ~sudo ./pci_unbind.py --status
     75    Network devices using IGB_UIO driver
     76    ====================================
     77    <none>
     78
     79    Network devices using kernel driver
     80    ===================================
     81    0000:01:00.0 '82580 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio
     82    0000:01:00.1 '82580 Gigabit Network Connection' if=eth2 drv=igb unused=igb_uio
     83    0000:03:00.0 'NetXtreme BCM5754 Gigabit Ethernet PCI Express' if=eth0 drv=tg3 unused=<none> *Active*
     84
     85    Other network devices
     86    =====================
     87    <none>
     88    ~sudo ./pci_unbind.py -b igb_uio 0000:01:00.0
     89    ~sudo ./pci_unbind.py --status
     90    Network devices using IGB_UIO driver
     91    ====================================
     92    0000:01:00.0 '82580 Gigabit Network Connection' drv=igb_uio unused=
     93
     94    Network devices using kernel driver
     95    ===================================
     96    0000:01:00.1 '82580 Gigabit Network Connection' if=eth2 drv=igb unused=igb_uio
     97    0000:03:00.0 'NetXtreme BCM5754 Gigabit Ethernet PCI Express' if=eth0 drv=tg3 unused=<none> *Active*
     98
     99    Other network devices
     100    =====================
     101    <none>
     102
     103}}}
     104
     10511. Test a libtrace tool here the pci address can be found with the pci_unbind tool
     106
     107{{{
     108    ~/tracesummary dpdk:0000:01:00.0
     109}}}
     110
     111
    76112= Advance Settings (Defines at the top of libtrace/lib/dpdk.c) =
    77 ''This is based upon testing using the Intel DPDK 1.3.1_7 and a Intel 82580 based Ethernet controller. Some of these settings are not supported by all controllers.''
     113''This is based upon testing using the Intel DPDK 1.3.1_7(No longer supported by libtrace) and a Intel 82580 based Ethernet controller. Some of these settings are not supported by all controllers.''
    78114
    79115== NB_RX_MBUF - Number of memory buffers i.e. number of packets in the ring buffer ==