Changes between Version 2 and Version 3 of DPDKNotes
- Timestamp:
- 02/13/14 15:29:39 (8 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
DPDKNotes
v2 v3 4 4 5 5 The Intel Data Plane Development Kit format allows packets to be captured in a truly zero copy manner and provides direct access to every packet with almost zero overhead. This means more CPU is left for your application to process the packet. Libtraces Intel DPDK capture format works in a very similar way to the DAG capture format. 6 The format supports most Intel NIC's see the release notes pdf.6 The format supports most Intel NIC's see the DPDK release notes pdf. 7 7 8 8 Documentation and source code for the Intel DPDK can be downloaded from [http://www.intel.com/go/DPDK] the links are in a box at the bottom of the page. 9 9 10 10 = System Requirements = 11 * Gettimeofday() and/or clock_gettime() must be implemented as virtual system calls for your linux kernel, these are called for every packet received so the advantage of using DPDK will be lost if a system call still has to be made.11 * Gettimeofday() and/or clock_gettime() must be implemented as virtual system calls for your Linux kernel, these are called for every packet received so the advantage of using DPDK will be lost if a system call still has to be made. 12 12 * DPDK is a polling format hence it is highly recommended to use a multicore system so other processes can be run on the remaining cores. 13 13 * For better performance the CPU core that has DPDK bound to it should only be running DPDK as such interrupts could be disabled on this core. 14 14 15 15 = Libtrace application requirements = 16 * DPDK v1.5 or newer is required 16 17 * The same thread must be used to create, start and read/write packets and all other calls to libtrace format dependent functions. 17 18 * Minimal processing should be done on the thread interacting with libtrace and the DPDK format, for two main reasons: … … 26 27 27 28 1. Read the DPDK Getting Started Guide and make sure the prerequisites are met such as hugepages. 28 2. Download DPDK from the Intel website [http://www.intel.com/go/DPDK] 29 2. Download DPDK from the Intel website [http://www.intel.com/go/DPDK] or dpdk.org [http://www.dpdk.org/] 29 30 3. Extract the archive: 30 31 31 32 {{{ 32 ~unzip IntelDPDK.L.1.3.1_7.zip-d IntelDPDK33 ~cd IntelDPDK 33 ~unzip DPDK-1.6.0-18 -d IntelDPDK 34 ~cd IntelDPDK/DPDK-1.6.0 34 35 }}} 35 36 36 4. Apply the patch named DPDK_libtrace.patch included within libtrace/Intel DPDK Patches, assuming we have copied this into the newly created IntelDPDK folder. This is '''required''' to allow libtrace to create shared libraries otherwise building libtrace will fail. 37 4. Apply optional patches (For a specific card HW timestamping etc.. make sure changes are also made to libtrace defines where needed)[[BR]] 38 39 5. Make the DPDK library with the CONFIG_RTE_BUILD_COMBINE_LIBS=y and EXTRA_CFLAGS="-fPIC" added. This should create a the static library x86_64-default-linuxapp-gcc/libs/libintel_dpdk.a required by libtrace, note prior to DPDK v1.5 CONFIG_RTE_BUILD_COMBINE_LIBS is not supported and this lirbary will not be created. 37 40 38 41 {{{ 39 ~ patch -p0 < DPDKLibtracePatch.diff42 ~make install T=x86_64-default-linuxapp-gcc CONFIG_RTE_BUILD_COMBINE_LIBS=y EXTRA_CFLAGS="-fPIC" 40 43 }}} 41 44 42 5. Apply optional patches (For a specific card HW timestamping etc.. make sure changes are also made to libtrace defines where needed)[[BR]] 43 44 6. Make the DPDK library 45 46 {{{ 47 ~cd DPDK 48 ~make install T=x86_64-default-linuxapp-gcc 49 }}} 50 51 7. Export RTE_SDK and RTE_TARGET 45 6. Export RTE_SDK and RTE_TARGET 52 46 53 47 {{{ … … 56 50 }}} 57 51 58 8. Load the DPDK modules 59 60 {{{ 61 ~cd $RTE_TARGET/kmod 62 ~sudo modprobe uio 63 ~sudo insmod ./igb_uio.ko 64 }}} 65 66 9. Set any advance options within libtrace if required (defines at top of ./lib/format_dpdk.c) 67 10. Configure and build - RTE_SDK and RTE_TARGET must be set in the environment for Intel DPDK to be detected 52 7. Set any advance options within libtrace if required if you have applied patches (defines at top of ./lib/format_dpdk.c) 53 8. Configure and build - RTE_SDK and RTE_TARGET must be set in the environment for Intel DPDK to be detected 68 54 69 55 {{{ … … 74 60 }}} 75 61 62 9. Load the DPDK modules 63 64 {{{ 65 ~cd $RTE_TARGET/kmod 66 ~sudo modprobe uio 67 ~sudo insmod ./igb_uio.ko 68 }}} 69 70 10. Use the pci_unbind.py tool (found in IntelDPDK/tools/) to bind the port you want to use to the igb_uio driver 71 72 {{{ 73 ~cd ../IntelDPDK/DPDK-1.6.0 74 ~sudo ./pci_unbind.py --status 75 Network devices using IGB_UIO driver 76 ==================================== 77 <none> 78 79 Network devices using kernel driver 80 =================================== 81 0000:01:00.0 '82580 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio 82 0000:01:00.1 '82580 Gigabit Network Connection' if=eth2 drv=igb unused=igb_uio 83 0000:03:00.0 'NetXtreme BCM5754 Gigabit Ethernet PCI Express' if=eth0 drv=tg3 unused=<none> *Active* 84 85 Other network devices 86 ===================== 87 <none> 88 ~sudo ./pci_unbind.py -b igb_uio 0000:01:00.0 89 ~sudo ./pci_unbind.py --status 90 Network devices using IGB_UIO driver 91 ==================================== 92 0000:01:00.0 '82580 Gigabit Network Connection' drv=igb_uio unused= 93 94 Network devices using kernel driver 95 =================================== 96 0000:01:00.1 '82580 Gigabit Network Connection' if=eth2 drv=igb unused=igb_uio 97 0000:03:00.0 'NetXtreme BCM5754 Gigabit Ethernet PCI Express' if=eth0 drv=tg3 unused=<none> *Active* 98 99 Other network devices 100 ===================== 101 <none> 102 103 }}} 104 105 11. Test a libtrace tool here the pci address can be found with the pci_unbind tool 106 107 {{{ 108 ~/tracesummary dpdk:0000:01:00.0 109 }}} 110 111 76 112 = Advance Settings (Defines at the top of libtrace/lib/dpdk.c) = 77 ''This is based upon testing using the Intel DPDK 1.3.1_7 and a Intel 82580 based Ethernet controller. Some of these settings are not supported by all controllers.''113 ''This is based upon testing using the Intel DPDK 1.3.1_7(No longer supported by libtrace) and a Intel 82580 based Ethernet controller. Some of these settings are not supported by all controllers.'' 78 114 79 115 == NB_RX_MBUF - Number of memory buffers i.e. number of packets in the ring buffer ==