DPDK, the Data Plane Development Toolkit, is an open source project of the Linux Foundation that provides offloading of TCP packet processing from system kernel interrupt-driven execution to the the user space, thereby significantly increasing performance.


Required tools & libraries to build the kernel & DPDK:

libxml2-dev gcc g++ make meson bison flex ninja-build libssl-dev git bc cmake pkg-config libnuma-dev


Download & build the the 4.19 kernel.

  • Branch: whle-ls1-usdpaa
  • Defconfig:  whle_ls1_usdpaa_defconfig

There are two FDT files configured for USDPAA:

  • freescale/fsl-ls1046a-whle-usdpaa.dtb
  • freescale/fsl-ls1046a-whle-usdpaa-2.dtb

First one sets all of the ethernet/SFP ports up for userspace, second one leaves one ethernet port for kernel use.


Set bootargs in U-Boot to setup hugepages, DPAA manager portals, isolate Linux so it runs on a single core & leave the other cores for DPDK:

env set bootargs console=ttyS0,115200 root=/dev/mmcblk0p1 earlycon=uart8250,mmio,0x21c0500 default_hugepagesz=1024m hugepagesz=1024m hugepages=4 isolcpus=1-3 bportals=s0 qportals=s0 iommu.passthrough=1
run mmc_boot

Building DPDK

In order for DPDK to work correctly with WHLE-LS1 we will have to build it with the DPAA driver & the rte_kni kernel module, that means we have to supply the path to 4.19 kernel headers & source files for DPDK. Newer versions of DPDK use meson/ninja for the build process, so make sure these are available on your build machine.

Keep in mind DPDK expects a specific directory structure for the kernel headers/source in order for the build process to finish correctly:

  1. Export 4.19 kernel headers to a certain directory, which we're going to call KERNEL_PATH.
  2. At KERNEL_PATH create a symlink called build which links to the kernel source directory KERNEL_SRC.
  3. You'll need to do a full kernel build once before running a DPDK build, else rte_kni part will fail.

Expected KERNEL_PATH directory tree looks like this:

    - build -> KERNEL_SRC
    - {Kernel headers}


Build on target device

meson arm64-build --cross-file config/arm/arm64_dpaa_linux_gcc -Denable_kmods=true -Dkernel_dir={KERNEL_PATH} -Dprefix=/usr
ninja -C arm64-build install

  • KERNEL_PATH is the kernel headers directory, as explained above.
  • By default DPDK builds with a debug-optimized build, which contains only partial debug information.  If you want a full debug build, add -Dbuildtype=debug . Keep in mind this will drastically increase binary sizes up to 70MB+ for a single example app, so you might encounter space problems on the target device!
  • Add -Dexamples=all to build all the example dpdk apps, or -Dexamples=l2fwd,ipsec-secgw,etc to build only specific ones.

Prerequisites for running DPDK applications

  1. Download dpdk-extras and put rte_kni.ko from the DPDK build folder in dpdk-extras/dpaa
  2. Install fmc either by:
    - Using  make install in dpdk-extras/fmc
    - or build & install from source
  3. Run dpdk-extras/dpaa/dpdk_env.sh

Example applications

We're assuming your device has at least one ethernet port configured in the FDT for userspace. If you're not using the fm1-mac9 and fm1-mac10 (SFP) ports, you'll most probably have to block them due to DPDK applications quitting abnormally if they're not connected. For testing purposes you might also want to create a virtual tap device.



This will run the L2 forwarding app setup to forward all packets between adjacent ports, e.g. 0 and 1.

dpdk-l2fwd -l 1,2,3 -n 1 -b "dpaa_bus:fm1-mac9" -b "dpaa_bus:fm1-mac10" --vdev="net_tap0,iface=dtap0" -- -p 0xff --portmap="(0,1)"


This is analogous to l2fwd with a crypto operation specified from the command line ran on each packet forwarded.

dpdk-l2fwd-crypto -l 1,2,3 -n 1 -b "dpaa_bus:fm1-mac9" -b "dpaa_bus:fm1-mac10" --vdev="net_tap0,iface=dtap0" -- -p 0xff --cdev_type HW --chain CIPHER_HASH --cipher_op ENCRYPT --cipher_algo aes-cbc --cipher_key 00:01:02:03:04:05:06:07:08:09:0a:0b:0c:0d:0e:0f --auth_op GENERATE --auth_algo sha1-hmac --auth_key 10:11:12:13:14:15:16:17:18:19:1a:1b:1c:1d:1e:1f


dpdk-ipsec-secgw -l 1,2,3 -n 1 -b "dpaa_bus:fm1-mac9" -b "dpaa_bus:fm1-mac10" --vdev=net_tap0,iface=dtap0,mac="de:de:de:de:de:de" -- -P -f "ep0.cfg/ep1.cfg" -p 0xff --config="(0,0,1),(1,0,2)"


Allows you to query and set parameters of userspace controlled NICs.

dpdk-ethtool -l 1,2,3 -n 1 -b "dpaa_bus:fm1-mac9" -b "dpaa_bus:fm1-mac10"

Created by Bartlomiej Nowak on 2021/04/08 11:41

Need help?

If you need help with XWiki you can contact: