Intona products are engineered and made in Germany. We ship daily worldwide.

IN3032UG: Ethernet Debugger User Guide

 

Introduction

The Intona Ethernet Debugger is a device to capture packets between two Gigabit Ethernet devices. It provides two ethernet ports, and each port forwards all traffic to the other port, as well as to a PC connected via USB. The intended purpose is low level debugging of anything above the ethernet physical layer, mainly using Wireshark and similar protocol analyzers. It helps when developing your own protocols layered on top of ethernet, developing your own MAC, or just for observing what is going on on your network.

Features

This device can log complete ethernet packets as received by the PHY. There is no processing of captured packets – preamble, SFD, and FCS are all left intact. Packets with incorrect CRC sums are not discarded. Ethernet packets which violate the specification are captured as far it is possible. Some normally invisible low level details are explicitly logged, such as interpacket gaps and CRC errors. Jumbo frames (ethernet packets longer than 1500 bytes) are supported and fully captured up to 16KB size.

Capture output is directly streamed to the PC. There is no kernel device driver. The device is accessed through a libusb userspace driver. You do not necessarily need elevated privileges. Installing the device will not destabilize your system. In particular, the device is not exposed as network device. This has the advantage that your OS will not mess with it. Neither will it attempt to drop or filter packets received through it, nor will it attempt to send random packets to it (ARP etc.). The latter would show up in Wireshark, and confuse your development efforts.

Capture can be directly started from Wireshark (if installed correctly). The userspace driver also provides a command line interface, which can be used to access advanced feature. An IPC interface is provided for use cases like scripting.

The debugger can block packets in one or both directions, corrupt packets, inject new packets. This is interesting for development and security research. (For example, you can test resilience of your ethernet connected device or software against random packet drops, test its behavior on flooding, or implement a network stack fuzzer.)

There are many other features. See Other features section.

Requirements

The software works on Windows, Linux, and macOS. We provide an installer for Windows. Windows 10 64 bit is required. Windows 7 may work as well (extra steps required, see installation instructions). For macOS, a homebrew tap is provided. For Linux, source code and build instructions are provided, which should work on any Linux distro.

USB 3.0 or later host and cable are recommended. USB 2.0 may work in low bandwidth scenarios. Using an USB hub, and/or connecting multiple USB devices to an USB hub/host may reduce the maximum bandwidth at which capture is possible without capture overflows.

Restrictions

Ethernet is intercepted by putting two PHYs between the two ports. There is no direct connection between the ethernet TX/RX wires of the ports. Each PHY negotiates the ethernet connection separately. No link can be established without USB power.

Old firmware (before 1.06) requires the user to set the ethernet speed manually if the devices connected to the debugger's ports negotiated different ethernet standards (for example 100 MBit vs. 1 Gb).

The PC needs to be fast to capture at full speed. Capturing in real time with maximum ethernet bandwidth directly to Wireshark or a slow hard disk may not be possible. (This is due to host performance problems outside of our control.) Packet buffer overflows should be expected when operating near maximum bandwidth. There is no hardware packet filter.

Firmware Changelog

The firmware version is reported in the bcdDevice field in the USB device descriptor, or can be determined by using the "hw_info" command in the host tool.

Firmware 1.09

Updating to host tool v1.5 is recommended.

Bug fixes

Firmware 1.08

Updating to host tool v1.3 is recommended.

Bug fixes

Potential incompatibilities and problems

Firmware 1.06

Updating to host tool v1.2 is recommended.

Bug fixes

Potential incompatibilities and problems

New features

Firmware 1.00

Host Tool Changelog

The host tool's version can be determined with the "hw_info" command, or simply running "nose --version".

Note: you can also look at the public git repository's commit log.

Host tool git master

Unreleased, but publicly available. (Also picked up by Homebrew formulae.)

May be referenced as host tool v1.6 in this document.

Host tool v1.5 (bfafb93, windows build 72)

Potential incompatibilities and problems

Bug fixes

New features

Host tool v1.4 (60a3f48, windows build 66)

Potential incompatibilities and problems

Bug fixes

New features

Host tool v1.3 (cfbdb79, windows build 64)

Bug fixes

New features

Host tool v1.2 (ff97f53)

Updating to firmware 1.06 is recommended.

Potential incompatibilities and problems

Bug fixes

New features

Host tool v1.1 (686fe4f)

Host tool v1 (f5eed9c)

 

Hardware Setup

The ethernet debugger needs to be powered via USB to forward any packets. Without power, communication between the two ports is blocked. Make sure USB cable and host port are at least USB 3.0 compliant. If you use USB hubs or USB isolators, make sure they all support USB 3.0. USB 2.0 will work, but will provide degraded functionality, as USB 2.0 bandwidth is lower than that of Gigabit Ethernet.

Attach the ethernet cables to the ports. Make sure you are breathing regularly. As soon as both port LEDs indicate a connection, and the negotiated ethernet speed matches between both ports, communication is possible.

LED Meaning

Port LEDs

The LED in use indicates the ethernet speed mode. If packets are sent or received, the corresponding LEDs will blink.

SpeedLED LLED R
1000 MBitoffon
100 MBitonon
10 MBitonoff
no linkoffoff

Note that it's possible for one port to have a link, while the other port does not. They also can have mismatched speed. In both cases, capture will not work.

Main LED

Normally, the main LED should be blue. It will blink if capturing is in progress. The following states exist:

LED stateMeaning
BlueUSB super speed connection is up
GreenUSB high speed connection is up (slow, but functional)
CyanUSB power only (will not work)
Blue blinkingcapturing at USB super speed is in progress
Green blinkingcapturing at USB high speed is in progress (will drop packets)
Blue/green blinkingthe blink_led command is being used (only for a short moment)
Redinitial bootloader failed (probably flash problem)
Red blinkingbootloader failed (probably flash problem)
Offlow level bootloader/firmware crash, or no USB power
Red/yellow blinkingfatal higher level firmware error
Red/blue or Red/green blinkingfirmware update failed; running factory firmware version
(The blue/green color indicates USB speed, see above.)

When you experience problems, it is useful to describe the LED state exactly in support requests, even if the observed variant is not listed above.

 

Software Installation

The software consists of a host tool, called "nose". It performs the following roles:

Linux

No binaries or packages are provided. You can build it from the public git repository. Binary builds or packages for popular Linux distributions may be provided if there is enough demand.

You may need to install an udev rule to get access to the USB device as normal user:

sudo cp udev.rules /etc/udev/rules.d/50-intona-ethernet-debugger.rules
sudo udevadm trigger

macOS

No binaries are provided. Hence the software is provided as source code, you can automatically download and build it using Homebrew. 

brew install --HEAD intona/ethernet-debugger/nose

Homebrew is a 3rd party project for installing freely available software on macOS. See https://brew.sh/ for details and how to install Homebrew itself.

You need to setup Wireshark extcap manually. See below.

We decided to not provide binaries for macOS because it is quite impossible to deliver unified, stable executables that will work on various releases of both software API and CPU types. Think of the platform change to ARM. This is no issue when compiling from sources using Homebrew.

Windows

Windows 10 or later 64 bit is required (but see Windows 7 section below). An installer is provided on the download page.

It does not matter whether the device is connected during installation. In fact, the installer does not try to access the device at all.

You can reinstall any time. Updating Wireshark might remove the Ethernet Debugger Wireshark integration. Reinstalling the Ethernet Debugger will restore it.

Windows 7

The device can be used with Windows 7. There are the following issues on Windows 7:

  1. Separate installation of a dummy device driver using Zadig is required. (See below.)
  2. Most Windows 7 installations are 32 bit only. The normal installer is 64 bit only, and will not run. A 32 bit build of the installer is provided separately on the download page.
  3. The host tool can not use as much memory for capture buffers, or may run out of memory. There is no solution for this, but as long as enough physical RAM is available, the default settings should not trigger an out of memory situation.
  4. Wireshark may stop working on Windows 7 in the future. In fact, they offer no 32 bit builds for the latest Wireshark stable release.
  5. There may be compatibility issues in the host tool, that are not present on Windows 10. There may be compatibility issues with older Wireshark versions, which you will be forced to use on Windows 7.

As noted above, you need to install a device driver with a 3rd party tool, or the host tool won't be able to access the device. This is not necessary on Windows 10, because we use libusb, and our device uses a feature, that essentially tells Windows 10 that it's a libusb-based device. (A technical explanation can be found here.) On Windows 7, a dummy device driver is required, which contains no executable code, and merely tells Windows that it needs to use the Microsoft WinUSB driver (used by libusb) with the Ethernet Debugger. Since signing and installing a device driver is a lot of work, the host tool installer has no support for this. Fortunately, a simple 3rd party tool called Zadig exists: https://zadig.akeo.ie/ This tool can generate and install a dummy driver for any libusb-based devices. Follow the instructions on its website, select the Ethernet Debugger device entry, and install a driver. Then capture should work.

 

Verifying Device Access

A simple way to verify whether the software works is by simply running the host tool. It is a command line program. On Windows, double-clicking nose.exe will open a terminal window, while on Unix, you need to open a terminal window manually, and then run nose in it.

If the installation succeeded, and the device is connected, you should see the following:

Device 2:3:7 opened.
PHY A: link=down speed=0MBit
PHY B: link=down speed=0MBit
Warning: no link.

The example above has the device on USB bus 2, port 3, device address 7.

If the device could not be found or accessed, the following is shown:

No devices found.

You can stop the host tool by closing the terminal or by entering the "exit" command.

Wireshark extcap Setup

Wireshark is the recommended way to use the Ethernet Debugger. It is 3rd party software and not developed by Intona. Download and install it from Wireshark's website: https://www.wireshark.org/download.html

Normally, the Windows installation procedure installs our host tool as a Wireshark "extcap". In short, "extcap" allows external programs (such as our host tool) to provide a capturing source. See the Wireshark extcap section for details. If the host tool is not correctly installed as extcap source, you will not be able to start capture from the Wireshark GUI (but other methods of capturing will still work.)

The host tool supports extcap directly via special command line parameters. It must be located in the "extcap" sub directory within the Wireshark installation directory or the user's Wireshark configuration directory. The paths depend on the operating system and the Wireshark installation location.

You can confirm whether it's installed correctly by opening the Wireshark about dialog, and switching to the "Plugins" tab. There should be an entry named "nose".

Old Wireshark versions

The non-global/user-specific extcap paths below require at least Wireshark 3.1.1. Older releases support global paths only.

Linux, macOS

It is recommended to create a symlink to the host tool. The global path is something like /usr/lib/wireshark/extcap/ or /usr/lib/x86_64-linux-gnu/wireshark/extcap/. The exact path depends on the Linux distro. The user-specific path is usually ~/.config/wireshark/extcap/.

The following should install it locally, assuming "nose" is already installed:

mkdir -p ~/.config/wireshark/extcap/
ln -s `which nose` ~/.config/wireshark/extcap/nose

macOS (app bundle)

The following is useful if you want to install the extcap globally, or on older Wireshark versions, assuming "nose" is already installed via Homebrew:

ln -s `which nose` /Applications/Wireshark.app/Contents/MacOS/extcap/nose

The /Applications/Wireshark/ part of the path can be different, depending on where exactly Wireshark is installed. Check the Wireshark about dialog ("Folders" tab) if you are unsure.

You need to start Wireshark at least once before you run the above command. Otherwise, macOS may show a security warning, and it won't work.

Windows

Since Windows does not support symlinks properly, it is recommended to create a .bat file in the Wireshark extcap sub-directory. The installer creates a file named intona-ethernet-debugger.bat with the following contents (assuming default paths and English locale):

"C:\Program Files\Intona\Ethernet Debugger\nose.exe" "%*"

This "redirects" all invocations to the actual installation location.

The user-specific path is C:\Users\USERNAME\AppData\Roaming\Wireshark\extcap\. Replace USERNAME with the actual username. You may need to create the last component of the path. The installer does not try to use this.

Updating Wireshark tends to remove and recreate the Wireshark installation directory. This will also remove the intona-ethernet-debugger.bat file created by the Ethernet Debugger installer. You could run the installer again to fix this. Manually moving the .bat to the Wireshark user-specific configuration path mentioned above avoids this.

Firmware Update

Firmware updates may be required to get new features and to apply bug fixes. Normally they are not necessary. Applying such an update must be done explicitly. The update process rewrites the device's flash memory, and should not be interrupted. Make sure the device is connected via USB 3.0, as the update will take a long time with USB 2.x. Firmware downloads are available here.

Unix-like

Plug in the device, and ensure it's using USB 3.0 (main LED is blue). Then run the following command on the terminal:

nose --firmware-update Downloads/firmware.dat

Where Downloads/firmware.dat is the path to the firmware binary you downloaded.

If the firmware file is accepted, and the device is accessible, something like this will appear:

Firmware file: version 1.08
Select firmware update action:

  Choice   Address   Serial           Firmware version
 -------------------------------------------------------
  4        2:7:12    08900037         1.06                          
 -------------------------------------------------------
  a        Update all devices with outdated firmware
  b        Force update all devices (dangerous)
  c        Do nothing and exit
 -------------------------------------------------------

Enter your choice: 

If you type 4 followed by the enter key, the device 08900037 will be updated. The tool will exit when the update is finished or aborted. On success, the device restarts on its own, and runs the new firmware immediately.

If update fails, the device will boot from a fallback factory image, and indicate the failure by flashing the main LED red. You should retry the update, or if the tool fails again, contact support. If update is slow and takes longer than at most 1 minute, make sure the device is connected via USB 3 instead of USB 2.

You can confirm successful update by comparing the device version number (bcdDevice or checking the output of the hw_info command) with the version indicated by the update.

If the --device option is provided, the tool will update the given device without asking for confirmation. --firmware-update-all will update all detected devices without asking for confirmation. In both cases (at least with v1.2) the firmware is written only if it's newer than the firmware on the device, unless --firmware-update-force is provided.

Old host tool versions (before release v1.2) do not ask for a choice, but start the update with the first device found without confirmation. It is recommended to download and install the newest host tool before performing a firmware update.

Windows

The same instructions as with Unix apply. You can double-click "firmware-update.bat" in Explorer in the Ethernet Debugger installation folder to avoid constructing a command line. This will use the firmware file that came with the installer (it's in the same folder). The latest installer usually comes with the latest release of the firmware. Installers for host tool v1.0 (build 53) do not have this yet; download a newer version here.

(The host tool never "phones home", and there is no automatic update over internet.)

 

Capturing

The main purpose of the ethernet debugger is to capture packets. The following methods are available.

Wireshark extcap

If you open Wireshark, it should display any plugged in Intona Ethernet Debuggers as Ethernet Debugger USB (08900037) in the list of capture interfaces. The 08900037 in the brackets is the serial number (as in the USB device descriptor). (Some versions of Wireshark also show the device address in the format used by the host tool.) Double click this entry, and Wireshark should start capturing. The Ethernet Debugger's main LED will start blinking.

There is no hotplug mechanism for Wireshark extcaps. If you connect or disconnect devices while Wireshark is running, you may need to press F5 or restart Wireshark to update the device list of Ethernet Debugger capture devices.

Note that if the bandwidth utilization is high, the internal FIFO may overrun, leading to lost packets. The host tool adds a packet comment to the first packet after a run of dropped packets.

It may also happen that Wireshark freezes if the amount of data is too large, because the GUI requires a large amount of resources to deal with packet input. (Capturing to disk via the "nose" tool may help reducing packet drop. You can open the capture file with Wireshark afterwards.)

Various error conditions may deadlock Wireshark and the host tool on capture start.

Capturing Options

Wireshark lets you set some Ethernet Debugger specific options before starting capture. Click on the gear-like symbol left of the Ethernet Debugger interface in Wireshark's Capture interface list.

Wireshark extcap Toolbar

The host tool provides a toolbar in Wireshark. This is implemented through the extcap mechanism. It is slightly clunky due to Wireshark restrictions (all GUI code is provided by Wireshark, and not everything can be realized). The toolbar can be shown by enabling it in the Wireshark "View" menu, "Interface Toolbars" sub-menu.

Directly Starting Wireshark from Host Tool

You may use the --wireshark option of the host tool to start Wireshark and capturing in one go. It attempts to find the installation path of Wireshark, sets up a named FIFO, and starts a new Wireshark process.

Example
nose --wireshark

Lifetime of Wireshark process on Unix

Since host tool version v1.2, Wireshark is not terminated anymore if capturing ends or nose is exited. Older versions always terminated it due to being in the same process session.

Statistics

The host tool --capture-stats option can be used to enable regular statistic updates on the terminal. The "set capture-stats true" command can be used to do this at runtime. (You can enter this command on the Wireshark extcap toolbar, for example.)

Capturing to a File

The host tool --fifo option can be used to capture either to a real file on disk, or a named FIFO. The capture_start command is similar, and can be used to start capturing via the host tool command line or IPC interface. The format of the output is PcapNG (see https://pcapng.github.io/pcapng/). You may use the third party open source libpcap library to parse such files. If you use an actual FIFO, you can stream in real time.

Note that if you capture to disk, overruns can happen due to waiting on disk I/O. The host tool tries to avoid this by using decoupled memory buffers, but these may be slowly filled up, until a software overrun happens.

Example
# Capturing to a file until Ctrl+C is hit, and log capture statistics to stdout.
nose --capture-stats --fifo target_file.pcapng

# Manually starting Wireshark.
# On terminal 1:
mkfifo /tmp/fifo
nose --fifo /tmp/fifo
# On terminal 2:
wireshark -k -i /tmp/fifo

Selecting the Device

If you have multiple Ethernet Debuggers, the --device option can be used to pick a specific device. Passing the special value help to this option lists all devices that were found.

Multi-Capture

Selecting multiple devices at once is not possible. However, if extcap is correctly installed, you can select multiple capture devices in Wireshark. This will provide a merged view of data coming from multiple devices and host tool instances.

Configuring the Buffer Size

The --capture-soft-buffer and --capture-usb-buffer can be used to fine-tune the sizes of the fixed size buffers allocated on the host. Raising them may reduce buffer overruns on the host PC.

 

Other Features

PoE Passthrough

Both ethernet ports are capable of handling Power over Ethernet (PoE) as specified in IEEE 802.3af, 802.3at and 802.3bt. Power is passed through in both directions without interception.

PTP Timestamps

The device provides high resolution timestamps for packets. This can help to debug PTP related issues, or any other timing issues. These timestamps are in nanoseconds with 10 ns resolution.

Each PHY has its own FIFO, which may affect accuracy. In addition, device-internal CDC may affect the accuracy. Internally, the timestamps are generated by a 100 MHz clock and are passed to the ethernet PHY's (RGMII) clock domain, which introduces jitter. The timestamps are relative to device start.

The host tool capture output adds a start offset to the timestamps to make them roughly line up with wall clock times. However, this offset is not precise, and there is no time correction. The absolute time of a packet event might be slightly different from real time. The longer the capture is running, the higher the deviation will be.

IN3083WP has additional information.

Supported Command Line Options

You can list supported options as follows:

nose --help

Current list of options:

NameDescription
--verbosity 0|1|2Set verbosity log level: 0 silent, 1 normal/default, 2 verbose messages
--versionPrint host software version and exit
--selftestRun internal self-test. (Requires a loop between the two ports.)
--selftest-serialInternal.
--wiresharkStart wireshark and dump packets to it. Terminate once done.
--device nameOpen this device. (Pass "help" to get a list. "none" prevents automatic opening of a device.)
--firmware-update filePerform a firmware update using this file.
--firmware-update-allUpdate firmware of all devices that have been found. (Since v1.2.)
--firmware-update-forceUpdate firmware even for devices which have the current or newer firmware version. (Since v1.2.)
--fifo fileStart capture and write to the given file or fifo. (Overwrites the target if it's a file.)
--ipc-connect pathConnect IPC to this socket/named pipe. Terminate on disconnect.
--ipc-server nameHost IPC on this socket/named pipe.
--capture-soft-buffer numCapture soft buffer (in bytes, accepts kib/mib/gib suffix)
--capture-usb-buffer numCapture libusb buffer (in bytes, accepts kib/mib/gib suffix)
--capture-statsShow capture statistics every 1 seconds.
--extcap-*Various options for use by the Wireshark extcap mechanism.
--captureUsed by Wireshark; ignored by host tool.
--strip-framesStrip preamble, SFD, and FCS from ethernet frames. (Changes pcapng output to LINKTYPE_ETHERNET.)
--cmd commandsRun commands on opening. (Must be a single string, separate commands with ; ) (Since v1.3.)
--run commandsRun commands after opening. (Syntax like --cmd.) (Since v1.3.)
--exit-onControl when exactly the tool should exit automatically. (Since v1.3.)
--exit-timeoutAlways exit after the given time of seconds has expired. (Since v1.3.)
--capture-speed-testTest how much USB bandwidth capture needs (discards capture data). (Since v1.5.)

Interactive Command Line

The host tool has an interactive command line interface. When starting the host tool without commands, it will wait for commands from the terminal. You can use the "help" command to list available commands and their parameters. Many advanced features (such as listed below) are accessible only through this interface.

By default, the host tool will read from stdin and accept commands. It will terminate if stdin is closed or returns EOF. If run on the terminal, it offers an interactive command line using libreadline. (If the host tool was built without libreadline, or you can use the rlwrap 3rd party tool to get comfortable line editing and history:

rlwrap nose <arguments for nose>

Scripting

The host tool can be used for scripting. The --run and --exit-on options (available since host tool v1.3) can be used to run individual host tool commands.

Example
nose --run 'inject A --file mypacket.dat' --exit-on always

(The --exit-on is needed to make the tool exit after running the command.)

For more complicated use cases, the IPC interface offers and out-of-process API. (See IPC Interface section.)

Supported Commands

NameDescription
blink_ledFlash main LED
block_portsBlock or unblock all packets
capture_startStart capturing to a file or FIFO
capture_stopStop current capture
cfg_packetSend internal command to device
device_closeClose the current device
device_listList all Ethernet Debuggers connected to this PC
device_openOpen a device
disruptSetup packet disruption and port blocking
disrupt_stopDisable packet disruption and port blocking
exitExit the host tool
helpShow all commands
hw_infoShow device information (including firmware version etc.)
injectSetup packet injector
inject_stopDisable packet injector
mdio_readRead a PHY's MDIO register
mdio_writeWrite a PHY's MDIO register
reset_device_settingsRestore default settings on the currently opened Ethernet Debugger
setSet a command line parameter
set_device_phy_waitConfigure PHY speed negotiation delay time
speedConfigure PHY speed negotiation mode
latency_tester_senderSetup device as latency tester sender (Since v1.3)
latency_tester_receiverSetup device as latency tester receiver (Since v1.3)

Some commands are described in further detail in the following sections. The help command lists parameters accepted by each command.

IPC Interface

The command line interface is available via IPC (unix domain sockets on UNIX, named pipes on Windows). This may be useful for scripting or creating GUI frontends. For example, you could inject or drop packets under specific situations, or start/stop capturing at specific times

The IPC server created with --ipc-server is available as long as the host tool runs. The path, at which the IPC communication channel is available, depends on the OS. (The tool prints the full path when it starts the server.)

The --ipc-connect option makes the host tool initiate the IPC connection to the given path. This connection behaves the same as a connection made to a server started with --ipc-server. This mode may be more convenient with certain frameworks, for example when the framework makes it easy to start a server.

Windows

The IPC communication channel is a local win32 named pipe (CreateNamedPipeA()). For example, "nose --ipc-server foo" will create the named pipe \\.\pipe\foo, which can be accessed by other programs in read/write mode. (Starting with host tool v1.3 you can simply pass a full path.) For testing, a third party program such as PuTTY can be used (enter the full named pipe path as a serial device; unfortunately, configuring the terminal to act decently is hard: by default there is no local echo, and CTRL+j must be used to send the required \n ASCII 10 line terminator).

With --ipc-connect, the argument is a full path, which will be passed to CreateFileA(). In theory it can be anything, as long as access with FILE_FLAG_OVERLAPPED works.

UNIX (Linux/Mac)

The IPC communication channel is a Unix domain socket located on /tmp. For example, "nose --ipc-server foo" will create the socket /tmp/too.socket. (If the filesystem entry already exists, it will be deleted, even if it's a normal file.) Since host tool v1.3, the argument can be a full path too (if / or \ appears in the argument, it's considered a full path). For testing, the third party program so cat can be used to interactively send commands:

socat /tmp/foo.socket -

--ipc-connect takes a full path. Both sockets are supported (connect() is used) and other types of files (open() is used).

Protocol

You can send commands in text or JSON form (the help command lists available commands and shows the basic syntax). The protocol uses 1 JSON object per line (in both directions of communication). A line is terminated with \n (ASCII line feed, byte 10). It is fairly self-describing:

IPC example
{"command":"mdio_read", "phy":1,"address":1,"id":1}   // client to host tool
{"id":1,"result":31049,"success":true}                // host tool to client

The id field is an arbitrary integer chosen by the client, and can be used to associate requests with replies. The C-style comments are for illustration, and not part of the protocol.

For example, on the shell and with socat,  you could do the above by:

Terminal 1
$ nose --ipc-server test
Terminal 2
$ echo '{"command":"mdio_read","phy":1,"address":1,"id":1}' \
    | socat -  /tmp/test.socket > /tmp/output.txt
$ cat /tmp/output.txt
{"id":1,"result":31081,"success":true}

A proper client can send multiple commands to the same connection. Replies are always sent immediately, though certain conditions like USB communication failing will block replies until a timeout. Replies are always sent back in order (it cannot happen that the reply for the first request comes after the reply for the second request). Events (such as log messages) can appear at any time, even between request/reply messages.

Log messages (such as output when PHY links change their state) are wrapped into special JSON messages:

Log message example
{"type":"log","msg":"PHY 1: link=up speed=1000MBit\n"}

Identify Function

The blink_led command will flash the device main LED blue/green for a moment. This is helpful to determine which device is opened on a host tool instance if multiple Ethernet Debuggers are connected to a PC.

Latency Tester

Introduction

This feature (available since host tool v1.3) involves sending generated ethernet packets from one Ethernet Debugger to a second one, and measuring the delay. This can be used to test the latency if a 3rd device, using a setup like this:

Ethernet Debugger 1 is the sender, Ethernet Debugger 2 is the receiver. The sender generates packets that get sent out of port A and B at the same time. DUT is expected to propagate the packet unchanged to the receiver. The receiver measures the difference in arrival time between port A and B.

The Latency Tester feature takes care of generating/sending the packets and analyzing them on the receiver side. It writes the computed delay time to a text file. The packets use a (hopefully) unused EtherType (0xBEEF). The feature as currently implemented can test at most Layer 2 devices (like ethernet switches). Testing devices that operate on a higher layer (such as IP routers) are not supported, as that would require implementing parts of ARP and IP (though it's theoretically possible).

Instructions

Connect the devices as in the diagram above. On the PC, open two "nose" instances, one for the sender, one for the receiver. Use the device_list command to confirm whether the correct device is opened (change it with the device_open command). Then run the latency_tester_sender command on the sender instance, and latency_tester_receiver on the receiver instance. By default the sender starts sending continuously with 1 packet per 1 ms, with a test sequence that lasts 10 seconds. The receiver ignores the packets until a start packet is detected, and then logs the status every 10 packets (it logs the delay for only the 10th packet, use file output to retrieve all data points).

Sender setup
Device serial1 / 4:2:15 opened.
PHY A: link=up speed=1000MBit (master)
PHY B: link=up speed=1000MBit (master)
> device_list
Devices:
 - 'serial1' (4:2:15) [opened]
 - 'serial2' (4:2:16)
Found 2 devices.
> device_open serial2
USB device closed.
Device serial2 / 4:2:16 opened.
Opening succeeded.
PHY A: link=up speed=1000MBit (slave)
PHY B: link=up speed=1000MBit (master)
> latency_tester_sender 
Latency tester: new test run (0)
Latency tester: new test run (1)
Latency tester: new test run (2)

If continuous sending is not desired, you can also just run "latency_tester_sender --once".

Receiver setup
Device serial1 / 4:2:15 opened.
PHY A: link=up speed=1000MBit (master)
PHY B: link=up speed=1000MBit (master)
> latency_tester_receiver 
Starting capture thread succeeded.
Receiver setup. Listening to incoming packets.
Starting recording sequence...
seq=99 diff=-410
seq=199 diff=-410
seq=299 diff=-410

The output gives you an idea whether it actually works. Actual data should be retrieved by using file output. File output can be used with for example:

Receiver file output
> latency_tester_receiver --out-file mydata.txt

You may want to use an absolute path here (perhaps using drag & drop from a file manager to get the path) if the concept of working directories is too uncomfortable.

The file is opened when a sequence starts, and closed when it ends. If the file already exists when a sequence starts, the file is not overwritten. Instead, the sequence is skipped repeatedly until the file is deleted or moved by the user. Example:

Terminal Output of Latency Tester Example Run
> latency_tester_receiver --out-file mydata.txt
Starting capture thread succeeded.
Receiver setup. Listening to incoming packets.
File 'mydata.txt' exists, skipping sequence.
File 'mydata.txt' exists, skipping sequence.
Opened file 'mydata.txt' for writing.
Starting recording sequence...
seq=99 diff=410
seq=199 diff=410
seq=299 diff=410
seq=399 diff=410
seq=499 diff=410
seq=599 diff=410
...
seq=9899 diff=420
seq=9999 diff=410
Recording sequence finished.
Problems detected: no
File 'mydata.txt' closed.
File 'mydata.txt' exists, skipping sequence.
File 'mydata.txt' exists, skipping sequence.

The file mydata.txt was deleted shortly after the receiver was started. The file was then created again and opened for writing. The host program picked up the next sequence of test packets and wrote them to mydata.txt. The sequences after that are skipped because mydata.txt still exists. (It will log two skip messages per sequence, because it encounters a first packet from both paths.) The idea behind this behavior is that tests can be repeated without much interaction, simply by renaming the output file (maybe giving it a more descriptive name), without the danger of accidentally overwriting files.

The output file simply contains the differences in receive timestamps in nanoseconds, with 10 ns resolution:

Example Output File (Partial)
420
430
410
410
420
410
420
430
410
420
420
420
420
420
420
420

These differences should be roughly equivalent to the latency the DUT incurs on ethernet. The values are positive if the latency values on port A are higher than on port b.

See the "PTP Timestamps" section for details on the quality of the timestamps.

By default, 10000 samples are sent (depending on sender configuration), and the receiver reports "Problems detected: no" only if all samples were included. If you see errors logged on the terminal, there is probably some sort of problem due to packet drop or corruption.

Packet Injection

It is possible to manually inject new packets into the ethernet connection. As no actual network interface is provided, OS mechanisms (such as sockets) cannot be used. Using the host tool is required.

The inject command provides this functionality. It is possible to send arbitrary ethernet packets, including packets that are not spec-compliant. This is not a high speed send path – full ethernet bandwidth cannot be reached. (Although you can instruct the command to repeat packets, in which case it will produce a high bandwidth stream of the same packet being repeated.)

Use inject_stop to disable the injector again. Use hw_info to check whether it's currently enabled.

CRC errors can be injected. Degenerate packets (too short, IPG too low) can be created with the "raw" parameter. Low level physical layer coding errors can be generated with the "gen-error" parameter (PHY emits symbol error for a specific byte). The IPG can be controlled with the "gap" parameter, which will set the distance to other generated packets as well as packets that are normally transferred through the wire.

Since host tool v1.3, packet data can be read from files with "--file filename".

Since host tool v1.5, the "--bw-packets" and "--bw-bytes" parameters can be used to repeat the packets at a specific bandwidth. --bw-bytes accepts "kib" or "mib" asd suffix for convenience. Using either of the parameters overwrite the "gap" parameter with a value needed to reach the specified bandwidth, and "num" is set to inf. If the "num" parameter passed by the user is not 1 (the default) or inf, or "loop-count" is used, the parameters are silently ignored. If the requested bandwidth cannot be achieved in theory, the command fails. If there is other communication on the ethernet line, the actual bandwidth may be different. If the ethernet speed mode changes, the command has to be run again to set the correct parameters. The bandwidth is computed using the full physical layer packet, and includes preamble/SFD/padding/FCS in the byte count.

Use "help inject" to list all parameters.

Using the command may drop packets coming from the opposite source port. The injection logic will wait until transmission is turned off, then it will inject the packet, and fully drop any other packets from the opposite port.

Example
# Inject a packet that starts with a AB:CD:12:34:56 dest. MAC,
# the rest of the packet filled with 0s, on port A (into the 
# stream of traffic flowing from B to A):
inject A ABCD123456

Command parameters

NamePossible valuesDescription
phy"A", "B", "AB"Port to affect.
dataHex string, like "ABCD12"Packet payload data to append (interpreted as hex string, converted to bytes). E.g. "ABCD" is converted to the 2 byte sequence 0xAB, 0xCD. (Default: empty)
raw"true", "false"If true, do not add preamble/SFD/CRC to the user provided payload data. (Default: false)
numInteger or "inf"Repeat mode: how many packets to inject until operation stops. (Default: 1)
gapIntegerMinimum interpacket gap before/after the injected packet in bytes. (Default: 12)
append-randomIntegerAppend this many random bytes to the payload. The random bytes the same for every injected packet if repeat mode is used. (Default: 0)
append-zeroIntegerAppend this many zero bytes to the payload. (Default: 0)
gen-errorIntegerOffset of byte for which the PHY should generate a symbol error. The offset is into the final, physical layer ethernet packet. A value of -1 disables this. (Default: -1)
fileString, path to a fileIf not empty, this string is opened as a file, and its contents are used as packet payload data. The "file" or the "data" parameters cannot be used at the same time (one of the parameters have to be set to empty or not provided). Unlike the "data" parameter, the file data is not interpreted as a hex string. (Default: empty)
loop-countIntegerRepeat the payload this often after the end of the packet. Obscure feature to test over-large packets. (Default: 0)
loop-offsetIntegerFor use with "loop-count". Byte offset into the packet after which data should be repeated. (Default: 0)
nopad"true", "false"If true, do not pad short packets to mandatory packet length. (Default: false)
bw-bytesInteger, or byte amount ("1kib", "1mib")Rate of raw bytes per packet. If not 0, this command overwrites the "gap" parameter to reach the requested rate. Packet repeat ("num" parameter) is set to "inf". (Default: 0)
bw-packetsIntegerSimilar to "bw-bytes", but the parameter specifies the number of packets per second to reach. Ignored if "bw-bytes" is set. (Default: 0)

The help inject output has some details on allowed value ranges.

Blocking Ports

The block_ports command blocks all traffic through the device in a specific direction:

CommandEffect
block_ports ABlock traffic from B→A, unblock A→B
block_ports BBlock traffic from A→B, unblock B→A
block_ports ABBlock all traffic
block_ports noneUnblock all traffic

This command uses the same hardware logic as the disrupt command (basically it's just a simpler version of the same command). Using either resets the state set by the other command.

Packet Disruption

The disrupt command can be used to drop or to destroy some or all packets in a certain direction. This is a form of error injection suited for testing reliability of low level protocols. It can either flip a single bit in a packet at a user-chosen byte offset, or discard entire packets.

(Restriction: cannot drop specific packet according to filter-like matching criteria etc.)

Use disrupt_stop to disable disruption again. Use hw_info to check whether it's currently enabled.

Example
# Disrupt a number of packets for a short time:
#  B                    on port B (traffic flowing from A to B)
#  --mode drop          just drop the packets (instead of creating CRC errors)
#  --num 20             drop 20 packets in total
#  --skip 10            drop only every 10th packet (this makes it active for ~200 packets)
disrupt B --mode drop --num 20 --skip 10

(The syntax changed in host tool v1.2.)

Command parameters

NamePossible valuesDescription
phy"A", "B", "AB"Port to affect.
mode"drop", "corrupt", "err"Operation mode. (Default: "drop")
  • "drop": drop packet completely
  • "corrupt": flip a single bit in the packet (see "offset" parameter; the bit cannot be selected, and is chosen by a pseudo random number generator on each packet)
  • "err": turn a packet byte into a symbol error (see "offset" parameter)
numInteger or "inf"Repeat mode: how many packets to disrupt until operation stops. (Default: 1 - disrupt only one packet)
skipIntegerIn repeat mode, do not disrupt this number of packets before disrupting another packet. The skipped packets do not count towards the total number of packets to disrupt as set by the "num" parameter. (Default: 0)
offsetIntegerByte to affect with the "corrupt" or "err" operations. The offset is into the final, physical layer ethernet packet. (For example, 0 would disrupt the first byte of the preamble.) Ignored in "drop" mode. (Default: 20 - this is the ethertype field)

The help disrupt output has some details on allowed value ranges.

MDIO Access

Raw MDIO access

The mdio_read and mdio_write commands provide direct access to the PHY's MDIO registers. Access to register 22 is intercepted/emulated by the firmware.

Changing PHY Ethernet Speed

Ethernet speed is controlled by the PHYs. By default, they are set to use a common speed, and the firmware automatically adjusts the PHYs to use the slowest negotiated speed of either PHY (devices with firmware 1.00 do not support this behavior - you need to set the speed manually).

You can use the speed command to force specific speed modes on both PHYs:

SpeedCommand
1000 MBit (full duplex)speed 1000
100 MBit (full duplex)speed 100
10 MBit (full duplex)speed 10
1000 MBit (half duplex)speed 1000half
100 MBit (half duplex)speed 100half
10 MBit (half duplex)speed 10half
Highest common speed mode of both PHYspeed same
Manual controlspeed manual

The same mode puts the PHYs into auto negotiation mode at first, and if the speed modes mismatch, the faster PHY is forced into a lower speed mode. If a link goes down and up again, the process is repeated. Technically, the process can re-establish the link multiple times, which makes it slower and may be annoying when troubleshooting connected devices. The set_device_phy_wait command can be used to set the fixed time after which the state machine assumes the PHY cannot establish a link.

The manual mode lets each PHY negotiate the speed independently, which will lead to problems if the device's port A and B PHYs do not negotiate the same speed. This mode is useful only if you want to manipulate the MDIO registers directly, or maybe for checking what each link ends up with auto negotiation.

The values set by the "speed" and the "set_device_phy_wait" commands are stored as permanent settings on the device.

Older host tool versions do not have the "same" mode implemented, and require the user to manually correct speed modes. Even older versions do not even offer the "speed" command, and require using manual mdio_write commands. In addition, the "same" mode requires at least version 1.06 firmware. In addition, the half duplex modes are available only since version 1.08 firmware. Older versions will always use full duplex, and disable auto negotiation, which tends to make connected PHYs go into half-duplex mode.

Device Settings

Since firmware 1.06, the device persistently stores some settings on its EEPROM. You can inspect these settings with the "hw_info " command. You can clear all persistently stored settings with the "reset_device_settings" command. Power cycle the device after running the latter command successfully in order to clear any runtime state (PHY registers, inject/disrupt features).

Known Problems

If you encounter problems, always update to the latest firmware version and host tool, available here.

Further Readings

White Papers

Ethernet Debugger Timing

The Ethernet Debugger can be used to intercept and capture traffic between two Ethernet devices. The device is normally not supposed to affect the Ethernet traffic itself. But for technical reasons, the device can affect the traffic by introducing additional latency and jitter compared to a simple Ethernet cable. This effect is minimal, but can matter for real time applications such as PTP. The goal of this document is to provide information about the introduced latency and jitter, as well as actual measurements.

Online-Link, PDF-Download

Statements

Letter of Volatility

This letter describes volatile, non-volatile, and storage media on the specified product. Customers can use this document to comply with security requirements and equipment handling procedures.

Online-Link, PDF-Download

Document version: 114 / Mär 04, 2024 10:39

Download this document as PDF

.
.