SAE J1939 To USB Gateway Records J1939 Data Frames (PGNs) To Micro SD Card

As of today, we have accomplished another milestone toward full SAE J1939 data recording using our  SAE J1939 Gateway Module With USB Port, RTC, MicroSD Memory Card. The device's firmware now allows the recording of SAE J1939 data frames (PGNs) to the onboard microSD card without requiring a PC connection. The previous version supported the data recording onto the connected PC, and that functionality is still available.

The jCOM.J1939.USB processor board is a high-performance, low-latency vehicle network adapter for SAE J1939 applications. It allows any host device with a USB COM port to monitor SAE J1939 data traffic and communicate with the SAE J1939 vehicle network.

The board supports the full SAE J1939 protocol according to J1939/81 Network Management (Address Claiming) and J1939/21 Transport Protocol (TP). It comes with an extensive programming interface for Windows and Linux/Ubuntu applications, including full C/C++/C# source code for short time-to-market developments.

Hardware Test Configuration

The above image describes our hardware test configuration. We use one of our  SAE J1939 ECU Simulator Board With USB Port (JCOM.J1939.USB-B) devices to simulate the data traffic. The JCOM.J1939.USB-B board served as the basis for the development of the advanced JCOM.J1939.USB-X, which provides additional features such as Real-Time Clock (RTC) and an onboard microSD card.

Note: The JCOM.J1939.USB-B board does not support data recording.

We set up the data traffic using our JCOM1939 Monitor program as shown here:

All PGNs, 65280 through 65284, were chosen randomly, as were the frequencies associated with them. The primary purpose was to create a sample SAE J1939 network.

SAE J1939 Data Recording Programming

We use the onboard microSD card to store SAE J1939 data traffic and to set up the recording configuration. For that purpose, we extended the JCOM1939 Monitor software by adding the "Gateway Recorder" section, as shown in the following image:

The setup allows defining which PGN to record and which sample frequency to use. A sample frequency of zero indicates that the recorder should store the data immediately, i.e., according to the frequency defined by the SAE J1939 Standard.

This model assumes the recording of selected PGNs (message filtering) for diagnostics and analysis purposes. Depending on the number of PGNs, the maximum recording time can be between 4 to 12 weeks at 8 hours/day using a 16 GB microSD card.

Note: We are in the process of adding another model to support reverse engineering, which requires the recording of the entire bus traffic without filters. Depending on the busload, the maximum recording time will be between 5 to 15 days at 8 hours/day using a 16 GB microSD card.

Once the board is programmed (clicking the "Program Gateway" command button), the PC connection can be removed, and the gateway can be connected to a running SAE J1939 network (as simulated in our setup). Upon the next power-up cycle (Reset), the gateway will record the PGNs as defined in the setup without a PC connection.

The board will resume regular gateway operation as soon as it is re-connected to the PC running the JCOM1939 Monitor software.

Data Record Retrieval

At this time, data retrieval is accomplished by removing the SD card from the board and using a PC card reader.

The recorder function creates regular ASCII text files to provide readability. The filename include the recording date, meaning there will be one file per day and a new file will be created at midnight (precision is plus/minus five seconds without loss of data). 

The file format is, however, MS-Excel-compatible. The screenshot below shows an example of one of our recording sessions.

Note: It almost appears that only PGN 65282 (FF02) is being recorded. However, this PGN is sampled at the highest frequency (20 msec). We checked, and the recording session does sample all PGNs as specified.

The data columns include time of day, milliseconds, Data Page, PGN, destination address (DA), source address (SA), and the data.

Note: We separated the milliseconds from the time of day to maintain a basic level of MS-Excel-Compatibility (Excel does not support time formatting down to milliseconds).

The next screenshot shows the same data as read by MS-Excel:

However, in order to create this view, we made a few modifications: We renamed the file from .txt to .csv to establish Excel-compatibility. We formatted the time column (HH:MM:SS) according to Excel format. Lastly, we formatted all columns to left margin and adjusted the column width.

The test setup created text file sizes of 250MB per 24-hour recording session, which translates into roughly 48 days recording capacity at 24 hours per day using a 16GB SD card.

What's Next?

We will release the new JCOM.J1939.USB-X firmware plus the updated JCOM1939 Monitor software by December 4, 2020 (we need the time to update the documentation).

In regard to the JCOM.J1939.USB-X board, we are working to implement an SAE J1939 data traffic simulator that operates without a PC connection with programming similar to the recording function.

We are also working on a more sophisticated data retrieval using a USB memory model plus a "replay" feature that allows to feed the recorded data back into a network for real-world simulations.

It appears, however, that these features will exceed the board's capacity in regard to memory size and execution speed, i.e., we will focus on our  SAE J1939 Gateway And Data Logger With Real-Time Clock for such extensions.

For more information, please feel free to contact us anytime.