1.1.5. com

1.1.5.1. Wrapper

This fox.py module implements a modular, process-based communication wrapper, designed for use in Python script that require robust and flexible data exchange via files or communication interfaces typically used in the battery and IoT applications.

Main Features:

• Unified Interface: Provides a common interface for different communication backends (e.g., file, MQTT), which can be easily integrated into user scripts.

• Process-based Architecture: Each communication backend (file or MQTT) runs in its own managed process, ensuring non-blocking operation and improved reliability through multiprocessing.

• Inter-Process Communication: Utilizes multiprocessing.Queue for safe, efficient message transfer between user code and communication processes.

• Event-based Control: Startup, readiness, and shutdown of communication processes is controlled via events, ensuring predictable and robust lifecycle management.

• Extensible: Easily add new communication backends by implementing the provided abstract base classes (ComInterface, ProcessInterface).

How it works:

• When you instantiate a communication object (e.g., File or MQTT), you provide a name and a parameter object describing the configuration.

• Calling .start() launches the appropriate communication process(es) in the background.

• Use .read() and .write() methods to exchange data with the communication backend. These methods interact with the process-safe queues and handle process health checks automatically.

• Call .shutdown(block=True) to terminate all background processes cleanly when done.

Module Structure:

• __init__.py: Declares the package and provides the high-level package docstring.

• com_interface.py: Defines the base classes for all communication interfaces and processes, handling process control, lifecycle, and inter-process events.

• can_com.py: Implements CAN-based communication with a background process managing the CAN connection.

• file_com.py: Implements file-based communication via separate reader and writer processes.

• mqtt_com.py: Implements MQTT-based communication with a background process managing the MQTT client connection and message routing.

• parameter.py: Contains all data classes for configuration and process control (including ComControl, MQTTParameter, FileParameter, and CANLoggerParameter).

This framework is particularly useful for applications that require decoupled or parallel data transfer.

1.1.5.1.1. Parameter Objects

• FileParameter

  • input_file (str or Path, optional): File to read from.

  • output_file (str or Path, optional): File to write to.

  • encoding (str): File encoding (default: “utf-8”).

• MQTTParameter

  • broker (str): MQTT broker address.

  • port (int): Broker port.

  • subscribe (list of str): Topics to subscribe to.

  • tls_cert (str, optional): Path to TLS certificate.

  • username (str, optional): MQTT username.

  • password (str, optional): MQTT password.

• CANParameter

  • interface (str): Used CAN interface.

  • channel (str | int, optional): Channel name or number.

  • bitrate (int, optional): Bitrate used for CAN communication.

  • dbc (Path, optional): Path to a .dbc file used for encoding/decoding.

1.1.5.1.2. Example

File Communication Example

# Setup parameters for file communication
file_para = FileParameter(
    input_file="input.txt",
    output_file="output.txt"
)
file_com = File("File Communication", file_para)
file_com.start()

# Read lines from input file (if input_file is set)
line = file_com.read()
while line:
    print("Read from file:", line)
    line = file_com.read()

# Write a line to output file (if output_file is set)
file_com.write("Hello output file!")

# Shutdown after communication and block until all processes have terminated
file_com.shutdown(block=True)

1.1.5.1.3. Architecture

Fig. 1.3 File Communication Architecture

Fig. 1.4 MQTT Communication Architecture

Fig. 1.5 CAN Communication Architecture

1.1.5.2. can

This program implements a command line interface (CLI) to test sending and receiving of CAN messages using a configuration and optional a input/output files.

1.1.5.2.1. Usage

Usage: fox.py com-test can [OPTIONS]

  Send and receive CAN messages using a YAML connection config.

Options:
  -c, --config FILE  Path to configuration yaml file  [required]
  -i, --input FILE   Path to the input file
  -o, --output FILE  Path to the output file
  -v, --verbose      Verbose information.
  -h, --help         Show this message and exit.

  Examples:

  As config should be passed a yaml file defining the communication of the CAN
  bus as the example below:

          connection:
              interface: pcan
              bitrate: 500000
              channel: PCAN_USBBUS2
          logger:
              max_bytes: int = 65536
              rollover_count: int = 0
              log_can_files: bool = False

  As input file should be used a text file with line deliminated json objects
  as below:

          {"id": 291, "data": [1, 2, 3, 4]}
          {"id": 291, "data": [5, 6, 7, 8]}

  The output file will contain the received CAN messages in a line deliminated
  json object format.

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 com-test can -c <config_file> -i [<input_file>] -o [<output_file>]

1.1.5.2.2. Features

• Reads CAN connection parameters from the provided configuration file.

• Sends CAN messages specified in the input file.

• Writes received CAN messages to the output file in the python-can format.

• Supports DBC-based encoding of signal dictionaries (if a DBC file is provided).

• Tolerates transient CAN receive errors; stops if too many errors occur in a short time.

1.1.5.2.3. Input File Format

Each line in the input file must be a valid JSON object describing a CAN message to send. Two formats are supported:

1. Raw payload (no DBC encoding):

{ "id": 291, "data": [1, 2, 3, 4, 5, 6, 7, 8] }

• id: integer message identifier in decimal representation (standard ID expected).

• data: list of integers (0-255), up to 8 bytes.

2. DBC-based signals (when a DBC is configured):

{ "id": 291, "data": {"SignalA": 42, "SignalB": 1 }}

• id: integer message identifier that exists in the DBC file.

• data: object mapping signal names to values; signals are encoded using the DBC definition.

Note

• Extended identifiers are not configurable via the input; standard IDs are used.

• If the input line is not valid JSON, that line is ignored.

1.1.5.2.4. Output File Format

Each received CAN message is written into a text file. A typical record contains the timestamp, arbitration id and data, for example:

Timestamp:        0.000000    ID:      123    S Rx                DL:  8    01 02 03 04 05 06 07 08

The exact structure may include additional fields depending on the python-can message representation used by the logger.

Important

The received CAN messages are saved into a file with the suffix .txt If a different file suffix is used, the SizedRotatingLogger from python-can will not be created and therefore the logging will not start!

1.1.5.2.5. Configuration

The configuration file is a YAML document with at least the following sections:

• connection: parameters to initialize the CAN bus (python-can).

• logger: parameters for CAN logging.

An example skeleton (adjust to your environment):

connection:
  interface: socketcan        # e.g., 'socketcan', 'pcan', 'kvaser', ...
  channel: can0               # e.g., 'can0', 'PCAN_USBBUS1', ...
  bitrate: 500000             # bus bitrate in bit/s
  dbc: ./example/example.dbc  # optional path to a DBC file

logger:                       # optional settings for the CAN logger
  max_bytes: 65536            # max. number of bytes in each log file
  rollover_count: 0           # The starting number for each log file

1.1.5.2.6. Example

The following example uses a PCAN interface on Linux with a 500 kbit/s bus. The command:

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 com-test can -c can_config_send.yaml -i can_input.jsonl

With the configuration file can_config_send.yaml as seen below:

Listing 1.1 Configuration for the can subcommand to send messages.

connection:
  interface: pcan
  bitrate: 500000
  channel: PCAN_USBBUS1

The input file could contain the following lines:

Listing 1.2 Input file for the can subcommand.

{ "id": 291, "data": [1, 2, 3, 4] }
{ "id": 291, "data": [5, 6, 7, 8] }
{ "id": 291, "data": [1, 2, 3, 4] }
{ "id": 291, "data": [5, 6, 7, 8] }

Running the following command in a second terminal:

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 com-test can -c can_config_recv.yaml -o can_output.txt

With the configuration file can_config_send.yaml as seen below:

Listing 1.3 Configuration for the can subcommand to recv messages.

connection:
  interface: pcan
  bitrate: 500000
  channel: PCAN_USBBUS2

The output file will contain the following lines:

Listing 1.4 Output file for the can subcommand.

Timestamp:        0.000000    ID:      123    S Rx                DL:  4    01 02 03 04
Timestamp:        0.000213    ID:      123    S Rx                DL:  4    05 06 07 08
Timestamp:        0.000384    ID:      123    S Rx                DL:  4    01 02 03 04

Note

The SizedRotatingLogger from python-can will always keep the last line in memory, therefore in the output file the last line is missing.

Hint

While sending CAN messages or logging to an output file, you can stop with Ctrl+C. The program will then shut down the CAN connection and the file logger gracefully.

The example configuration and the input can be downloaded below:

Send Configuration

Input

Recv Configuration

Output

1.1.5.3. modbus

This program implements a command line interface (CLI) to execute Modbus TCP commands (requests) as a client using configuration and input/output files. It also provides a simple built-in Modbus TCP test device (server).

1.1.5.3.1. Subcommands

• modbus client - runs the Modbus client that reads JSON line commands, sends them to a Modbus device and writes response to another file.

• modbus device - starts a local Modbus TCP device (server) for testing.

1.1.5.3.2. Usage

1.1.5.3.2.1. Client:

Usage: fox.py com-test modbus client [OPTIONS]

  Run the Modbus client workflow using a YAML configuration.

  Reads requests from ``input_file`` and writes results to ``output_file``.

Options:
  -c, --config FILE  Path to configuration yaml file  [required]
  -i, --input FILE   Path to the input file  [required]
  -o, --output FILE  Path to the output file  [required]
  -v, --verbose      Verbose information.
  -h, --help         Show this message and exit.

  The configuration file defines connection and register mapping details. The
  input file contains the Modbus commands (requests).

  Example execution:

      fox.py modbus client -c config.yaml -i input.jsonl -o output.jsonl

  Example command:

      {"date": "2025-01-01T00:00:00.000", "code": "read_coils", "address": 9,
      "length": 1, "values": null}

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 modbus client -c <config_file> -i <input_file> -o <output_file>

1.1.5.3.2.2. Device (test server):

Usage: fox.py com-test modbus device [OPTIONS]

  Start a local Modbus TCP device (server) for testing and development.

  The call blocks until the server is stopped (e.g., via Ctrl+C).

Options:
  -a, --address TEXT  Ip address of the Modbus device.
  -p, --port INTEGER  Path to configuration yaml file
  -v, --verbose       Verbose information.
  -h, --help          Show this message and exit.

  Available register:

      - coil: address 10, length 4, default value 0

      - discrete input: address 20, length 8, default value 1

      - input register:

                 - address 40, length 4, default values 20, 210, 450, 800

                 - address 60, length 1, default value 100

      - holding register:

                 - address 10, length 1, default value 225

                 - address 20, length 1, default value 400

  On this device, the registers don't share a memory. The client command uses
  zeromode for the request addresses, therefore to read from coil at address
  10, the request address must be 9.

  Example request:

      - {"date": "2025-11-25T00:00:00","code": "read_coils", "address": 9,
      "length": 1, "values": null}

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 modbus device -a [address] -p [port]

1.1.5.3.3. Client Features

• Reads Modbus TCP connection parameters from the provided configuration file.

• Executes Modbus commands specified in the input file (JSON per line).

• Writes command results to the output file (JSON per line).

• Supports common Modbus function codes for coils, discrete inputs, input registers, and holding registers.

• Handles I/O via background processes for Modbus and file operations.

1.1.5.3.4. Supported Command Codes

Table 1.7 Modbus device command arguments

Code	Description
read_coils	Reads coil values. Response provides values as a list of bits.
read_discrete_input	Reads discrete input values. Response provides values as a list of bits.
read_input_register	Reads input registers. Response provides values as a list of register values.
read_holding_register	Reads holding registers. Response provides values as a list of register values.
write_coils	Writes multiple coil values. Response echoes written values (if provided by server).
write_holding_register	Writes multiple holding registers. Response echoes written values (if provided by server).

1.1.5.3.5. Configuration File

The YAML configuration file for the Modbus TCP parameters is depicted below:

Listing 1.5 Configuration for the modbus client subcommand

host: "localhost"
port: 502
timeout: 1
retries: 3
ignore: False

1.1.5.3.6. Input File Format

Each line in the input file represents a command (request) and must be a valid JSON object with the following structure:

{ "date": "2025-01-01T12:00:00", "code": "read_coils", "address": 9, "length": 4, "values": null }
{ "date": "2025-01-01T12:00:01", "code": "write_holding_register", "address": 19, "length": 2, "values": [100, 200] }

Important

In the used Python package PyModbus, register addresses start at 0. Hence if a register in a device starts at 10, the command requires the address 9.

Note

• The parameter values is ignored for read operations.

• For write operations, values must be provided and is echoed in the response (if available).

1.1.5.3.7. Output File Format

Each result is written as a line-delimited JSON object mirroring the input structure, with values populated from the Modbus response:

{ "date": "2025-01-01T12:00:00", "code": "read_coils", "address": 9, "length": 4, "values": [0, 0, 1, 1] }
{ "date": "2025-01-01T12:00:01", "code": "write_holding_register", "address": 19, "length": 2, "values": [100, 200] }

Hint

If values is None, an exception occurred and the command (request) could not be processed. Additionally an error: exception key-value-pair is added to the output.

1.1.5.3.8. Device Register Map

The table below documents the initial register state of the built-in Modbus TCP device (test server).

Table 1.8 Modbus Test Device Register Map

Register	Start Address	Length	Default Values	Description
Coils	10	4	[0, 0, 0, 0]	Writable coils used for testing; used by read_coils and write_coils.
Discrete Inputs	20	8	[1, 1, 1, 1, 1, 1, 1, 1]	Read-only digital inputs.
Input Registers	40	4	[20, 210, 450, 800]	Read-only analog input sample values.
Input Registers	60	1	[100]	Single read-only input register.
Holding Registers	10	1	[225]	Writable holding register.
Holding Registers	20	1	[400]	Writable holding register.

Note

• Coils and Holding Registers are writable in device (test server); Discrete Inputs and Input Registers are read-only.

• Default values shown are the initial values when the server starts.

Hint

The register memory is not shared between blocks. See explanation.

1.1.5.3.9. Example

Start the local test server:

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 modbus device -a localhost -p 502

Run the client:

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 modbus client -c modbus_config.yaml -i modbus_input.jsonl -o modbus_output.jsonl

Prepare the input file:

Listing 1.6 Example input file for the modbus client subcommand.

{"date": "0","code": "read_coils", "address": 9, "length": 1, "values": null}
{"date": "1","code": "read_coils", "address": 9, "length": 2, "values": null}
{"date": "2","code": "read_coils", "address": 9, "length": 3, "values": null}
{"date": "3","code": "read_coils", "address": 9, "length": 4, "values": null}
{"date": "4","code": "write_coils", "address": 9, "length": 2, "values": [1,1]}
{"date": "5","code": "read_coils", "address": 9, "length": 4, "values": null}
{"date": "6","code": "read_holding_register", "address": 9, "length": 1, "values": null}
{"date": "7","code": "write_holding_register", "address": 9, "length": 1, "values": [69]}
{"date": "8","code": "read_holding_register", "address": 9, "length": 1, "values": null}

Resulting output file:

Listing 1.7 Example output file for the modbus client subcommand.

{"date": "0", "code": "read_coils", "address": 9, "length": 1, "values": [0]}
{"date": "1", "code": "read_coils", "address": 9, "length": 2, "values": [0, 0]}
{"date": "2", "code": "read_coils", "address": 9, "length": 3, "values": [0, 0, 0]}
{"date": "3", "code": "read_coils", "address": 9, "length": 4, "values": [0, 0, 0, 0]}
{"date": "4", "code": "write_coils", "address": 9, "length": 2, "values": []}
{"date": "5", "code": "read_coils", "address": 9, "length": 4, "values": [1, 1, 0, 0]}
{"date": "6", "code": "read_holding_register", "address": 9, "length": 1, "values": [225]}
{"date": "7", "code": "write_holding_register", "address": 9, "length": 1, "values": []}
{"date": "8", "code": "read_holding_register", "address": 9, "length": 1, "values": [69]}

The example configuration and the input/output can be downloaded below:

Configuration

Input

Output

1.1.5.3.10. Implementation Notes

• The CLI orchestrates two child processes:

  • A Modbus process that owns the pymodbus.client.ModbusTcpClient and executes commands.

  • A file handler that reads input JSON lines and writes output JSON lines.

• Communication between processes uses queues; the client blocks up to 1 second while waiting for a result.

• If the Modbus server returns an exception (pymodbus.pdu.ExceptionResponse), the behavior depends on ignore:

  • ignore: true –> log as debug and continue.

  • ignore: false –> log as error and stop the loop.

1.1.5.4. mqtt

This program implements a command line interface (CLI) to test sending and receiving of MQTT messages as client using a configuration and optional a input/output files.

1.1.5.4.1. Usage

Usage: fox.py com-test mqtt [OPTIONS]

  Send and receive MQTT messages using a YAML connection config.

Options:
  -c, --config FILE  Path to configuration yaml file  [required]
  -i, --input FILE   Path to the input file
  -o, --output FILE  Path to the output file
  -v, --verbose      Verbose information.
  -h, --help         Show this message and exit.

  Examples:

  As config should be passed a yaml file defining the communication to the
  MQTT broker as the example below:

          broker: "broker.emqx.io"
          port: 1883
          subscribe: ["test_fox"]

  As input file should be used a text file with line deliminated json objects
  as below:

          {"topic": "test_fox", "data": "msg_1"}
          {"topic": "test_fox", "data": "msg_2"}

  The output file will contain the received MQTT messages in a line
  deliminated json object format.

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 com-test mqtt -c <config_file> -i [<input_file>] -o [<output_file>]

1.1.5.4.2. Features

• Reads MQTT connection parameters from the provided configuration file.

• Sends MQTT messages specified in the input file.

• Writes received MQTT messages to the output file.

• Handles input/output as JSON lines.

1.1.5.4.3. Input File Format

Each line in the input file should be a valid JSON object with the following structure:

{ "topic": "your/topic", "data": "your message" }

1.1.5.4.4. Output File Format

Each received MQTT message is written as a line deliminated JSON object identical to input example.

1.1.5.4.5. Example

The following example uses the public ‘broker.emqx.io’ MQTT broker with no password, username or certificate.

Run the command:

Win32/PowerShellWin32/Git bashLinux

.\fox.ps1 com-test mqtt -c config.yaml -i input.jsonl -o output.jsonl

Use the configuration file shown below:

Listing 1.8 Configuration for the mqtt subcommand

broker: "broker.emqx.io"
port: 1883
# username: "your_username"
# password: "your_password"
# tls_cert: "/path/to/ca.crt"
subscribe:
  - "test/topic"

Prepare the input file:

Listing 1.9 Input file for the mqtt subcommand

{"topic":"fox/topic", "data": "msg_1"}
{"topic":"fox/topic", "data": "msg_2"}
{"topic":"fox/another", "data": "msg_3"}
{"topic":"fox/another", "data": "msg_4"}
{"topic":"fox/topic", "data": "msg_5"}
{"topic":"fox/topic", "data": "msg_6"}
{"topic":"fox/another", "data": "msg_7"}
{"topic":"fox/topic", "data": "msg_8"}
{"topic":"fox/another", "data": "msg_9"}

Resulting output file:

Listing 1.10 Output file produced by the mqtt subcommand

{"topic":"fox/topic", "data": "msg_1"}
{"topic":"fox/topic", "data": "msg_2"}
{"topic":"fox/topic", "data": "msg_5"}
{"topic":"fox/topic", "data": "msg_6"}
{"topic":"fox/topic", "data": "msg_8"}

The example configuration and the input can be downloaded below:

Configuration

Input

Output