By Benjamín Arratia

<aside> 💡 Turn on dark mode with cmd/ctrl + shift + L for the best viewing experience...

</aside>

Motivation

The Arduino Nicla Vision is a powerful board that can run tinyML models. However, when using models from Edge Impulse, there's a challenge: the default firmware assigns UART pins that might not be suitable for all applications. This guide will walk you through the process of customizing the UART pins for better compatibility.

Problem

The firmware generated by Edge Impulse for the Arduino Nicla Vision has UART pins assigned that might not be optimal for every use case.

Solution

To address this, we'll modify the firmware to assign the desired UART pins. Here's a step-by-step guide:

<aside> 🔥 If you don't intend to make further modifications to the pins and simply want to access UART with your Nicla Vision using a custom model, you can take a shortcut

• Quick Solution for UART Access with Custom Model </aside>

1. Fork the Repositories:

   • Fork the OpenMV repository and the MicroPython repository on GitHub.

2. Clone and Checkout:

   1. Clone your forked OpenMV repository:

      git clone <https://github.com/YOUR_USERNAME/openmv.git>
      cd openmv
      

   2. Checkout the specific commit for OpenMV:

      git checkout 4455c030c31e43d7be26b854109dbe3999e24869
      

   3. Create a new branch based on this commit:

      git checkout -b custom_changes_openmv
      

   4. Update the submodules:

      git submodule update --init --recursive
      

   5. Navigate to the MicroPython submodule:

      cd src/micropython
      

   6. Checkout the specific commit for MicroPython:

      git checkout c9e87104a0129d7d104d8e015e85901705c9e6cc
      

   7. Create a new branch based on this commit:

      git checkout -b custom_changes_micropython
      

3. Modify UART Pins:

   • Navigate to the file: openmv/micropython/ports/stm32/boards/NICLAV/mpconfigboard.h.

   • Replace the UART definition with:

     #define MICROPY_HW_UART4_TX         (pin_B9)
     #define MICROPY_HW_UART4_RX         (pin_B8)
     // LPUART1 config
     #define MICROPY_HW_LPUART1_TX       (pin_A9)
     #define MICROPY_HW_LPUART1_RX       (pin_A10)
     

4. Add the tinyML Model:

   • Place your tinyML model (.tflite file) and the labels (.txt file) in the directory: openmv/src/lib/libtf/models/.
   • Ensure both files have the same name (e.g., custom_model.tflite and custom_model.txt).

5. Commit and Push Changes:

   • Commit the changes you've made to both repositories and push them to your forked repositories on GitHub.
   • Check that everything is ok by navigating to your src/micropython folder in your OpenMV fork and it should direct you to your last commit on your micropython fork.

6. Set Up GitHub Actions:

   1. Modify Workflow Files: Adjust the workflow files (inside the .github folder in the root of the OpenMV repo) to activate on changes to your custom branch (custom_changes_openmv in this example).
   2. Specify Desired Boards: Ensure that the actions are set up to compile and generate the firmware specifically for the boards you're interested in, such as the Arduino Nicla Vision.
   3. Navigate to Actions: Go to the "Actions" tab of your forked OpenMV repository and set up the necessary actions to build the firmware. This will automatically compile and generate the custom firmware for the Arduino Nicla Vision. If not, you should trigger it by pushing a change inside the src folder (A comment should trigger it).

Testing the Custom Firmware

1. Load the Firmware: Use the Bootloader on OpenMV to load the custom firmware onto your Arduino Nicla Vision.

2. Set Up UART: In your MicroPython script, set up the UART with the following configuration:

   uart = UART(1, 9600)
   

   This configuration uses pins PA_9 and PA_10, as indicated in the original pinout of the Arduino Nicla Vision.

   

Achievements: With this setup, I was able to capture an image, run inference with my model, and then send both the image and the inference result via UART to an ESP32. This approach also provided a workaround for an issue I encountered with the Edge Impulse’s Python SDK, which couldn't deploy my custom models to the OpenMV firmware for my Nicla Vision. Now, I can simply place the model I want to test in the appropriate folder, push it to my forked repository, and after a few minutes, download the built firmware. While it's not as straightforward as using the deploy command in Python, it's a functional and effective workaround.