diff --git a/boards/beagleplay/03-design.rst b/boards/beagleplay/03-design.rst index 0f287bf88446dd757798e3cac1d0b46124604b4b..500304080273230d8b00ec3c56a8516049f63abb 100644 --- a/boards/beagleplay/03-design.rst +++ b/boards/beagleplay/03-design.rst @@ -30,7 +30,7 @@ more detail with schematic diagrams. :alt: BeaglePlay block diagram System on Chip (SoC) -********************* +******************** `AM62x Sitara™ Processors <https://www.ti.com/product/AM625>`_ from Texas Instruments are Human-machine-interaction SoC with Arm® Cortex®-A53-based edge AI and full-HD dual display. @@ -52,7 +52,7 @@ are interested to know more about the AM62x SoC you may take a look at AM6254 SoC block diagram Power management -***************** +**************** Different parts of the board requires different voltages to operate and to fulfill requirements of all the chips on BeaglePlay we have Low Drop Out (LDO) voltage regulators for fixed voltage output @@ -65,7 +65,7 @@ and Power Management Integrated Circuit (PMIC) that interface with SoC to genera :alt: BeaglePlay power block diagram 1.0V LDO -========= +======== TLV75801 is an adjustable 500-mA low-dropout (LDO) regulator. Consumes very low quiescent current and provides fast line and load transient performance. The TLV758P features an ultra-low dropout of 130 mV at 500 mA that can help improve the power efficiency of the system. @@ -93,7 +93,7 @@ when running low on power rails. optimization could be performed if concerned about sleep modes. 3.3V DCDC buck -=============== +============== TLV62595 is a high-frequency synchronous step-down converter optimized for compact solution size and high efficiency. The device integrates switches capable of delivering an output current up to 4 A. @@ -117,7 +117,7 @@ gigabit Ethernet PHY. Due to the relatively high current rating (3A), a highly e (VDD_3V3_PG) is available at TP19 and is unused on the rest of the board. PMIC -===== +==== The TPS65219 is a Power Management IC (PMIC) designed to supply a wide range of SoCs in both portable and stationary applications. The DC-DC converters are capable of 1x 3.5 A and 2x @@ -146,7 +146,7 @@ Reference Manual `SLVUCJ2 <https://www.ti.com/lit/pdf/slvucj2>`_. Add specific power-up/down sequence notes here as well a highlight any limitations and known issues. General Connectivity and Expansion -*********************************** +********************************** One of the main advantage of using a Single Board Computer (SBC) is having direct accessibility of general purpose input & output (GPIO) pins and other interfaces like I2C, SPI, ADC, PWM. Your BeaglePlay @@ -160,7 +160,7 @@ The USB-C connector allows you to power the board and to connect the board to a use the pre-installed VisualStudio Code editor by putting the address ``192.168.7.2:3000`` in your web browser. USB A & USB C -============== +============= Below is the schematic of full size USB A for pripheral connection and USB C for device power & tethering. @@ -172,7 +172,7 @@ Below is the schematic of full size USB A for pripheral connection and USB C for USB-A and USB-C 2ch 10bit ADC -============== +============= The ADC102S051 is a low-power, two-channel CMOS 10-bit analog-to-digital converter with a high- speed serial interface. Unlike the conventional practice of specifying performance at a single sample @@ -190,12 +190,12 @@ reduces the power consumption to just 0.12 μW using a +3V supply, or 0.47 μW u .. figure:: images/hardware-design/ADC102S051.svg :width: 1247 :align: center - :alt: ADC102S051 - 12bit Aanalog to Digital Converter (ADC) + :alt: ADC102S051 - 12bit Analog to Digital Converter (ADC) - ADC102S051 - 12bit Aanalog to Digital Converter (ADC) + ADC102S051 - 12bit Analog to Digital Converter (ADC) mikroBUS -========= +======== mikroBUS is a standard specification by MikroElektronika that can be freely used by anyone following the guidelines. It includes SPI, I2C, UART, PWM, ADC, reset, interrupt, and power (3.3V and 5V) connections to common embedded peripherals. @@ -208,7 +208,7 @@ It includes SPI, I2C, UART, PWM, ADC, reset, interrupt, and power (3.3V and 5V) mikroBUS connector schematic Grove -====== +===== Seeed Studio Grove System is a modular, standardized connector prototyping ecosystem. The Grove System takes a building block approach to assembling electronics. Compared to the jumper or solder based system, @@ -234,7 +234,7 @@ Qwiic, or STEMMA QT are 4pin JST SH 1.00 connectors for easy I2C connection. QWIIC connnector for I2C modules Buttons and LEDs -***************** +**************** To interact with the Single Board Computers we use buttons for input and LEDs for visual feedback. On your BeaglePlay board you will find 3 buttons each with a specific purpose: power, reset, and user. @@ -242,7 +242,7 @@ For visual feedback you will find 5 user LEDs near USB-C port and 6 more indicat Single Pair ethernet port. Schematic diagrams below show how these buttons and LEDs are wired. Buttons -======== +======= Power, Reset and User buttons for turning board ON/OFF, resetting board, and boot selection or user assigned control. @@ -257,7 +257,7 @@ Power, Reset and User buttons for turning board ON/OFF, resetting board, and boo +-------------------------------------------------------------+-------------------------------------------------------------+--------------------------------------------------------+ LEDs -===== +==== Power and user LEDs for status and general purpose usage. @@ -269,12 +269,12 @@ Power and user LEDs for status and general purpose usage. BeaglePlay LEDs Wired and wireless connectivity -******************************** +******************************* For internet connection or general connectivity between BeaglePlay and other devices. Gigabit ethernet -================= +================ The Realtek RTL8211F-CG is a highly integrated Ethernet transceiver that is compatible with 10Base-T, 100Base-TX, and 1000Base-T IEEE 802.3 standards. It provides all the necessary physical layer functions @@ -293,7 +293,7 @@ implemented in the RTL8211F(I)-CG to provide robust transmission and reception c Gigabit ethernet Single pair ethernet -===================== +==================== The DP83TD510E is an ultra-low power Ethernet physical layer transceiver compliant with the IEEE 802.3cg 10Base-T1L specification. The PHY has very low noise coupled receiver architecture enabling @@ -312,7 +312,7 @@ test, and loopback capabilities for ease of design or debug Single pair ethernet WiFi 2.4G/5G -============= +============ The WL18x7MOD is a Wi-Fi, dual-band, 2.4- and 5-GHz module solution with two antennas supporting industrial temperature grade. The device is FCC, IC, ETSI/CE, and TELEC certified for AP (with DFS support) and client. TI offers drivers for high-level @@ -327,7 +327,7 @@ QNX, Nucleus, ThreadX, and FreeRTOS, are supported through third parties. WL1807MOD dual-band (2.4G/5G) WiFi BLE & SubGHz -============= +============ The SimpleLink™ CC1352P7 device is a multiprotocol and multi-band Sub-1 GHz and 2.4-GHz wireless microcontroller (MCU) supporting Thread, Zigbee®, Bluetooth® 5.2 Low Energy, IEEE 802.15.4g, IPv6-enabled @@ -352,7 +352,7 @@ applications. CC1352P7 Bluetooth Low Energy (BLW) and SubGHz connectivity Memory, Media and Data storage -******************************** +****************************** DDR4 ==== @@ -375,7 +375,7 @@ eMMC/SD eMMC/SD storage microSD Card -============= +============ .. figure:: images/hardware-design/micro-sd-card.svg :width: 1247 @@ -395,10 +395,10 @@ Board EEPROM Board EEPROM ID Multimedia I/O -*************** +************** HDMI -===== +==== .. figure:: images/hardware-design/hdmi.svg :width: 1247 @@ -408,7 +408,7 @@ HDMI HDMI output OLDI -===== +==== .. figure:: images/hardware-design/oldi.svg :width: 1247 @@ -419,7 +419,7 @@ OLDI CSI -==== +=== .. figure:: images/hardware-design/csi.svg :width: 1247 @@ -429,10 +429,10 @@ CSI CSI camera interface RTC & Debug -************ +*********** RTC -==== +=== .. figure:: images/hardware-design/rtc.svg :width: 940 @@ -442,7 +442,7 @@ RTC Real Time Clock (RTC) UART Debug Port -================ +=============== .. figure:: images/hardware-design/debug.svg :width: 940 @@ -474,7 +474,7 @@ CC1352 JTAG & TagConnect .. _beagleplay-mechanical-specifications: Mechanical Specifications -************************** +************************* Dimensions & Weight =================== diff --git a/boards/beagleplay/demos-and-tutorials/understanding-boot.rst b/boards/beagleplay/demos-and-tutorials/understanding-boot.rst index f348f75e9a5b1eb2718f0e9fc7620c23cf0687f7..43bf82d94cd1a7a8f38bca37fba0d727af85d281 100644 --- a/boards/beagleplay/demos-and-tutorials/understanding-boot.rst +++ b/boards/beagleplay/demos-and-tutorials/understanding-boot.rst @@ -15,7 +15,7 @@ Distro Boot For some background on distro boot, see `the u-boot documentation on distro boot <https://docs.u-boot.org/en/latest/develop/distro.html>`_. There is -also `BeaglePlay specific u-boot documentation <https://docs.u-boot.org/en/latest/board/ti/am62x_beagleplay.html>`_. +also `BeaglePlay specific u-boot documentation <https://docs.u-boot.org/en/latest/board/beagle/am62x_beagleplay.html>`_. In :ref:`play-typical-extlinux-conf`, you can see line 1 provides a label and subsequent indented lines provide parameters for that boot option. diff --git a/boards/beaglev/fire/05-demos.rst b/boards/beaglev/fire/05-demos.rst index 229ba076ad427d8592aad9c8ed2cd73f488789d3..5bf8e3bff163c5e3095562fbcc562b9bcbbe3208 100644 --- a/boards/beaglev/fire/05-demos.rst +++ b/boards/beaglev/fire/05-demos.rst @@ -24,3 +24,4 @@ Demos demos-and-tutorials/gateware/customize-cape-gateware-verilog demos-and-tutorials/gateware/running-the-bitstream-builder-on-windows demos-and-tutorials/gateware/exploring-gateware-design-libero + demos-and-tutorials/gateware/axi-apb-interfaces-demo diff --git a/boards/beaglev/fire/demos-and-tutorials/gateware/axi-apb-interfaces-demo.rst b/boards/beaglev/fire/demos-and-tutorials/gateware/axi-apb-interfaces-demo.rst new file mode 100644 index 0000000000000000000000000000000000000000..e922d2b3efc630223555e6141e8ea6a4d8095298 --- /dev/null +++ b/boards/beaglev/fire/demos-and-tutorials/gateware/axi-apb-interfaces-demo.rst @@ -0,0 +1,165 @@ +.. _beaglev-fire-axi-apb-interfaces-demo: + +Accessing APB and AXI Peripherals Through Linux +############################################### + +AXI +*** + +.. line-block:: + `AXI <https://developer.arm.com/documentation/ihi0022/latest/>`_ is part of the `ARM AMBA <https://developer.arm.com/Architectures/AMBA>`_ (Advanced Microcontroller Bus Architecture) protocol family. + It is designed for high-performance, high-frequency system-on-chip (SoC) designs. + AXI provides high-speed data transfer with minimal latency and is widely used in various applications, including high-end embedded systems and complex digital circuits. + +APB +*** + +.. line-block:: + `APB <https://developer.arm.com/documentation/ihi0024/latest/>`_ is also part of the ARM AMBA protocol family, designed for low-power and low-latency communication with peripheral devices. + It is simpler and lower performance compared to AXI, making it suitable for slower peripheral devices. An APB peripheral also consumes less resources on the FPGA fabric compared to an AXI peripheral. + +Accessing AXI and APB Peripherals from Linux +******************************************** + +.. line-block:: + To access AXI and APB peripherals from Linux, memory-mapped I/O (MMIO) is commonly used. + This involves mapping the physical addresses of the peripherals into the virtual address space of a user-space application. + The following sections demonstrate how to access APB peripherals using the Linux ``/dev/mem`` interface and AXI peripherals using the UIO (Userspace I/O) framework. + +.. note:: + The codes for accessing the interfaces are available in the snippets here: `APB Interfaces <https://openbeagle.org/-/snippets/13>`_ and `AXI Interfaces <https://openbeagle.org/-/snippets/11>`_ + +APB Interfaces +============== + +.. line-block:: + The MSS includes fabric interfaces for interfacing FPGA fabric with the CPU Core Complex. + It provides one 32-bit APB master interface, FIC3, and can be connected to a slave in the fabric. + +Design Details +-------------- + +For this example, you can try to write to the APB slave present in the Verilog Tutorial Cape gateware. +Select the gateware by changing `custom-fpga-design/my_custom_fpga_design.yaml` to include ``VERILOG_TUTORIAL`` as the cape option. + +The APB Slave has two registers, one read-only register at ``0x00``, one read-write register at ``0x10``, and a status register containing the last read value at ``0x20``. + +.. line-block:: + Having a look at the design, we can see that the APB slave is connected with a CoreAPB3 interconnect, which assigns it the ``0xXX10_0000`` address, the top two bits being ignored. + Tracing to the master connected with the CoreAPB3 device, we can see that another interconnect is present, which gives our slave the ``0xX100_0000`` address. + The polarfire technical manual shows that FIC3 peripherals can start from the ``0x4000_0000`` address. + Therefore, the final address of our APB slave becomes ``0x4110_0000``. + +Now, we shall access this address through a memory-mapped interface in Linux. + +.. important:: + + | The following paragraphs will present to you several ways to test APB/AXI traffic. + | Normally, this isn't harmful, but reading/writing to addresses + | with no gateware behind it **will** lead to you stalling a CPU. + | In rare cases, this stalling of a CPU *can* lead to loss of content on your eMMC, + | so please make sure you have a known good backup! + | For more information about the stalls, please read the section :ref:`on issues faced with the interfaces <issues-with-axi-apb>`. + +Accessing the Interface +------------------------ + +There are two ways to access such registers. One can use the `devmem2` utility or write a C program for accessing the memory region. +The first method is quite simple. + +1. To read from a register: + + .. code-block:: shell + + sudo devmem2 0x41100000 w + +2. To write to a register: + + .. code-block:: shell + + sudo devmem2 0x41100010 w 0x1 + +In the second method, we can use the ``/dev/mem`` interface to access the registers inside the APB Slave. +Here is an example C program which demonstrates this: + +.. raw:: html + + <script src="https://openbeagle.org/-/snippets/13.js"></script> + + +AXI Interfaces +============== + +The MSS includes three 64-bit AXI FICs out of which FIC0 is used for data transfers to/from the fabric. +FIC0 is connected as both master and slave. For usage of AXI peripherals, an example is also provided by microchip +in their `Polarfire SoC Linux examples <https://github.com/polarfire-soc/polarfire-soc-linux-examples>`_. The example here takes reference from +the `AXI LSRAM example <https://github.com/polarfire-soc/polarfire-soc-linux-examples/tree/master/fpga-fabric-interfaces/lsram>`_. + +Design Details +-------------- + +.. line-block:: + A simple design can be created by first connecting the FIC0 Initiator from the MSS to a `CoreAXI4Interconnect <https://www.microchip.com/en-us/products/fpgas-and-plds/ip-core-tools/coreaxi4interconnect>`_. + Now, you can connect an AXI slave to this interconnect. We will be using the Polarfire AXI LSRAM. + + Both the CoreAXI4Interconnect and the PF AXI LSRAM will have to be configured. + The AXI ID Width of both the modules will have to be matched, as well as the address space of the only slave will have to be configured. + In this example, LSRAM gets an address of `0x6000_0000` to `0x6000_ffff`, and the AWID is kept at 9 bits. + +.. figure:: images/axi-slave-demo.png + :width: 1040 + :alt: AXI LSRAM slave + + AXI LSRAM slave (example design) + +Finally, an entry will be added to the device tree to make a UIO device point to our LSRAM's memory region. + +.. code-block:: + + &{/} { + fabric-bus@40000000 { + fpgalsram: uio@60000000 { + compatible = "generic-uio"; + linux,uio-name = "fpga_lsram"; // mandatory for program. If changed, please update program as well. + reg = <0x0 0x60000000 0x0 0x1000>; + status = "enabled"; + }; + }; + }; + +Once the gateware is compiled, we can access the memory-mapped interface by the same methods, and by the UIO device as well. + +1. Using devmem2: + + .. code-block:: shell + + sudo devmem2 0x60000000 w # for read + sudo devmem2 0x60000000 w 0x1 # for write + +2. Using the UIO device: + +.. raw:: html + + <script src="https://openbeagle.org/-/snippets/11.js"></script> + +.. _issues-with-axi-apb: + +Issues that can be faced when using an improperly configured AXI/APB interface +=============================================================================== + +A CPU stall can be faced when accessing the FIC interfaces without any slaves connected to the memory region being accessed. +Your BVF will stop responding if connected to SSH, and on serial you will see the following kernel messages: + +.. code-block:: shell + + [ 24.110099] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: + [ 24.116041] rcu: 0-...0: (1 GPs behind) idle=e00c/0/0x1 softirq=40/41 fqs=2626 + [ 24.123377] (detected by 3, t=5255 jiffies, g=-1131, q=9 ncpus=4) + [ 24.129573] Task dump for CPU 0: + [ 24.132810] task:swapper/0 state:R running task stack:0 pid:0 ppid:0 flags:0x00000008 + [ 24.142757] Call Trace: + [ 24.145213] [<ffffffff80a67ba0>] __schedule+0x27c/0x834 + +If this happens, please double check your design. Specifically, check the address configured for the slaves, the AXI ID wire width and other AXI parameters. + +In any case, this state is virtually impossible to recover from gracefully, so the **reset** button may be your last resort. diff --git a/boards/beaglev/fire/demos-and-tutorials/gateware/images/axi-slave-demo.png b/boards/beaglev/fire/demos-and-tutorials/gateware/images/axi-slave-demo.png new file mode 100644 index 0000000000000000000000000000000000000000..73605b8225a8779458e7f6aed67c5ef862bcad27 Binary files /dev/null and b/boards/beaglev/fire/demos-and-tutorials/gateware/images/axi-slave-demo.png differ diff --git a/boards/beagley/ai/02-quick-start.rst b/boards/beagley/ai/02-quick-start.rst index df0a49b40b39f16c52edccabad6ce627534a0abd..50ab4d824cc214acf66957e0b61e51e495c5bdab 100644 --- a/boards/beagley/ai/02-quick-start.rst +++ b/boards/beagley/ai/02-quick-start.rst @@ -26,7 +26,7 @@ You may need additional accessories based on the mode of operation, you can use 1. :ref:`USB Tethering by directly connecting via USB type-c port <beagley-ai-usb-tethering>` 2. :ref:`Headless connection via UART debug port <beagley-ai-headless>` -3. :ref:`Standalone connection with Monitor and other peripherals attached <standalone-connection>` +3. :ref:`Standalone connection with Monitor and other peripherals attached <beagley-ai-standalone-connection>` Easiest option is to connect the board directly to your PC or Laptop using a USB type-C to type-c cable. There is only one USB type-C port on board, if you choose to use a dedicated power supply for first time setup, you may choose to access the board via any other methods listed above. @@ -228,7 +228,7 @@ In ``sysconf.txt`` file you have to edit the two lines highlighted below. .. important:: 1. Make sure to remove ``#`` from ``#user_name=`` and ``#user_password=`` else the lines will be interpreted as a comment and your username & password will not be updated. - 2. If you do not change your username and passord here then you will not see any output on your HDMI monitor when you do a :ref:`standalone-connection` setup. + 2. If you do not change your username and passord here then you will not see any output on your HDMI monitor when you do a :ref:`beagley-ai-standalone-connection` setup. Once username and password are updated, you can insert the microSD card into @@ -329,7 +329,7 @@ line utility like ``tio`` on Linux of Putty on any operating system. Check :ref: Connecting Raspberry Pi debug probe to BeagleY-AI -.. _standalone-connection: +.. _beagley-ai-standalone-connection: Standalone connection ===================== diff --git a/boards/beagley/ai/05-demos.rst b/boards/beagley/ai/05-demos.rst index 948934e9fa43647d335dddbfbedc444c21f8c719..7393b068ad78141d7a1e002eea3e11dbaf208ccc 100644 --- a/boards/beagley/ai/05-demos.rst +++ b/boards/beagley/ai/05-demos.rst @@ -15,7 +15,8 @@ Demos and tutorials demos/beagley-ai-expansion-nvme demos/beagley-ai-using-imx219-csi-cameras demos/beagley-ai-arducam-imx219-v3link-dual-camera-kit - + demos/beagley-ai-object-detection-tutorial + diff --git a/boards/beagley/ai/demos/beagley-ai-object-detection-tutorial.rst b/boards/beagley/ai/demos/beagley-ai-object-detection-tutorial.rst new file mode 100644 index 0000000000000000000000000000000000000000..2bc8d77726b02d4b4d4177f85514065147453da5 --- /dev/null +++ b/boards/beagley/ai/demos/beagley-ai-object-detection-tutorial.rst @@ -0,0 +1,256 @@ +.. _beagley-ai-object-detection-tutorial: + +TensorFlow Lite Object Detection +############################################# + +This document describes how to set up and run an object detection model using TensorFlow Lite on the BeagleY-AI platform. Below is a demonstration. + +.. image:: ../images/ObjectDetectionModel.png + :alt: Object Detection Demo + + +To run the object detection model on the BeagleY-AI, you will need the following: + +- **BeagleY-AI Board**: Make sure to refer to the :ref:`BeagleY-AI standalone connection <beagley-ai-standalone-connection>` for proper setup. +- **USB Webcam**: The model has been tested with the Logitech Webcam C270, but it should work well with other webcam too. +- **Active Internet Connection**: Necessary for the installation of modules. Please check the :ref:`WiFi connection guide <beagley-ai-connecting-wifi>` for setting up the network. + +Step 1: Installation of Conda +============================= + +In this step, we will install a lightweight version of Conda. + +.. code-block:: bash + + wget https://github.com/conda-forge/miniforge/releases/download/24.3.0-0/Mambaforge-24.3.0-0-Linux-aarch64.sh + bash Mambaforge-24.3.0-0-Linux-aarch64.sh + +After accepting the license terms you can verify the installation by + +.. code-block:: bash + + conda --version + +Step 2: Create Virtual Environment +================================================ + +Create a virtual environment with Python 3.9. + +.. code-block:: bash + + conda create --name myenv python=3.9 + +Step 3: Activate the Virtual Environment +======================================== + +Activate the virtual environment created in the previous step. + +.. code-block:: bash + + conda activate myenv + +Step 4: Install Necessary Modules +================================= + +Install the required Python modules. + +.. code-block:: bash + + pip install https://github.com/google-coral/pycoral/releases/download/v2.0.0/tflite_runtime-2.5.0.post1-cp39-cp39-linux_aarch64.whl + pip install numpy==1.26.4 + pip install opencv-python + +Step 5: Load Necessary Pretrained Models +======================================== + +Create a directory for the object recognition models and download a pretrained model. + +.. code-block:: bash + + mkdir object-recognition + cd object-recognition + wget https://storage.googleapis.com/download.tensorflow.org/models/tflite/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip + unzip coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip -d TFLite_model + +.. tip:: You can train your own model using TensorFlow Lite.Here are some resources + + 1. `Train TensorFlow Lite Object Detection Model <https://colab.research.google.com/github/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/Train_TFLite2_Object_Detction_Model.ipynb#scrollTo=4VAvZo8qE4u5>`_. + + 2. `TensorFlow Lite Model Maker <https://www.tensorflow.org/lite/models/modify/model_maker>`_. + +Step 6: Connect Your USB Webcam +================================================== + +Connect your USB webcam via a USB socket. + +.. code-block:: bash + + ls -l /dev | grep video + +.. image:: ../images/video_driver_get.png + :alt: Get Video Driver +.. note:: Check the video driver with the above command. Here its 3 in my case. + +Step 7: Create the Code File +============================ + +Create a Python file for running object detection. + +.. code-block:: bash + + nano object-detection.py + +Paste the following code into the file: + +.. code-block:: python + + import os + import argparse + import cv2 + import numpy as np + import time + from threading import Thread + import importlib.util + from typing import List + import sys + from tflite_runtime.interpreter import Interpreter, load_delegate + + video_driver_id = 3 + + class VideoStream: + """Handles video streaming from the webcam.""" + def __init__(self, resolution=(640, 480), framerate=30): + self.stream = cv2.VideoCapture(video_driver_id) + self.stream.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'MJPG')) + self.stream.set(3, resolution[0]) + self.stream.set(4, resolution[1]) + self.grabbed, self.frame = self.stream.read() + self.stopped = False + + def start(self): + """Starts the thread that reads frames from the video stream.""" + Thread(target=self.update, args=()).start() + return self + + def update(self): + """Continuously updates the frame from the video stream.""" + while True: + if self.stopped: + self.stream.release() + return + self.grabbed, self.frame = self.stream.read() + + def read(self): + """Returns the most recent frame.""" + return self.frame + + def stop(self): + """Stops the video stream and closes resources.""" + self.stopped = True + + def load_labels(labelmap_path: str) -> List[str]: + """Loads labels from a label map file.""" + try: + with open(labelmap_path, 'r') as f: + labels = [line.strip() for line in f.readlines()] + if labels[0] == '???': + labels.pop(0) + return labels + except IOError as e: + print(f"Error reading label map file: {e}") + sys.exit() + + def main(): + # Argument parsing + parser = argparse.ArgumentParser() + parser.add_argument('--modeldir', required=True, help='Folder the .tflite file is located in') + parser.add_argument('--graph', default='detect.tflite', help='Name of the .tflite file') + parser.add_argument('--labels', default='labelmap.txt', help='Name of the labelmap file') + parser.add_argument('--threshold', default='0.5', help='Minimum confidence threshold') + parser.add_argument('--resolution', default='1280x720', help='Desired webcam resolution') + args = parser.parse_args() + + # Configuration + model_path = os.path.join(os.getcwd(), args.modeldir, args.graph) + labelmap_path = os.path.join(os.getcwd(), args.modeldir, args.labels) + min_conf_threshold = float(args.threshold) + resW, resH = map(int, args.resolution.split('x')) + + # Load labels and interpreter + labels = load_labels(labelmap_path) + interpreter = Interpreter(model_path=model_path) + interpreter.allocate_tensors() + + # Get model details + input_details = interpreter.get_input_details() + output_details = interpreter.get_output_details() + height, width = input_details[0]['shape'][1:3] + floating_model = (input_details[0]['dtype'] == np.float32) + + outname = output_details[0]['name'] + boxes_idx, classes_idx, scores_idx = (1, 3, 0) if 'StatefulPartitionedCall' in outname else (0, 1, 2) + + # Initialize video stream + videostream = VideoStream(resolution=(resW, resH), framerate=30).start() + time.sleep(1) + + frame_rate_calc = 1 + freq = cv2.getTickFrequency() + + while True: + t1 = cv2.getTickCount() + frame = videostream.read() + frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) + frame_resized = cv2.resize(frame_rgb, (width, height)) + input_data = np.expand_dims(frame_resized, axis=0) + + if floating_model: + input_data = (np.float32(input_data) - 127.5) / 127.5 + + interpreter.set_tensor(input_details[0]['index'], input_data) + interpreter.invoke() + + boxes = interpreter.get_tensor(output_details[boxes_idx]['index'])[0] + classes = interpreter.get_tensor(output_details[classes_idx]['index'])[0] + scores = interpreter.get_tensor(output_details[scores_idx]['index'])[0] + + for i in range(len(scores)): + if min_conf_threshold < scores[i] <= 1.0: + ymin, xmin, ymax, xmax = [int(coord) for coord in (boxes[i] * [resH, resW, resH, resW])] + cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), (10, 255, 0), 2) + object_name = labels[int(classes[i])] + label = f'{object_name}: {int(scores[i] * 100)}%' + labelSize, baseLine = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.7, 2) + label_ymin = max(ymin, labelSize[1] + 10) + cv2.rectangle(frame, (xmin, label_ymin - labelSize[1] - 10), (xmin + labelSize[0], label_ymin + baseLine - 10), (255, 255, 255), cv2.FILLED) + cv2.putText(frame, label, (xmin, label_ymin - 7), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2) + + cv2.putText(frame, f'FPS: {frame_rate_calc:.2f}', (30, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 0), 2, cv2.LINE_AA) + cv2.imshow('Object detector', frame) + + t2 = cv2.getTickCount() + time1 = (t2 - t1) / freq + frame_rate_calc = 1 / time1 + + if cv2.waitKey(1) == ord('q'): + break + + cv2.destroyAllWindows() + videostream.stop() + + if __name__ == "__main__": + main() + +.. note:: Make sure to change your video driver ID depending on your video driver. Here, the video driver ID is set to 3. + +Step 8: Run the Object Detection Script +======================================= + + +To run the object detection script, use the following command. Replace `TFLite_model` with the path to your model directory if it differs: + +.. code-block:: bash + + python3 object_detection.py --modeldir=TFLite_model + +A window will open, displaying the object detection model in action. \ No newline at end of file diff --git a/boards/beagley/ai/images/ObjectDetectionModel.png b/boards/beagley/ai/images/ObjectDetectionModel.png new file mode 100644 index 0000000000000000000000000000000000000000..942afb9713ae9222be6472cdcd4d8663e9364fae Binary files /dev/null and b/boards/beagley/ai/images/ObjectDetectionModel.png differ diff --git a/boards/beagley/ai/images/video_driver_get.png b/boards/beagley/ai/images/video_driver_get.png new file mode 100644 index 0000000000000000000000000000000000000000..d1b6f352a689a7d5936272c98f7eee0b100dbd10 Binary files /dev/null and b/boards/beagley/ai/images/video_driver_get.png differ