Industrial automation and computer vision applications
The Industry 4.0 initiative is responsible for the dramatic growth in deploying a broad range of industrial internet of things (IIoT) devices used for industrial automation applications. IIoT devices are used to monitor and control production plant assets and are also employed to measure and analyse equipment conditions as part of a maintain, repair, and overhaul regime. Sensors are vital, ranging from temperature measurements to liquid pressure sensors and cameras for vision processing applications. Sensors are also used to monitor vibration using audio processing neural network algorithms on data acquired from digital microphones. Vision is a crucial sensing technique, too, used, for example, to ensure bottle labels are attached correctly on a bottling production line or in functional safety applications to isolate equipment should production staff enter an unsafe area.
Architecture of a vision processing solution
In any modern factory, computer-based vision processing can be found employed in a variety of different applications. Computer vision relies on the use of one or more video cameras attached to a computer system. The nature of the task will determine the complexity of the processing involved and the compute resources required. For example, an application accepting or rejecting bottles based on how accurately a label has been affixed might not require multiple cameras and intensive compute resources. An algorithm that detects the edges of the label within a pre-defined area on the bottle is relatively straightforward using software based Kalman filters. However, an application that needs to differentiate the limbs of a hydraulic press operator from the metal panels being processed and the equipment itself is more involved.
This application requires a machine learning neural network and may require several different camera angles to satisfy functional safety requirements. The convolutional neural network is best suited to vision-based object detection tasks and needs to be trained before deployment. Once trained, the use of a neural network is termed inference. The training process examines hundreds of correct and incorrect images to identify and extract features to aid classification. See an example vision processing solution architecture Figure 1.
From an application perspective, there are several practical considerations that the development team should review. These include the type of objects to be detected, the ambient light conditions in the detection zone, how fast the objects pass within the camera’s view, the speed of object identification required, and whether multiple camera angles are required to complete the task reliably.
The operating environment also needs investigation; for example, ingress protection to the camera and the control electronics are required for an application subject to dust and moisture. Also, vibration and shock may disrupt the image quality making detection less reliable.
The timing of object or feature detection is critical. Production lines are often run at high speed to maintain factory efficiency and meeting throughput goals. In the simple bottling plant example above, the computer vision application needs to capture the image accurately, process it, determine the label status in time for the keep or reject action to be taken on the identified bottle.
Introducing the Arduino Pro Portenta and the Portenta Vision Shield
The recently introduced Arduino Pro Portenta platform is an ideal microcontroller platform to prototype an industrial machine vision solution.
Initially developed as an educational prototyping platform for hobbyists and makers, the Arduino family of microcontroller development boards has continued to evolve to cater for the demanding needs of innovators and industrial equipment manufacturers.
Arduino is responsible for starting the careers of many young electronics students and is an affordable and highly versatile single-board embedded system for innovators to prototype new product ideas. Arduino’s open-source approach, coupled with its growing community of add-on extension boards and extensive software support from across the electronics industry, has established Arduino as a go-to embedded platform for many new product designs. The next stage in Arduino’s journey started recently with the Arduino Pro series of boards and modules aimed at the industrial market.
The dual-core Arduino Portenta H7 is the first of the Pro board series to meet the demands of industrial and commercial applications. The Portenta H7 is well equipped for industrial machine vision applications. It is fabricated in the popular Arduino MKR board package format allowing access to all Arduino existing accessory boards and shields. The Portenta Vision shield, available with either an Ethernet wired or a LoRa wireless interface, is ideal for industrial computer vision applications.
The Arduino Portenta H7 features an STMicro STM32H747XI low power microcontroller equipped with Arm Cortex-M7 and an Arm Cortex-M4 cores. Both cores feature floating-point units (FPU), the M7 FPU providing double-precision calculations. The M7 core can operate up to 480 MHz and the M4 core up to 240 MHz. The M4 core also has an integrated adaptive real-time accelerator GPU designed specifically for running neural network algorithms. Power management of the H7 is enabled by using three power domains that can be either controlled on/off by the application or by clock gating. A single 5 VDC source powers the Portenta, and when in the standby sleep mode with the RTC enabled, the power consumption is typically 2.9 uA. An on-board PMIC features the ability to charge a Li-Po/Li-Ion battery.
The board’s peripheral interface features include multiple I2C, USART, SPI, and CAN ports, an 8-bit camera interface, and a MIPI DSI host. Analog to digital support includes two 12-bit DACs and three 16-bit ADCs.
The Arduino Portenta Vision shield connects to the Portenta H7 via high-density connectors. The integrated very low power Himan HM-01B0 camera can provide QQVGA and QVGA outputs at frame rates from 15 fps to 120 fps. Two MEMS microphones are also integrated into the shield. The dual-core capabilities of the Portenta H7 allow running MicroPython code and Arduino C sketches on the separate cores.
Software support speeds computer vision application deployment
The Portenta H7 and the Portenta Vision shield are fully supported by an enhanced version of the popular Arduino IDE. In addition, a cloud-based editor and a command-line interface editor are also available. Complete camera and microphone library support is available through the Arduino 2 IDE, and the Arduino website features several different tutorials.
The open-source machine vision learning framework, OpenMV, and the OpenMV IDE support the Portenta boards. OpenMV is based on Python and permits the development of MicroPython-based vision applications on the Portenta H7. For example, the Arduino website hosts a machine vision classification tutorial using OpenMV and Edge Impulse.
The neural network uses image classification techniques to detect apples and bananas, but the concepts involved, and the process followed can equally apply to a bottle labelling application.
Arduino Pro Portenta – fast-tracking your industrial computer vision design
For embedded developers and industrial application designers, getting started with a machine vision project can be a daunting experience. Grasping the concepts of a neural network algorithm is enough of a challenge before selecting a suitable prototyping platform. However, the Arduino Portenta H7 and Portenta Vision shield significantly simplify the development process. In addition, with support from industry-proven and well-supported machine learning frameworks, libraries, and tutorials, the challenges of developing a machine vision application have become a lot simpler.
Arduino Portenta H7 Mirocontroller Board
Arduino Portenta Vision Shield