Case Objective

Program the CoCube robot to recognize traffic sign cards and automatically perform corresponding actions, realizing a smart traffic creative sandbox.

Understand artificial intelligence technology in image vision during the process.

1. Materials Used

2. Software Platform

MicroBlocks-CoCube

Or directly open the programming environment with the AI Vision Perception module added: MicroBlocks-AI Camera

3. Algorithm Knowledge

Algorithm Introduction

Identify whether there are specified card patterns in the image, returning information such as card coordinates, size, classification labels, etc. Including traffic signs, graphic symbols, and numbers. The classification labels are shown in the table below.

Traffic Signs

Configuration Parameters: None

Return Results

This algorithm supports simultaneous recognition of multiple cards. Cards rotated within 30 degrees can still be recognized; excessive rotation cannot be recognized.

When reading registers through the main controller, the following data will be returned:

Usage Tips

1. This algorithm can detect cards at a long distance, but this is not always the desired detection position for users. At this time, "card width" can be used to exclude those distant cards. For example, actions will only be triggered when the card width > 50%.

2. When there are multiple cards in the image (e.g., a row of cards), their detection output order is based on the card’s center point, scanned row by row from the top-left corner (0,0), from top to bottom, and from left to right.

Block Descriptions Needed

1. Sentry2 Initialization Block

An optional parameter is the i2c address. Default is 96 (0x60).

Before using Sentry2, execute the initialization block first. Typically placed under the "when started" hat block.

2. Sentry2 Set Mode Block

The mode needs to be set to "card," i.e., traffic sign card recognition mode.

3. Sentry2 Detection Result

Before using this block, ensure that the card algorithm mode has been enabled.

This block is also used to trigger detection; only after using this block can you obtain detection results.

The returned result is the number of results currently identified by the card algorithm.

4. Sentry2 Detected Card

Returns the tag attribute of the detected object id. This block determines whether the object with ID 1 is a forward card—if yes, return True; otherwise, return False.

5. Sentry2 Detected Object Attribute

Returns the attributes of the detected object id, including the color block center x coordinate, color block center y coordinate, color block width w, color block height h, and tag.

Each tag corresponds to a different card category (such as red light, green light, numbers, etc.)

4. Start Programming

1. Connect Device: Connect MicroBlocks IDE to the CoCube robot via wired or wireless means. Connect the metal contacts of the gripper module to CoCube's contacts magnetically.

2. Load Block Library: Load the CoCube external module library and the Sentry2 AI camera library. Refer to All-in-One Kit.

3. Camera Module Initialization: Choose to start by enabling the external module power, then wait 4 seconds for the camera module to successfully start before initializing the I2C interface. Then set the camera's algorithm mode to "card" mode to detect card information.

4. Recognize Cards and Perform Logic Processing: First, we need to use the "Sentry2 Detected Card Object ID ##" block, which determines whether the tag detection result of the card with id sequence number "##" is "**".

For ease of starting, we can set up logic to begin judgment after pressing button A. Now, just use a combination of loops and conditional judgments to implement different logical processing for different card tags, as shown in the figure below, including handling stop at red light, go at green light, and parking:

5. Challenge Yourself

The camera's card mode also supports several other tag recognitions, including speed limits, turns, etc. Try adding them!

Reference code: Case 02 Smart Traffic