Robotic Arm Specification

The robotic arm integrates speech recognition, computer vision, a large language model, and real-time servo control into a single coordinated pipeline running on a Raspberry Pi Zero 2W. Voice commands are captured via smallest.ai’s Pulse STT and camera frames are streamed through OpenCV, both feeding into GPT-4o which issues structured JSON commands. A Python orchestrator dispatches these commands to a custom Inverse Kinematics engine that maps 3D coordinates to joint angles, which are then executed through a PCA9685 PWM driver connected to MG996R and SG90 servos.

End-to-end data flow from user input through AI processing to physical servo motion:

  +---------------------------+     +---------------------------+
  |   smallest.ai STT         |     |   USB Camera + OpenCV    |
  |   (voice commands)        |     |   (base64 frame capture) |
  +------------+--------------+     +------------+--------------+
               |                                |
               |                                |
               v                                v
  +------------------------------------------------------------+
  |                    GPT-4o (Brain)                           |
  |         "pick up the orange"  ->  JSON commands            |
  +----------------------------+-------------------------------+
                               |
                               v
  +------------------------------------------------------------+
  |          Python Orchestrator (Raspberry Pi Zero 2W)         |
  |              Parse -> Validate -> Dispatch                  |
  +----------------------------+-------------------------------+
                               |
                          (x,y,z)
                               |
                               v
  +------------------------------------------------------------+
  |          Inverse Kinematics Engine                          |
  |          trig/geometry -> theta_1,2,3,claw                  |
  +----------------------------+-------------------------------+
                               |
                         I2C duty cycles
                               |
                               v
  +------------------------------------------------------------+
  |       PCA9685 PWM Driver Board                              |
  |       Channel 0-3 -> 50 Hz servo pulses                    |
  +----------------------------+-------------------------------+
                               |
                               v
  +------------------------------------------------------------+
  |   MG996R (BASE)     MG996R (SHOULDER)  MG996R (ELBOW)      |
  |   SG90 (CLAW)       5V 5A PSU  (common ground with Pi)     |
  +------------------------------------------------------------+

The bill of materials for the robotic arm system:

4.1 Python Orchestrator

The orchestrator runs on the Raspberry Pi, receives text from smallest.ai STT and Base64-encoded frames from OpenCV, sends them to GPT-4o, parses the returned JSON command array, validates each command, and dispatches target (x, y, z) coordinates to the IK solver.

4.2 Inverse Kinematics Engine

Custom Python IK solver using trigonometric geometry:

Joint Space:
  theta_1 = Base rotation (yaw)        - MG996R @ CH 0
  theta_2 = Shoulder elevation (pitch) - MG996R @ CH 1
  theta_3 = Elbow flexion (pitch)      - MG996R @ CH 2
  claw    = Gripper open/close         - SG90   @ CH 3

Pipeline:
  1. Receive (x, y, z) target
  2. theta_1 = atan2(y, x)
  3. r = sqrt(x^2 + y^2)  (sagittal projection)
  4. Solve 2-link IK via cosine law (shoulder + elbow)
  5. Return [theta_1, theta_2, theta_3, claw]

4.3 JSON Command Protocol

GPT-4o issues structured JSON. All coordinates in millimeters, angles in degrees.

{
  "commands": [
    { "action": "move", "x": 10, "y": 15, "z": 5 },
    { "action": "grab" }
  ]
}

Available Actions:

4.4 Data Flow Diagram

End-to-end signal routing:

[USER VOICE]                     [CAMERA]
     |                               |
     v                               v
[smallest.ai STT]            [OpenCV capture]
     |                               |
     |                    Base64 encode frame
     |                               |
     +-------> [GPT-4o API] <--------+
                    |
            JSON command array
                    |
                    v
          [Python Orchestrator]
                    |
           Parse & validate JSON
                    |
                    v
            [IK Solver Engine]
            trig/geometry ----
                    |
        [theta_1, theta_2, theta_3, claw]
                    |
                    v
           [PCA9685 PWM Driver]
           I2C duty cycles ----
                    |
                    v
           [MG996R / SG90 Servos]
           Physical arm motion