Reinforcement Learning with SPIKE Prime

Watch your robot learn, one try at a time
Not connected
Not connected
What to do now
Connect to your hub with USB or Bluetooth, then start the program on the hub.

The track

Bricks appear as the robot discovers them. Arrows show the move it thinks is best from each brick (darker = more sure).

Q-table

The robot's score for each move from each color it has found. Higher = better.

What just happened

The last decision, and how the Q-table changed.

Nothing yet. Start a run on the hub.

Steps to finish, run by run

Fewer steps = a smarter search. The tick on each column shows the shortest possible path from that start, once the map knows it.

Hub console

Everything the hub prints. Lines starting with >> are instructions. Lines starting with @ are data for this dashboard.

Guide: getting started, key ideas, troubleshooting

Getting started

  1. Build your robot.
    Color Search: arm motor in port B, color sensor in port D.
    Smart Walker: left leg motor in port B, right leg motor in port A.
  2. Color Search only: download and run read_color.py first. Hold LEFT, then RIGHT, to sweep the arm and check that every brick reads correctly and the gaps read as NONE.
  3. Open rl_color_search.py or rl_smart_walker.py in the SPIKE App. Check the SETTINGS at the top, then download it to a program slot on the hub.
  4. Close the SPIKE App (or disconnect it from the hub). Only one app can talk to the hub at a time.
  5. Start the program on the hub: pick its slot with the left/right buttons, then press the center button.
  6. Click USB or Bluetooth above and choose your hub. For Bluetooth, press the hub's Bluetooth button first so it blinks.
  7. Follow What to do now. On the hub, RIGHT = train and LEFT = test.

The robot works without this page too. The dashboard only watches what the hub prints, so you can connect or reconnect at any time. It catches up at the start of the next run.

Key ideas

  • State: what the robot senses. The color under the sensor, or the walker's heading zone.
  • Action: what it can do. Sweep left/right, or a pair of leg speeds.
  • Reward: a score for what happened after the action.
  • Q-value: the robot's guess of how good an action is in a state, counting future rewards too. The Q-table holds one for every state–action pair.
  • Explore vs. exploit: with chance ε (epsilon) the robot tries a random action (explore). Otherwise it picks the best one it knows (exploit).
  • Learning rate α (alpha): how far each new experience moves a Q-value.
  • Discount γ (gamma): how much future rewards count compared to the reward right now.

After every training step the robot nudges one Q-value toward a target:
new Q = old Q + α × (reward + γ × best next − old Q)

Troubleshooting

  • Buttons are grayed out: use Chrome or Edge on a laptop or desktop. Safari and Firefox can't connect to the hub.
  • The hub isn't in the list: close the SPIKE App. For USB, try another cable. For Bluetooth, press the hub's Bluetooth button.
  • Connected, but nothing happens: make sure the program is running on the hub and DASHBOARD_LINES = True in its settings. Data appears at the start of the next run.
  • Color Search finds colors that aren't bricks (an extra row in the Q-table): run read_color.py. If the gaps read as a color, add it to IGNORE_COLORS. If a brick flickers between two colors, improve the lighting or move the sensor closer.
  • Rearranged the bricks? Keep training to watch the robot re-learn (the map redraws itself), or restart the program to start fresh.
  • Wrong activity is showing: click the activity tab at the top.
  • The walker dial turns the wrong way: tick Mirror left / right.
  • Save your work: Save log downloads everything the hub printed. Open log… replays it later.

For teachers: the data lines

Each @ line is a type followed by key=value pairs, for example @STEP run=4 t=2 state=BLUE action=RIGHT reward=-1 old=0.00 best_next=0.00 new=-0.50. The types are HELLO, CONFIG, QTABLE, QROW, DISCOVER, START, CHOOSE, STEP and END. Lines starting with >> become the instruction banner. You can add your own print() lines freely. The dashboard shows them in the console.