Strawberry Harvester.

Replacing command-line robot controls with an interface a farmer can use in direct sunlight.

Role
Product and UX design, project manager, scrum master
Team
5-person senior design team, UCF
Timeline
May 2026 – present
Tools
Figma, React, TypeScript, Jira
Sponsor
Autonomous Robots and Controls Lab (ArcLab), UCF
Home screen with four large actions: Calibrate, Run (greyed out), History, and Manual, plus a status sidebar.
Four actions. Run is greyed out until the robot is calibrated.

The problem

The lab’s robot works. Computer vision finds the ripe berries and a robotic arm picks them.

Starting it was the problem. Before the robot picked anything, the operator had to do all of this:

  1. Power on the hardware.
  2. Launch a Python script.
  3. Start MATLAB.
  4. Install an IDE extension.
  5. Set the file paths.
  6. Answer prompts in a command window.

The lab showed the robot to farmers and got little interest. The hardware was not the obstacle. The setup was.

My job was to build an interface that lets a farmer start, check, calibrate, run and review the robot without using a command line.

Who it is for

The product has two kinds of user.

Farmers and operators have low to mixed technical literacy. The design targets operators aged 55 and up. They work outdoors, on a laptop, in glare, sometimes in work gear.

The lab’s engineers are mechanical engineers. They need enough system detail to diagnose a fault and to extend the robot.

So the interface stays simple on the surface, and shows the detail at the moment something fails.

Where the references came from

I found few good interface references for outdoor robotics. Most of them were dense and hard to read. I used music software instead.

Digital audio workstations and hardware synthesizers put many controls on one screen and stay readable at a glance. An operator in a field needs the same thing. I kept a reference library of layouts and control schemes, and used it throughout the project.

That gave me one rule. Where appearance and readability conflict, readability wins. The screen has to work in direct sun.

Flow before frames

I mapped the operator’s journey with a teammate before I designed any screens. The map showed the gaps early.

  1. First launch
    1. First-time setupbed layout and defaults
    2. Homethe hub. Run stays locked until the robot is ready.
      • Calibrate
        1. Equipment checklist
        2. Power checklist
        3. Diagnostic warm-up
          • Issue found Startup issue fix Try againreturns to the warm-up check
          • All clear Robot ready
        4. Choose calibrationautomatic by robot height, or manual arm calibration
        5. Checking calibration
        6. Calibration completereturns Home, and Run unlocks
      • Run
        1. Runtime statisticsmap or camera view, with an emergency stop
        2. Stopautomatic on a full basket, low battery or fault, or manual
        3. Run completesummary, then Home
      • Historypast sessions, export data
      • Manualthe built-in user manual
      • Settings
  • One hub. Every path starts at Home and returns to it.
  • Run stays locked. It is greyed out until the robot passes diagnostics and is calibrated, so the operator cannot start a harvest in the wrong order.
  • Errors name three things. The system that failed, where it sits on the robot, and the next action. The operator never sees a fault code.
  • Each run ends in a summary. The summary has a route back to Home.
One-time setup form for claw height, log path, row spacing, number of rows, row length, row width, and bed height.
The values the command window used to ask for, collected once in plain labels.

Key screens

Connect the robot equipment: a checklist of the laptop charger, ZED2 camera, Raspberry Pico, Arduino, rangefinder, power train, battery, manipulator arm, and starting point, with a single confirmation box.
Each physical connection is named and confirmed before boot-up.
Boot-up screen checking the camera connection, arm controls, wheel controls, and robot system one at a time, with a warning not to power off during warm-up.
The robot checks each system in turn, on screen.
Startup issue screen: camera and wheels pass in green, arm controls fail in red, the claw is circled on a line drawing of the robot, and the diagnostic reads that the claw cannot move down to the bed.
The system that failed, its location on the robot, and one action.
Confirmation screen reading Robot is ready, with camera, arms, wheels and control system all listed as online.
The same check, passed. Success is stated, not implied.
Select calibration method, offering automatic calibration by choosing the robot's height, or manual calibration.
Two paths in plain language. The automatic one asks a single question.
Arm calibration screen with two target positions beside a live camera feed showing strawberries labelled mature and immature.
The live camera feed sits beside the target positions.
Calibration complete screen with a green check and the live camera feed alongside.
Calibration is confirmed before a run can start.
Runtime statistics with a field map showing the robot's position, time running, strawberries picked, battery remaining, and a large red emergency stop.
Time, count and battery at a size that reads from a distance. The stop is the largest control on screen.
The same runtime statistics with the live camera view in place of the field map, showing strawberries labelled mature, immature and flower.
The same layout, with the camera view in place of the map.
Run complete summary showing 128 strawberries picked, total runtime, battery remaining, and a Return Home button.
Each run ends in a summary that stays on screen.
Harvest history table with total picked, sessions and average, above per-session rows listing date, duration, picked, attempts, bed segments and claw height.
Every session is logged, and the data can be exported.

Iterating with the sponsor

I demoed the prototype to the lab’s director and asked for feedback on layout, type, colour, flow, and missing information. That feedback reshaped the product.

  • Bigger type. Operators may have eyesight issues, and summer glare makes small text harder to read.
  • Boot-up and calibration merged. Calibration moved into the startup sequence instead of running after Run is pressed, so pressing Run now starts harvesting immediately. That is why the first card on Home is Calibrate.
  • More setup detail and safety checks. Extra field settings, more pre-boot checks, and an alert when the strawberry basket is full.
  • Pop-ups became full-screen steps. Boot-up and calibration started as modals. Full-screen made them larger and easier to follow one step at a time.

One system, not per-screen decisions

I made these decisions once, for the whole interface, so no screen invents its own version.

  • Canvas. 1440×900, matching the field laptop. Early modals shared one fixed footprint so they never jumped around.
  • Type. Inter only, in four weights, with large display sizes for the numbers that must read from a distance.
  • Colour. Green means healthy or ready. Blue means the control does something. Red is only for failure and the emergency stop. No colour does two jobs, which matters in a flow full of yes or no branches.
  • Shape and spacing. Mostly sharp corners for an equipment-grade feel, pills only for status chips, and a 4px spacing grid to keep dense screens orderly.
#FFFFFF #121823 #6B7481 #E0E2E6 #4CAF50 #269C59 #217D4A #E6F6EA #4469C5

Design to code

I built the interface in Figma as reusable components. I named the frames in PascalCase to match their React components, so the engineers could build from them directly.

My role

I held three roles. As project manager I was the main sponsor contact, and turned a broad brief into scoped problems. As scrum master I built the team’s Jira workflow, then rebuilt it with the team after the first version confused everyone. As designer I owned the user flow, the interface and the design system.

What is next

The interface has not been field-tested with farmers yet, so there are no outcome numbers to report here.

  • Tighten the type scale into a strict set of named sizes. A few one-off sizes crept in during fast iteration.
  • Consolidate the green shades into fewer named tokens.
  • Test with farmers in the field, and measure setup time and errors against the old command-line process.

Figma, React, TypeScript and Jira. Senior design project at UCF, sponsored by the Autonomous Robots and Controls Lab.