AI vision and robotic automation for piccolo tomatoes

The challenge

Piccolo tomatoes are deceptively difficult products to automate. Unlike manufactured objects, every bunch is different. Tomatoes vary in size, position, ripeness and weight, while stems and branches form complex three-dimensional structures that must be understood before a robot can safely handle or cut them.

The system therefore had to do much more than simply detect tomatoes. It needed to understand the complete scene and turn that information into decisions a robot could execute — for one of the largest vegetable producers in Norway, whose greenhouse crops include many tomato varieties alongside cucumbers, snack peppers and chili.

The R&D project investigated automation of

• Ripeness assessment
• Crack and defect detection
• Individual tomato detection
• Weight estimation
• Stem and branch detection
• 3D localization
• Safe robotic handling
• Identification of suitable cutting points
• Pruning decisions
• Packaging and target-weight planning

What the AI needed to understand

Solving this required combining two complementary types of camera data: appearance, drawn mainly from 2D imaging, and geometry, drawn mainly from 3D imaging. Only by combining both could the software turn camera observations into decisions a robot could act on.

2D vision — appearance

Used primarily for information derived from how each tomato looks:

3D vision — geometry

Used for spatial understanding of the scene:

Camera → Perception → Scene understanding → Decision → Robot action

• Colour and ripeness
• Cracks
• Visual defects
• Detailed surface inspection
• Tomato position
• Bunch geometry
• Stem and branch geometry
• Depth
• Robot grasp locations
• Cutting locations
• Collision avoidance

Two Generations of Robotic Development

Byte Motion spent several years developing and production-testing AI, vision and robotic automation for tomatoes. Two generations of robotic systems were built, each testing a different way of turning perception into reliable production output.

Generation 1 — Multi-Robot Production Concept

The first architecture distributed the process across multiple robotic and process stages, investigating a production flow involving:

These operations were strongly interconnected. A cutting decision, for example, changes the available tomato bunches and therefore affects downstream weight planning and packaging. This meant that errors or uncertainty in an upstream operation could propagate through the rest of the production line.

• Inspection and localization
• Cutting and separation
• Quality assessment
• Weight planning
• Packaging

Generation 2 — Integrated Robotic Cell

The next generation moved toward a more integrated architecture, built around:

Freshly harvested tomato crates were scanned using 3D vision, and the software determined suitable robot targets while accounting for tomato, stem and branch geometry. Tomatoes were then inspected using high-resolution 2D imaging for cracks and visual quality.

AI models identified the individual tomatoes and the relevant parts of the bunch. The software used this information to determine pruning and cutting decisions and to create a packaging plan, executed by the custom gripper designed to support both cutting and packaging operations.

Tomatoes that the automated process could not handle confidently were diverted for manual handling, rather than forcing an uncertain robot action.

• Robot: ABB IRB 1200
• 3D cameras: 2× Zivid
• 2D cameras: 2× Genie Nano
• Material flow: Conveyor and industrial sensors
• End effector: Custom multi-purpose gripper
• Functions: Integrated cutting and packaging

The Intelligence Behind the Machine

Byte Motion's primary responsibility was the intelligence of the system — transforming camera data into reliable production decisions and robot actions.

Byte Motion — the brain

Automation partners — the machine

Robotic hardware, grippers, mechanics and complete machine integration are developed together with specialist automation partners. Byte Motion develops the brain of the system; automation partners build and integrate the machine around it.

• AI and computer vision
• 2D and 3D camera architecture
• Camera selection
• Camera calibration
• Object detection and segmentation
• Tomato and bunch understanding
• Ripeness classification
• Defect inspection
• Weight estimation
• Stem and branch detection
• 3D localization
• Cutting and pruning decisions
• Robot target generation
• Packaging planning
• Production software
• Data collection
• Dashboards and production statistics
• Integration between vision software and robotic automation

From Camera to Robot Action

• Tomato bunch
• 2D + 3D cameras
• Byte Motion AI & Vision — Ripeness · Defects · Geometry · Weight · Stem detection
• Decision & Planning — Pick point · Cut point · Pruning · Packaging plan
• Robot
• Packed tray / production data

What We Learned

The project demonstrated that modern AI and vision can solve many of the perception and decision-making tasks that previously required experienced human operators.

It also demonstrated that solving the vision problem is only part of successful agricultural automation. Overall production reliability depends heavily on gripper design, mechanical handling, product presentation, biological variation and the interaction between individual process stages.

These findings drove the evolution from the original multi-stage concept toward a more integrated robotic architecture, and gave Byte Motion practical experience that can now be reused in future food and agricultural automation projects.

R&D Outcomes

AI perception demonstrated

Detection and interpretation of tomatoes, ripeness, defects, stems and branches using combined 2D and 3D vision.

Robot-ready decisions

Vision data was converted into robot targets, cutting decisions, pruning actions and packaging plans.

Production experience

The system was tested in a real production-oriented environment rather than only on laboratory datasets.

Integrated production data

The system collected production statistics, including tray output and items diverted for manual handling.

Foundation for future systems

Several years of development created reusable AI, vision and automation knowledge for new tomato and produce applications.

From Piccolo Tomatoes to Other Produce

The original project focused on piccolo tomatoes, but much of the underlying technology is applicable to other visually complex agricultural products. Modern systems can reuse the same core approach — combining AI, 2D/3D vision and robotic decision-making — while cameras, robots, grippers and models are selected specifically for the new product and production environment.

Potential produce applications

Potential process operations

• Piccolo tomatoes
• Vine tomatoes
• Cherry tomatoes
• Other tomato varieties
• Peppers
• Cucumbers
• Fruit
• Other vegetables
• Inspection
• Ripeness assessment
• Defect detection
• Grading
• Sorting
• Picking
• Cutting
• Pruning
• Weight estimation
• Packaging

Developing a New Automation Application

Difficult agricultural automation normally starts as a staged development project rather than an immediate fixed machine sale.

Understand the product and process

Review production requirements, product variation, throughput, quality criteria and existing machinery.

Collect real production data

Request photos, videos and representative product samples. Mobile-phone photos and videos are sufficient for an initial assessment. We also recommend photographing individual products separately as well as showing the complete production flow.

Feasibility and pre-study

Test cameras, AI methods, handling concepts and key technical risks before committing to a full production machine.

Industrial integration

Byte Motion works together with robotic and automation partners to design and integrate the complete solution.

Projects involving previously unsolved automation are often structured as staged R&D collaborations. Depending on the customer, project and country, innovation grants or other public development funding may be available to help support parts of the development work.

What hardware was used?

The second-generation cell used an ABB IRB 1200 robot, two Zivid 3D cameras for geometry, two Genie Nano 2D cameras for appearance, a conveyor with industrial sensors, and a custom multi-purpose gripper for cutting and packaging. Byte Motion supplied the AI, vision and decision software; automation partners built the machine.

Is this system in commercial production?

No. It was a multi-year, production-tested R&D project for a major Norwegian vegetable producer, not a live customer installation. Two robot generations were built and tested in a production-oriented environment; the resulting AI, vision and automation knowledge is reusable for new tomato and produce applications.

Related guides