LEGO

Anyone Can Build (Masterfully)

I want to let anyone be a Lego Master Builder. For example, anybody should be enabled to:

• Take reference views of the Toppat Clan airship, and create a 42-stud-long digital LEGO airship and parts list.
• Find all part 2780 pins in a 78-piece pile image and label the five matches.
• Reconstruct a digital LEGO tank from video frames and adapt its style to a custom faction.

Let Kids Play

I spent 23 years training to be a master builder. Each example above is something I did myself. Yet even today, organic shapes are hard for me to build.

Wouldn’t it be wonderful to give people a tool that helps them master Lego building? In addition to finding communities online, growing their collection large enough to experiment, and making the time to learn by trial and error?

Save Me Time & Build In Public

A system that saves me time sorting pieces out after building is the highest leverage improvement to my quality of life. After that I’ll reuse the system to achieve the other goals (img->Lego, ...), which helps me create the Lego animation studio my brother and I dreamed about running as kids.

We want to help people become Lego Master Builders. Posting publicly about my work does two things to accelerate reaching that goal:

• If someone else tries to build this, failed experiment logs save them time.
• If someone really cares, they might donate a gpu to speed up my experiments :)

(seriously, email me or tag me on twitter)

Also I want to try visualizing experiments as version control trees! Branches that spread, eventually the best ones merging into the main baseline.

Approach

I’m a scientist, I like science - Richard Sutton’s bitter lesson suggests scaling compute and data outperform human hand-crafted heuristics. But Lego is a discrete and continuous verifiable orientation problem. I’ll start with recognition and move to the harder problems after that.