An open-source skill for interactive 2D and 3D lessons

Yi-Yi@yiyibuilds

A few days ago I shared the first version of Teachable Figures, and I honestly didn't expect this many people to react. Here's an update on what I built since, and why I built it this way.

Updated version here （15+ full lesson plans with interaction mode and learning history builder) : t.co/fu60t5MfnJ

- What the X community reaction taught me

The response made me realize something about AI and education. The biggest change isn't some fancy product. It's how fast we can now turn a complex idea into a version a person can actually understand.

Many of us have been waiting for the perfect AI tutor that understands you and adapts to you. That will take time, and it matters. But it's about the process: AI in the loop while you learn. What people responded to here is about the result. Not AI doing the learning for you, but AI cutting the workload of making really high-quality learning materials. Other forms of AI education aren't less important. I was just surprised by how universal this problem is.

- Teaching and learning are not the same thing

Because this is such a visual skill, many replies asked what it means for teaching versus learning. I've been asking myself the same question.

As a software engineer with a doctorate in education, here's my view: teaching and learning don't work in silos, but we do need to tell them apart.

When a teacher or parent uses this, they already have the learning plan. They do the voice-over, they explain more, and they know their learners. What they need is a precise, targeted material, made fast. The skill now serves them directly: it builds an illustration or classroom exercise and leaves the scaffolding to the human. Solo learners who already know what they want to learn get the same speed.

- Giving the tool a teacher's memory, not a teacher's job

On the learning side, I didn't want something that just looks cool.

I'm a big fan of schema theory. To me it's one of the best ways to operationalize learning science, turning pedagogy into software and user experience. New knowledge only sticks when it connects to what you already have.

So I'm not asking the tool to teach; that role still belongs to humans. Instead, I gave it something like a teacher's
memory. Through the conversation it collects what the learner already knows, what they want to know and how they like to learn. It keeps that in a few local files: a lesson registry, a learner profile, and a knowledge-point-level learning plan. The agent reads them every time before it builds anything. Drafting a reasonable lesson plan from that is something AI is genuinely good at.

- Attention is all you need (really)

When I started using the skill to build learning materials, adding exercises to the side panel was easy. The hard part was getting sensemaking. How do you make sure learners understand what they're looking at instead of just being amazed by a nice visualization?

So I went back to the visualization itself and added attention:
1. At the prediction step, the figure dims everything except what the question is about, so you know where to look.
2. On request, an ordered "look here → now notice this" sequence walks through the figure one observation at a time.
3.Legends and key terms are linked to the parts of the figure: point at a word and the part lights up, and the reverse works too, so words and pictures build the same idea.

- Learning artifacts shouldn't be static

Last, a belief I've held for a while: an interactive learning artifact should not be static. And "static" doesn't mean
"no animation". A page can be full of animation and beautiful 3D and still be static, because it never records or understands what the learner has done.

I think the learner's steps should show up in the visualization and in the artifact itself. Then the artifact becomes a record of what that learner has learned, something they can always come back to. I've started prototyping this: artifacts that evolve with the learner, where every change is checked before it ships.

Try it

There are now 15 bilingual lessons: Moon phases and a four-stroke engine in 3D, the seasons, Bayes' base rates, information entropy, natural selection, price ceilings, the Ship of Theseus, and more. All are open source.

Live: t.co/tz8BkBfJSP
Repo: t.co/fu60t5MfnJ

And a small spoiler: I'm building another one to open up the black box of large language models. Stay tuned.