Is this how the brain renders 3D scenes?

01 / THE QUESTION

Why draw every leaf?

Look at a pile of leaves and you don’t see a thousand leaves. You see a pile. Every leaf has its own shape and color, yet the whole thing reads as one surface. Somehow your brain makes all that detail simple.

Video games cheat at this. Instead of modeling every leaf on the ground, they repeat one small patch of texture, flipped or rotated, until a little data covers a whole forest floor.

Maybe vision cheats too. Not by tiling; real leaves are too messy. But perhaps by noticing what repeats and spending its effort on what doesn’t.

What made me curious is something you can see yourself. Look closely at one leaf and its quirks stand out: a torn edge, a brown spot. Look across the pile and the pattern comes forward. Sometimes focusing on one leaf seems to break the pattern. Why would it?

I don’t know if this is how the brain renders a scene, but it seemed worth poking at. So I built Symmetry Studio, which makes textures from a small patch and a few repeating rules.

02 / TRY IT WITH LEAVES

Start with real leaves, not my pattern.

Try this on something ordinary first: leaves, bark, or the photo below. Nothing in it is tiled or mirrored, so if you notice a change, my code didn’t put it there.

Look straight at one leaf and trace its outline. Let your gaze drift slightly to the side and stop picking out individual leaves. Take in the whole pile. Then look back at the detail.

Does anything start to repeat: a shape, a mirrored arrangement, a rhythm? Does it fade when you inspect one part? Switch back and forth and see if it returns.

Now a harder version. Keep your eyes still and move only your attention, from one leaf to the texture around it. Moving your eyes and moving your attention are different things, and I’d like to know whether each matters.

If something changes, that’s worth explaining, but it won’t tell us the cause. If nothing changes, that’s worth knowing too.

03 / WHY EFFICIENCY?

A patch and a rule can describe a whole field.

A generated texture needs only a small patch and a rule for repeating, reflecting, or rotating it. That’s enough to fill an image of any size. Could vision use a similar economy?

Real textures are messier. A pile of leaves isn’t wallpaper. The question isn’t whether the brain tiles the world, but whether it can capture what repeats and still keep the differences that matter.

There’s research pointing this way. Kohler and Clarke’s 2021 study found that brain responses and detection thresholds reflected the relationships among wallpaper symmetry groups. They suggest that efficient coding, the idea that the brain represents what it sees economically, may extend to the redundancy in symmetry. That’s a reason to take symmetry seriously. It doesn’t prove my story about the leaves.

Other work on efficient coding of natural texture statistics links how often texture features appear in nature to how sensitive people are to them. So efficiency is a sensible thing to look for.

What got me started was Qualia Computing’s “Kaleidoscopic Integration”, which pulls together writing on symmetry, compressed descriptions of experience, and Steven Lehar’s ideas about standing waves in visual representation. It gave me a way to connect a texture experiment to how we represent a whole scene.

04 / SYMMETRY STUDIO

Change the patch, or change the rule.

The studio repeats a small crop of an image using one of the seventeen wallpaper groups. These classify every way a pattern can repeat in two directions across a flat surface, using shifts, rotations, mirrors, and glide reflections (a mirror plus a shift).

Keep the patch and change the rule, or keep the rule and change the patch, and watch a small piece become a large pattern.

For the attention experiment, open the leaves example, choose a patch, and pick a group. Set Focus depth to zero so the image stays put while you change how you look at it. Try the blur control separately.

The studio puts the construction in plain view: a patch, a rule, a texture. Compare that with what you saw in the real photo. Your photos never leave your browser.

05 / WHAT WOULD TEST IT?

Can you make it happen twice?

A real test would compare ordinary photos, symmetrized versions, and irregular controls at the same scale. I’d record what people notice looking directly and slightly away, including when they notice nothing. And I’d ask for their description before mentioning symmetry, so I don’t plant the answer.

There’s a complication. A 2024 study of attended and unattended symmetry found neural responses in both cases, slightly stronger with attention. So if focusing seems to “break” a pattern, that can’t simply mean attention switches symmetry processing off.

Is symmetry part of what makes texture cheap to represent? Is the shift you see in leaves a clue, or does something else explain it? I don’t know yet. That’s what I want to find out.

Try it on an ordinary texture. If something changes, tell me what you looked at, where your eyes were, and whether you could make it happen again.