## 19.12.07

### Image recognition and puzzle solving

I have an interest in computer algorithms and AI. I also have a two-year-old sister. How are the two related? My sister is an example of a simpler algorithm (to some extent). I was watching her solve a puzzle. Now, normally when we solve puzzles, we look at all the pieces and have an idea of what the puzzle looks like before we solve it, and so we can take the pieces and put them where they go in the mental image. My sister, being two and new to puzzles, hasn't yet figured out how to make mental images of puzzles while solving them. And so watching her solve the puzzle was rather interesting.

She put the first (of four) piece in the frame:

And then she took another and put it in:

It took me a while to figure out why she was putting it like this. But then I realized that what she was doing was taking the image of the dog on the first piece and mating it with the closest match on the puzzle. In other words, she was not looking at the background or what she thought the puzzle should look like but was rather using simple matching skills: this looks similar to this, so it must match. She eventually got frustrated and asked for help. Instead of showing her where the piece went, I told her to get another piece and try it. So she did:

Again, she wasn't looking at all at the picture (dog feet don't go sideways), but rather simply at what matched along the edges of the puzzle. This didn't work either, so she asked for help. I told her to get another piece:

This one fit. And so she picked up another and tried to fit it on:

Again, no image, even though she could see even more of the dog. She took the other one and fitted it on, and then the last one:

(for brevity's sake I condensed the last two steps into one photo. Sorry puzzle lovers)

So I was wondering if this could at least begin an AI puzzle solving thing. Train the computer to see the edge of the puzzle pieces. It picks a corner piece and starts matching from there. Match along the edges, and if they don't match well enough, discard that piece and try another. It's a brute force way of solving puzzles, but I suppose that it'd make a simpler algorithm than trying to, say, get the computer to get a 'mental image' of the puzzle and then solve the puzzle based on that.

I suppose someone's already thought of it, but I hadn't =] I think every AI designer should hang around little kids as they really show the beginnings of years of mental practice and would provide a good foundation for AI ideas and algorithms.

(for all you wondering, I used masking tape to hold the puzzle together in my scanner =])

edit: AARGH!!! Blogger, your image handling is embarrasing, to say the least, and I can't wholeheartedly support you until you fix it! To everyone else: sorry, I post by email (Blogger's post form takes 100% CPU on my machine and is abysmally slow), and in the email, the pictures were fine. Not so on Blogger (you'd think that they'd be able to attach emails from a Gmail, for goodness sake!). I went in and found that the easiest way to do images was in the HTML edit. So pardon any irregularites, etc. I'm not doing an embedded image post again for a while.