The question of which is the city's best falafel is a contentious one. The last restaurant review I wrote laid out my choice and arguments, but I realized before it was even written that something stuck up in northern North York hardly even qualifies as the same city to many people.
When confining yourself to the downtown core, or when restricted because of circumstances beyond your control, I'll admit that things can get a lot hazier. I think personal taste becomes a much larger factor, and opinions can quite reasonably range all over the place.
Personally, some of the best falafel I've found after quite a bit of searching is Pita Village, located in the food court of the Village by the Grange (across McCaul from the AGO and OCAD's main buildings).
It took a lot of time before I found out about Pita Village, and I was very unsatisfied with the quality and value of most of the falafel offerings I'd encountered in and around the central commercial core. Every time I got a falafel, I felt like I was settling, getting ripped off, or both.
It wasn't until an OCAD-student friend mentioned Pita Village to me that I even first set foot into the building. While the location is really quite central, being essentially right at Dundas and McCaul, the business is nonetheless hidden from public view and for the most part serves a specific target market.
This target is reflected in the student price specials offered, but what I was more interested in was the array of toppings.
I find the biggest complaint I have with most falafel places is that - as far as what I'm used to goes - they're woefully lacking in topping diversity.
This is what made me want to check out Pita Village in the first place, and it has made me go back consistently when I'm in the area. The pricing is pretty average and while I do like the final product, I know some people find other, simpler falafels from around downtown to taste better.
The inclusion of carrot in the offerings is pretty unusual, and as you can see in the picture, the falafels can get so full that you lose sight of the actual falafel component.
I can't promise you'll fall in love, but I definitely do recommend checking Pita Village out if you find yourself in the same downtown falafel boat I always did. It may turn out that they offer a type of falafel you didn't even realize existed in the area.