Toronto of the 1860s was a burgeoning industrial and commercial hub thanks mostly to the construction of major railway lines in and out of the city, which got underway on a grand scale the previous decade. With connections to Montreal, the Atlantic colonies/provinces and various U.S. cities completed or well underway, what had been the modestly populated town of York just 30 years before was now a rapidly growing industrial city.
With the rise of industry in Toronto, there was a pronounced shift in class structure. As historian Carl Benn explains, "Toronto's rise as an industrial city saw the accompanying emergence of industrial classes in place of the older hierarchies that had divided society. Families like the Gooderhams, Masseys, and Eatons formed a commercial-industrial elite. Below them a large middle class developed, as did a significant working class and a smaller underclass. Much of the working and underclass lived in marginal conditions because of unemployment, infirmity, age, or other affliction at a time when social services were in their infancy."
Architecturally speaking the city was still very much Georgian, though the style was no longer in fashion for new construction. Notable additions to the city included (but were not limited to): the Gooderham Worts Malt House, the Don Jail, St. Peter's Church, Spadina House and Euclid Hall (now the Keg Mansion). As was the case a decade later, the commercial and political centre of the city was located to the east of Yonge Street, in the Market District.
With Confederation in 1867, Toronto became the capital of the province with the highest population in the country, a mantle it obviously still holds. And though it doesn't much look the part in the pictures below, Toronto had taken the first steps toward becoming the international city it is today.
Here what it looked like back then (captions above each image).
Gooderam and Worts (painted 1890s, depicts a scene circa 1860)

Agricultural Hall

Provincial Lunatic Asylum Right Wing

Bank of British North America

Bank of Toronto

College Avenue (Now University Ave.)

The Don Jail

View from St. Lawrence Hall, looking east

Great Western Railway Station

Mechanics Institute

Ontario Bank

Osgoode Hall

Looking north from the St. Lawrence Market

Post Office, Toronto Street

Queen Street Bridge (over the Don River)

Rossin House Hotel

Shakespeare Hotel

Taddle Creek?

Toronto City Hall (1868)

Toronto General Hospital

Toronto Normal School

Davenport Station (on what is now Caledonia Rd.)

Rolling Mills

Check out the rest of the series here:
Images from the Wikimedia Commons, Toronto Archives and the Toronto Public Library.