Brixton was the meeting place of the ancient Brixton hundred of Surrey. It remained mostly waste land until the beginning of the 19th century, the main settlements being near Stockwell, Brixton Hill and Coldharbour Lane.

With the opening of Vauxhall Bridge in 1816, improved access to Central London led to a process of suburban development. The largest single development, and one of the last in suburban character, was Angell Town, laid out in the 1850s on the east side of Brixton Road, and so named after a family which owned land in Lambeth from the late 17th century until well into the 20th. It was part of Surrey until the creation of the County of London in 1889.

One of a few surviving windmills in London, built in 1816, and surrounded by houses built during Brixton's Victorian expansion, is to be found just off Brixton Hill. The nearby 'Windmill' pub is named after it.