Initially named Wicaco, Southwark is the oldest settlement in Philadelphia. King Gustavus Adolphus of Sweden granted a charter for settlement in 1638, and Swedish settlers were already established before William Penn's arrival. The Southwark district was created in 1762, named after a neighborhood in London. In the 1960's, the district became known as Queen Village.

The original Southwark Library was a Carnegie Building located at 5th and Ellsworth on land donated by the Young Man's Institute. It opened on November 15, 1912.

The Library opened in its current location on November 1, 1963. It was renovated in 1998 as part of the "Changing Lives" campaign, which refurbished branches and ensured Internet access.

The library was re-named on March 26, 2004 to honor Charles Santore. A South Philadelphia native born in 1910, Santore was a former boxer, Republican ward leader, and founder of Municipal Employees Union Local 696. At the time of his death he was the oldest active ward leader in any party.

The library is a short distance from the Italian Market on 9th Street. There is a lovely courtyard, accessible from the children's room, which is perfect for relaxed reading.