Walterboro Library Society Building is a historic library building located at Walterboro, Colleton County, South Carolina. It was built in 1820, and is a small, white, Federal style frame building with a side-gabled roof. The front façade features a Palladian-style door surround capped by a fanlight. The building is occupied by the Colleton County Historical Society. When Walterboro was incorporated in 1826, the town boundaries were established as “3/4 of a mile in every direction from the Walterboro Library.”

It was listed in the National Register of Historic Places in 1971. It is located in the Walterboro Historic District.