This congregation was first organized as the Zion Welsh Congregational Church December 19, 1849. By 1856, the congregation had purchased this site and erected its first building. A larger church was built in 1868. Services were held in Welsh until 1888, after which English was used, but the church remained the center of Welsh heritage in Oshkosh. In 1894 the current building was constructed, and the name changed from Zion to Plymouth Congregational.
The church is constructed of red brick, with a stone foundation. Stylistically it is similar to the Richardsonian Romanesque Presbyterian Church nearby, but the details are thinner, as the building is smaller overall. The church was severely damaged by fire after being struck by lightning in May 1959. The entrance was remodeled at that time.
Due to weathering, the original bell tower was replaced in 1969 with a small fiberglass spire. In 1979-80 the congregation built a new church at the corner of Georgia and South Park. This building has been used by other religious denominations since that time.