In 1913 Moses Schlotzhauer acquired the northeast portion of Lot 4 on Birmingham Street. There in 1914 he built a house which he sold to Henry M. Schaefer, who had just moved, with his family, to Stratford. That house is 136 Birmingham Street.  In the early 1900s the Schlotzhauer brothers, Moses and Edward, were well-known builders in Stratford and district.

The Schaefer family had been living in Milverton, a small town to the north in Mornington Township (now in Perth East) since 1884. There Henry was in business as a merchant tailor. This term was used in earlier times to refer to tailors who owned their own business where they stocked and sold materials for the garments they made. In a 1924 Stratford directory there were 16 merchant tailors in business here.

Henry had moved to Milverton from his 1860 birthplace in Woolich Township, Waterloo County (where his parents, immigrants from Germany had settled) shortly after his marriage to Isabella McKay. Isabella was born in Michigan and at the time of the marriage was a Milverton resident. Together they raised a family there of two sons and three daughters. The eldest son, William Henry, who would grow up to serve his country as a member of Canada's Expeditionary Force in the First World War, was killed in battle in 1918. The second son, Maurice, began his career as an apprentice with the Grand Trunk Railway, while the second daughter, Margaret, had a position as a saleslady at Duncan Ferguson's, a prominent dress maker. Her husband, Stratford resident Harold Wreford, also served overseas in the war with the Canadian Expeditionary Force. The youngest daughter, Dorothy, married an accountant from Kincardine while the husband of the eldest daughter, Ida May, was a shoemaker in Milverton.

Following his arrival in Stratford Henry experienced a career change when he was hired by the Dominion Customs office as an appraiser. He continued in that position until his retirement 17 years later. In a 1924 city directory the local customs office had a staff of seven. Henry continued to use his tailoring skills in a voluntary way — his Beacon Herald obituary notes that he spent many hours in the Red Cross workrooms during war time and subsequent depression years remaking clothing for the church relief committees.

The family's stay on Birmingham Street was brief. The city directory lists them as residing at 236 William Street in 1916. Isabella died in 1930 and Henry in 1944. They are buried in Avondale Cemetery where their son William's wartime death is remembered on their tombstone.