These are a bit mishmash of items over quite a span of years, so I'll leave it up to you to figure out if it's your Horace.
I did also come across a Horace William Bailey, of Little London, Spalding, who was in front of tribunal in 1941 as a conscientious objector. He was a Methodist Sunday school teacher and preacher but he was 38 and looks like he married Doris Evelyn Webster in Sep 1928; so probably not your Horace.
Sheffield Daily Telegraph07 August 1909
Mid-Derbyshire
At a meeting of the members of the New Mills Higher Education Trust, on Thursday night, scholarships tenable at the Secondary School were awarded to Dorenda Booth, George McNee, George Goulding, Marion Clayton, Fred Brocklehurst, and Alice Crabtree.
Yorkshire Post and Leeds Intelligencer29 June 1928
The Rev A. L. Phillips, who has been Vicar of Blyth, near Woksop, for 16 years, has accepted the living of Ware, Herts.
The Rev. Horace Bailey, assistant Primitive Methodist minister at Selby, has been appointed to Thornley, Durham.
Hartlepool Northern Daily Mail02 September 1929
THORNLEY
P.M. Circuit Meeting - The Primitive Methodist Church circuit quarterly meeting was held in the Wheatley Hill Primitive Methodist Church on Saturday, the Rev. John Dyson (Thornley) presiding. The secretarial duties were carried out by the Rev. Horace Bailey (Haswell). A circuit membership of 583 was reported. Coun. Wm. Snaith (Wheatley Hill), the circuit junior steward, presented the financial statement, which showed a large deficit. It was decided to seek permission to borrow £100. The income from the last circuit effort was £11 7s 7d. It was decided to hold a competitive musical festival in February, choirs in the district to be invited to compete. A shield is to be presented to the successful choir.
Sheffield Independent29 July 1937
£250 CHALLENGE
Progress With Frecheville Church Plans
Efforts are still being made to establish a Methodist church for the Frecheville Estate near Sheffield.
As was reported in the "Daily Independent" in June, Alderman Edwin Peat, chairman of the Chesterfield Rural District Council and a member of the Derbyshire County Council, has promised to give £250 to start a building fund if five other people would give similar amounts.
The Rev. Fred Wimbrush, superintendent minister of Surrey street Methodist Circuit, Sheffield, in which the church will come, has accepted the challenge.
In September Pastor J. W. Martin will take charge of the church under the direction of Mr. Wimbrush and not the Rev. Horace Bailey, as was thought.
"The Rev. Horace Bailey is succeeding the Rev. Walter Tunley in charge of the Intake section of the circuit," said Mr. Wimbrush yesterday. "Mr. Tunley is retiring and leaving the district."
Lancashire EveningPost 31 August 1946
Public Notices
Gregson Lane Methodist Church
Harvest Services, Sunday, September, 1st. Afternoon 2 30, Evening 6 30. Preacher: Rev. Horace Bailey. Soloist: Miss Irene M. Bennett (of Chorley). Collections for Circuit Funds.
Lancashire Evening Post02 November 1946
Public Notices
Deepdale Mill Street Methodist Church - Aniversary, Sunday, November 3rd, 10 45-6 30. Rev. Horace Bailey. At 2 15. Musical Service by Miss Baxendale, soprano; Miss Haslam, contralto; Mr S. Desmond, tenor; Mr S. Heaps, baritone. Accompanist: Mrs Chadwick. Chairman: Mr F. G. E. lewis. Collection for Trust Fund.