I am pretty certain that orphanage records from less than 100 years ago would be closed, though I am not 100% certain. I am certain that health board records are closed for 100 years, and that adoption records are also closed.
I wonder why, if his parents were married in Edinburgh, he was born in Edinburgh and his mother died in Edinburgh, he would have been sent to an orphanage in West Lothian? His father was born in Midlothian and his mother in Angus (Dundee) so there is no obvious link to West Lothian.
The catalogue of the National Records of Scotland does have some material relating to Wallhouse, but it mostly seems to be inspectors' reports and a request for a public enquiry into the home. Have a look for yourself - go to
https://catalogue.nrscotland.gov.uk/nrsonlinecatalogue/welcome.aspx and search for Wallhouse using reference starting ED11.
If I were you I would start by asking the West Lothian Local History Library
https://www.westlothian.gov.uk/article/44856/Local-History-Library whether or not there are records of children who were placed in Wallhouse, and if so how they can be accessed for research purposes.