Maybe Martha did not know where she was born, or made up a location; it happened. Could there be something similar in Derbyshire, that a Welsh enumerator has made a mistake with? If my theory is right, finding the birth name of Martha may not be possible; was Holmes her birth name or married name? Travellers did move long distances, even between birth and baptism. Many worked from Lancashire to Cornwall, and for some, they also ventured into Wales. Llantarnam is not far into Wales and there were ferries across the rivers.
Until reaching Hay, the places they visited, are mainly river based. Boat people were tough hard workers, taking goods as quickly as possible though the canal/river system. Most were heavy drinkers and fighters. The whole family would have lived on the boat. They also needed to eat, so selling fruit and vegetables as they passed suited both sides. Joseph would pick up large bags of vegetables from farmers. By the time Joseph lived in Hay, he was probably too old physically to mix it with those on the boats. The rivers were important carriers of goods throughout the country, with the Usk navigable to Brecon and the Wye navigable to Monmouth, though smaller boats could have gone further. It would have been easier selling around the streets of Hay and Talgarth.