Richard, a very warm welcome to Rootschat. This is certainly a blast from the past.
Oak Terrace appears on the 1871 Census (4 properties, including the "Hearts of Oak Inn" at No. 4). They appear between "4 Llangyfelach Road and Brynmelin Terrace". The entry after "Brynmelyn Terrace" is "Anne Street".
An "Oak Terrace" does not appear in Censuses 1881-1911.
An "Oak Terrace" cannot be found on any of the maps (earliest 1879) that I've checked. The National Library of Wales has an 1879 Swansea Town Plan but no mention of "Oak Terrace". The map shows "Brynmelyn Terrace" incorporated in "Llangyfelach Street". The Swansea Lost Pubs site has "Hearts of Oak Inn" in Brynmelin Street
However, checking the Welsh newspapers on line I can find dozens and dozens of newspaper articles 1880-1919 referring to "Oak Terrace, Swansea" (occasionally "Oak Terrace, Greenhill" or "Oak Terrace, Pentre"). A lot of the articles have reasonably unusual names for the residents of "Oak Terrace" which makes it possible to check them out on the Census. In a couple of instances where I was reasonably sure I'd found the person referred to in the newspaper article, and the date of Census was close the the date of newspaper article, the Census address turned out to be "Llangyfelach Street" not "Oak Terrace". There were a few more where the article was within a few years of Census and "Llangyfelach Street" again as the Census address.
I need to do a bit more checking!!