Alvin and Jane were buried in Welford Rd (the main civic cemetery on the south side of the town).
Alvin was living at Milton St,St Margaret, at the time of his death; his age was given as 39. He was buried in a grave with 4 other people, none of whom shared names, so I suspect this wasnot a family plot; these are the plot details and "inhabitants"
Plot uS977
MASON MARK buried 1874 SEP 5 age 37 abode SOAR LANE ALL SAINTS
ELLMORE ALVIN 1890 APR 2 age 39 MILTON STREET SAINT MARGARET
MARSH JOHN WILLIAM 1898 JUL 27 age 9MTHS DORSET STREET LEICESTER
CORBELL OLIVE 1919 MAR 8 age 16 ROYAL INFIRMARY LEICESTER
HUGHES JOHN HENRY 1938 MAR 28 age 74 R.I. / 68 WHEAT STREET LEICESTER [ROYAL INFIRMARY]
Jane was buried in a different plot, in 1885, age given as 30.
Plot uR 2066
ELMORE JANE buried 1885 NOV 5 age 30 abode LEWIN STREET SAINT MARGARET
NEEDHAM EDWIN JOHN 1895 JUL 18 9MTHS JUNCTION ROAD SAINT MARGARET
SIMPKIN THOMAS 1919 DEC 3 age 68 146 HARROW ROAD LEICESTER
SIMPKIN EMMA 1933 NOV 18 age 77 146 HARROW ROAD LEICESTER
With this information you have a better chance of finding them in censuses, but don'tforget to look at the various possible spellings of the surname.
I think it's unlikely that either of these graves will actually have a gravestone.
