Did you miss the Oregon Electric tracks the first time around?
You can see another segment!
They run straight up the center of High - presumably all the way to just past Madison, where they would join the other Oregon Electric tracks, which go up Front Street.
Though the rubble conceals them at present, there appear to be cobbles rather than bricks here. (Bricks were at State and High.)
(During a five minute window, two car drivers seemed to have "difficulty" understanding the detour and one way grid.)