Hello and welcome.
I assume you mean GarageBand version 6 that came with iLife '11? If so, you should be able to update to version GarageBand version 10.3.5 which is compatible with High Sierra.
From memory, the headphone icon in the track name panel on the left side of the screen is the track solo button. The speaker with line through it is the track mute button. They are still the same in the latest version.
C