The Siri Remote is pretty tightly designed, so swapping the touchpad or front cover (with the buttons) probably won’t work. The touchpad and the main board are paired together, and they communicate through specific components. It’s not just a simple swap. They’re usually paired during manufacturing, so unless you can reprogram or re-pair them, it’s not going to function right.
You could try replacing the faulty parts individually, but swapping parts from different remotes probably won’t work without special tools or knowledge to re-pair the hardware. It’s awesome that you love tinkering, but this might be a bit trickier than it looks!