1190 Sportrak n/s outer door handle


I need to change a broken outer passenger door handle, the mechanism is ok, and I now have a replacement handle, do i need to remove the door trim, and bolt it in place, or can i just swop the pull handle on its own Unknw
Any advice please ! Thanks