WELCOME TO THE FORUMS!!! woohoo!!
So you need to remove the whole door panelt o get the plug on? Let's see.. well first you pull off that little triangle piece on the inside right where the mirror is (you must have done that already to swap the mirror on). Then there's I believe 2 screws on the grab handle thing on the armrest. take those out. There's another screw hidden in the door handle (the lever you use to release the door). After that, along the edge of the door panel I think there might be some plastic clips, the kind where you have to use a flathead to pop the center circle piece out to release the clip.
The motion to remove the door panel is kind of an up and out. The bottom half of the panel will just pull off the door (lots of little clips will pop if I recall correctly) but the upper part is resting on the window edge so you have to lift up to get it over the ledge there and the thing for the door lock. Once you get it off there's a plug or two to disconnect the electronics from the door panel and then you can get it out of your way.
I could get into more detail but I'm at work right now so I can't go examine my car
Edit: there's a bit of a trick for getting the door lever off.. once you get the panel moving a bit you can kinda rotate that handle so that it can fit through the hole in the door.. the handle stays with the door when you take the panel off.
And since I forgot to mention it, I think the grab handle on the armrest has to be pulled off before you can remove the door too.. I may be thinking of the panels on the rear doors though with that one.
good luck!