I struggled with the removing the mats this past weekend. The "Buttons" in the center of the grommets are not really Buttons. That center knob visible when the mats are in place is just the end of the fastener attached to the carpets and is held in place by friction. There is no spring-loaded action that you equate to a button.
To pop the fastener out of the grommet, push down on the "button" with the thumb of one hand and put the fingers of your other hand in between the mat and the fastener that is attached to the carpet. As you push down on the button with the thumb, place your fingers of the same hand under the mat so that you can pull it up at the same time. Place your other hand under the mat, hold the fastener in place and pinch your fingers together to cause the mat to be pushed up. The fastener is not attached very securely to the carpet/floor so be careful not to rip it out.