Markup on used cars are huge, but here's some 1st hand info. Went to look at a used 2016 Infiniti last year at an Infiniti dealer. High end dealer in Scottsdale, AZ (I'm sure you've heard of it). Listed on the lot for $24,000, he ended up trying to sell it to me for about $20,000. Just sayin'
Also, some dealers can sell you a car that they CPO'd for less without the CPO. Typically they pay the manufacturer $600-$800 to CPO the cart, then sell it for about $1,200-$1,600 more than a similar non CPO'd car. So, they may taker an offer of $1,000 less and sell it to you without the CPO. Again, Just sayin'.
I'd going in with a starting offer of at least $2,000-$3,000 less than their asking price on a used car, telling tham that you're readuy go go home and get your checkbook (okay, cashiers check). They may say no but they won't kick you out. Here's the thing, they KNOW that if you leave the store you may find something else and not come back. I worked in the industry, they know their best chance to sell you a car is today, not next week.
As always, YMMV.