Luckily this issue didn't present itself until the summer, but the heater core is normally cold at idle and heats up well when the car is at speed. I've tried to flush the heater core with air and water with minimal effect. I took a quick look at removing the heater core but quickly decided against it since you need to dismantle the entire dashboard and AC unit to get the heater core out.

I looked at the thermostat. The thermostat is a temperature switch that opens the coolant path into the engine when up to temperature. It didn't seem hard to replace so I bought it at the dealer and swapped it. I unfortunately broke one of the bolts but it seems to hold fine with a new gasket with two of the three bolts. It seems as if the new thermostat did the trick. While the heater core isn't hot at idle once the car warms up, but it is heating up at much lower speeds and quickly after starting to drive.

I believe that the old thermostat was opening at too high of a temperature, which prevented significant flow of hot coolant from the engine into the bypass hose into the heater core. I think the new one is opening up at just slightly too high of a temperature, but the heater core seems to work much better than before. Hopefully it'll work just as well in the winter.