They both compare two input voltages and drive their output high or low depending on which voltage is higher. An op-amp however also has a third mode when voltages are equal, holding the voltage level.
Comparators compare input voltages
Short explanation; if the positive input voltage is greater than the negative, the output goes high, otherwise low.
The two images above show the two states a comparator can be in, either drive the output high, or low, depending on the which of the input voltages is higher.
Op-amps can deal with equal inputs
Consider the circuit above, when the op-amp starts, its output is at 0V, and because the negative input is connected to the output, the negative input is also at 0V.
Now the op-amps does what its made for, positive input is greater than the negative, so drive the output to high. I put a graph next to the circuit to show you the output voltage.
As soon as the output voltage hits the +2V level, both inputs are in balance, and so the output voltage holds at that level. If the positive input would change, the output would again follow it. This is a voltage-follower.