EDIT: Anyone know how to avoid those cmyk pixels showing up in the colour part? The source pic I used didn't have them.
are you sure its not something you did in lab colour then converted over / saved in rgb or cmyk colour ? LAB = has colours the other 2 do not have, which ='s fucked up pixels.