I have light skin and have never experienced any color pay off with this product. People who are experiencing this may have darker skin and/or may be using too much product. A little bit goes a long way - I only use 1-2 pumps and it will cover my entire face.
I use this alone sometimes and it has never left my skin looking white at all. I just make sure to rub it in so it really sinks in and in the end I'm just left with a luminous, shiny look to my skin (but not in an oily or greasy way)