Given that it's a silicone toy, there shouldn't be any "plasticizers" in it at all, and things can't leak out of good silicone. Period. And VixSkin is very high quality. That said, something could be on the surface of your toy.
Possibilities that come to mind are that perhaps there was a lubricant (perhaps oil-based so as not to interact with the toy) applied to the mold to make removal easier when it was made, that hasn't washed off properly. A light one might not be noticeable to the touch, but still be enough to stain. Also, consider if your lubricant of choice might not be coming off completely using your washing method of choice (I don't know what you use or how you wash it, so I couldn't say). If you're using a regular soap, things like vitamin E or aloe may be left behind on the toy when you're done cleaning it.
Either way, a heavy washing with a basic soap, or even a gentle detergent, might be in order. Don't scrub it with a sponge or anything, as the soft outer silicone could tear, but a gentle rub-down with something stronger than you usually use could help. Bleach or boiling may or may not help, depending what the problem "gunk" is made of.
Good luck clearing this up!