After having trouble figuring this out I finally had an epiphany. The attached video is pretty crappy but searching the web, there's not too much good advice on this. Over on Pirate, one guy said "unbolt the wheel, take off the wheel, pull out the insert. What the problem?" Other posts use a tree or pole and a truck to pull out the insert. That would be great except when I take the wheels off my jeep it's really hard to get traction. So here's what I did.
I have the same PVC inserts in a set of pitbull rockers and all I did was flip the insert perpendicular and grab one side and roll it out as I sat on the other side. No pulleys or yanking out with truck or anything. You might try some soapy water next time to.