Solved! Go to Solution.
Aaaggghhh!!! That's the level where my gyroscope seems to not work! I had huge a feeling it was that level. Thanks a million for the info, at least I know to focus my efforts on that level. The few other mini games can wait.
I don't know if this has anything to do with it, but I think it helps to keep your Vita horizontal and level when starting up the level, then the controls seemed to be how they were supposed to be. A couple of times I went into it and it had really messed up control. Finally got it though, sheesh...