This Bible contradiction is from the Skeptic's Annotated Bible.
Yes, he took them to Gilgal.
Joshua 4:20 View context
And those twelve stones, which they took out of Jordan, did Joshua pitch in Gilgal.
No, they are still there.
Joshua 4:9 View context
And Joshua set up twelve stones in the midst of Jordan, in the place where the feet of the priests which bare the ark of the covenant stood: and they are there unto this day.