Up on closer examination, we have to conclude that deliberately putting objects to NaN, infinity, or very large positions is not exactly supported by the physics engine in use today (PhysX). In the current versions, it's known to cause hits missed because of the issues arising in the spatial search trees up on the presence of such objects. It's not something that can be fixed at this moment, unfortunately, and in the newer Unity versions we explicitly disallow setting infinite positions to transforms (there'll be an error and the position will be discarded). We hope that in your particular project there is a way to avoid relying on invalid positions still being somehow processed without artefacts.