The road past the 2WD trail head is not that bad as long as you have an SUV/truck with decent clearance. There are some rocky sections, but did not prove too difficult. Crossing castle creek was a cake walk, even in the afternoon with higher levels due to snow melt. It was maybe 6 inches deep. There was a Toyota Highlander we saw further up the trail, so any vehicle with comparable clearance can make it. We ended up staying the night at the Pearl Pass junction, at 11,200, allowing us to get a great head start on the trail up to Castle.
Photos (click for slideshow):