Charles and William decided never to meet Harry alone again after interview with Oprah interview