I'm taking this at face value -- Hedo started 44 games that year for the Spurs and started every game that he played in after January 31, 2004. Manu only started 38 games that year and didn't start after January 31, 2004. That switch took place, reportedly, to maximize the Spurs roster.
Here, for example, is a column Kori wrote in February of 2004 that detailed that situation a bit. I'm sure I could find others if you needed convincing. It had nothing to do with a lack of confidence in Manu -- it had everything to do with trying to get something useful out of Turkoglu.