Answer: wished. “Returned” establishes a completed past event, followed by his wish to visit the temple. The simple past wished expresses that subsequent action naturally. Had wished would place the wish before another past reference point rather than after the return.