In Fate/Zero, he was passed over as heir to the family due to holding almost no magical potential and thus resents his more capable brother Kariya. A bitter alcoholic, he apathetically helped Zouken do his bidding, by acclimating Sakura to the worms and training her to be worthy of becoming the next head of the Matou house. Despite this, he's very determined to keep his son away from the truths of the Matou family, so he sends him to study abroad when the 4th Grail War starts.
Byakuya died 3 years prior to the events of Fate/Stay Night. He kept hiding more things from Shinji, such as the truth of Sakura's adoption. After he accidentally finds the worms basement and the truth about the Matou family, Byakuya cuts all ties with his son and dies shortly after.
