Williams attended Grambling State University, where he played for legendary head coach Eddie Robinson. Williams guided the Tigers to a 36-7 record as a four-year starter, and led the Tigers to three Southwestern Athletic Conference Championships. Williams was named Black College Player of the Year twice.
In 1977, Williams led the NCAA in several categories, including total yards from scrimmage (3,249), passing yards (3,286), touchdown passes (38), and yards per play (8.6). Williams finished fourth in the Heisman Trophy voting, behind Earl Campbell, Terry Miller, and Ken MacAfee. Williams graduated from Grambling with a degree in education, and began work on a graduate degree before the 1978 NFL Draft.
Despite the success that he enjoyed on the field, Tampa Bay Buccaneers offensive coordinator Joe Gibbs was the only NFL coach that visited Grambling to work Williams out and scout him. Gibbs spent two days with the 6-foot-4, 220-pound quarterback, reviewing play books, film, and going through passing drills. Impressed by his poise, work ethic, and studious nature, Gibbs wrote in his scouting report that Williams had “a big-time arm with perfect passing mechanics” and was “a natural leader… very academic and extremely prepared… football smart,” and recommended that the Buccaneers select Williams with their first round draft choice.