Shenmue II begins shortly after the first installment concluded. While Shenmue told the story of the 1st chapter of the saga, the second game tells the story of the third, fourth, and fifth chapters. The second chapter happens between Shenmue and Shenmue II during Ryo Hazuki's trip from Yokosuka to Hong Kong and is briefly mentioned during the beginning of the game.
The third chapter of the saga begins when Ryo, the main protagonist of the series, arrives in Hong Kong in order to locate Master Lishao Tao as he was instructed to do by his friend and confidant, Master Chen Yao Wen, at the end of the series' first chapter. This mysterious and elusive individual is Ryo's only possible link to Lan Di, the Chinese man who murdered his father. After a difficult search, Ryo finally meets Master Lishao Tao, a woman named Xiuying; but she is unwilling to assist him in what she considers a futile quest for vengeance. The two part ways, although Xiuying continues to monitor Ryo's progress and they continue to meet on occasion. Through his continued search, Ryo discovers another individual who may be able to assist him in locating Lan Di.
Wuying Ren, commonly referred to as Ren throughout, is the leader of a gang named The Heavens, who engage in a variety of illegal activities. After initially attempting to wile Ryo, Ren decides to assist him in his quest after discovering that there are large sums of money tied up in the mysterious and ancient Phoenix Mirror. A young boy who holds Ren in high regard (considering him like a brother, although they are not related by blood) named Wong, and an adventurous and free-spirited woman named Joy also befriend and assist Ryo in his continued search for Lan Di.
The fourth chapter of the saga takes place in Kowloon, as Ryo attempts to locate Yuanda Zhu; a martial arts expert who had sent Iwao Hazuki a letter warning of his impending murder. At this juncture, several confrontations ensue between Ryo and his allies and the dangerous Yellowheads organization, who are aiming to kidnap Yuanda Zhu on behalf of Lan Di. After following several clues, Ryo and Ren finally find Yuanda Zhu; but the meeting is cut short when they are ambushed by the Yellowheads leader, Dou Niu, and Yuanda Zhu is kidnapped in the resulting encounter. Wong and Joy are also held captive, although Ryo later ensures their release through his victory against a powerful martial artist. In the climatic scene that follows, Ryo and his allies enter the building in which Yuanda Zhu is being held and after Ryo does battle with Dou Niu for the final time, they are able to prevent Lan Di from receiving a captured Yuanda Zhu as originally planned. Elusive as ever, Lan Di escapes once again. Yuanda Zhu provides Ryo with crucial information regarding the true purpose of the dragon and phoenix mirrors. It is revealed that the mirrors will lead to the resurrection of an ancient order, known as the Qing Dynasty. Ryo is advised to continue his search in Bailu Village, in remote Guilin. He parts ways with Ren, Wong and Joy; continuing his journey and heading for the same destination as his rival, Lan Di.
The fifth chapter takes place in Guilin. Shortly after arriving, Ryo encounters a young woman named Ling Shenhua. She had previously appeared to Ryo through several dreams throughout the first chapter of the series. As the two converse, it is revealed that the Shenhua family is connected with the legacy of the dragon and phoenix mirrors. Shenhua leads Ryo to a stone quarry on the outskirts of the village to meet with her father, but he is nowhere to be found. The episode comes to an ambiguous end when the pair discover a cryptic note and sword, which Ryo combines with the phoenix mirror and unwittingly sets off a device revealing a huge depiction of the two mirrors. At the game's cliffhanger ending, the sword is seen to float in mid-air.