About 18,000 people in the city of Wajima and another 12,000 in Suzu have been told to seek shelter in Ishikawa prefecture, Honshu island.
Another 16,000 residents in the Niigata and Yamagata prefectures north of Ishikawa were also told to evacuate, the AFP news agency said.
The Japan Meteorological Agency (Jma) has issued a heavy rain emergency - the highest alert level - for parts of the region.
Japan's public broadcaster NHK quoted government officials as saying 12 rivers had breached their banks.
NHK has also aired footage showing an entire street in Wajima submerged under water.
Three rivers in Ishikawa have overflowed into nearby communities, a local official told AFP news agency.
At least one house was hit by a landslide, the Fire and Disaster Management Agency said, without giving further details.
Wajima and Suzu, in central Japan's Noto peninsula, were among the areas hardest hit by a huge New Year's Day earthquake that killed at least 236 people.
The region is still recovering after a deadly magnitude 7.5 earthquake on New Year's Day.
More than 161 people were killed by the powerful quake which toppled buildings, ripped up roads and sparked a major fire.
Japan has seen unprecedented rainfall in parts of the country in recent years, with floods and landslides sometimes causing casualties.
Jaroslav Lukiv