It says that “death will be no more.” In order for something to be no more, it had to exist in the first place. Since there has never been death in heaven and people die only here on earth, logically, then, Revelation 21:4 must be referring to future blessings here on earth.—Ps. 37:29.
The prophets all spoke about the coming Paradise on earth. They did not even entertain the idea of going to heaven. This hope was opened up by Jesus himself. He started choosing member of his "little flock" of anointed ones after his baptism in 29 C.E. THESE ones would have the privilege of ruling in his Kingdom government over the Earthly paradise.