AMD does an analysis and shows the benefit of increasing your video RAM above 4GB.
Recent releases have shown marked performance increases when switching from a Radeon 5500 XT 4GB to a Radeon 5500 XT 8GB. In DOOM Eternal, the 8GB card runs the game at Ultra Nightmare settings at 75FPS (1080p), while the 4GB card can’t apply the graphics settings with that level of VRAM. Looking at titles such as Borderlands 3, Call of Duty Modern Warfare, Forza Horizon 4, Ghost Recon Breakpoint, and Wolfenstein 2: The New Colossus, there is a performance improvement on average of up to 19% across these games when using the same card and increasing the amount of VRAM from 4GB to 8GB.
For more, check out this AMD blog.