Save the project and restart the computer to purge most of the stuff AE stashes here and there.
You're going to start a binary test to attempt to isolate the cause.
Open a new 2kx2k comp in AE and put your photos in it, one at a time. Scale the picture to fit. Export a still image. Import the file back into your project.
Make a copy of your problem comp and replace your photos with the new, smaller versions. What happens now?
Turn off all effects, one at a time, and evaluate the situation with each change.
Turn off each layer, one at a time, and evaluate the situation.
Blurs on huge files like 4k images take a long time to process. You can get around that by applying the blur to the image in a pre comp and exporting it as a still or as a movie if the effects change over time.