Why are oil-exporting nations now prioritizing economic diversification and food security?

Because they see the writing on the wall. Oil is a finite resource and can be banked on for only so long. Compare how Norway managed their oil revenue to Saudi Arabia and you will see that one recognized this principal early on.