Hi Beth,
There is a security token that is passed in the url to the file service used by the file manager as a way to prevent cross site request forgery attacks that could attempt file system operations. Currently this token is a guid stored as an application variable that is regenerated in the Application_Start event.
The error can happen in a web farm scenario if the file manager page is served from one node but then the post to the file service is handled by another node becasue the applicaiton variables are not shared across nodes so the guid would not match and the operation would be denied and this message would be logged.
I just changed this in my copy to use cache instead of an application variable, but I'm afraid that won't solve it for your scenario because you are not using a distributed cache shared across nodes so in your scenario each node has a separate cache.
This security token was introduced a long time ago in version 2.3.5.3, to fix the security bug of a potential cross site request forgery, so it isn't something new.
In a fail over cluster this problem should not happen very often (only if the user is uploading right at the time when failover occurs) but if using multiple nodes concurrently and no shared cache I don't have a solution for it except to suggest using the Alternate (older) File Manager which does no use the file service and therefore doesn't rely on this token.
Best,
Joe