Hi Joe,
I found the culprit. The "Quota" settings (listed below) in the web.config were blocking any further uploads when using the FileDialog.aspx page. The File Manager isn't using these "Quota" settings yet so that explains why uploads were working fine there.
I know the settings were in the web.config in 2.3.6.6 but I don't think they were used by the FileDialog.aspx page. Can you confirm?
At any rate, this client has nearly 30GB of data and over 60k files in their data\sites\1 directory so I upped the quota values and everything is fine again.
I managed to find this because I tried to create a new folder using the FileDialog and I received an error stating "Your permissions don't allow you to create any more folders" which prompted me to look for that string in the Resource.resx and then find it's resource name in the source.
Now for a feature request: can you add a similar message for any other scenarios when a quota is met?
Thanks,
Joe D.