Starting in 20.0.0, the browser needs more applet memory to function, so we can't steal as much any more.
Thus, we now steal 14 MB on 20.0.0+ instead of 40MB.
However, since this reduces memory available for custom system modules, we are adjusting to compensate.
ams.mitm's heap size has been reduced from 32MB to 12MB (recovering 20MB).
In addition, fs.mitm now uses a new mechanism for stealing memory from the applet pool while romfs is being built.
On net, we are compromising:
* Custom sysmodules lose memory available to them.
On 19.0.0/AMS 1.8.0, there was 30 MB available for custom sysmodules.
Stealing 14 MB instead of 40 MB, we lose 26 MB of that. Reducing ams.mitm's usage will gain us back 20.
Nintendo also appears to...use 4 extra MB, in 20.0.0, from my test homebrew.
So on 20.0.0/AMS 1.9.0, there should be 20 MB available for custom sysmodules.
On the bright side, on <20.0.0/AMS 1.9.0, I guess there will be 50 MB available for custom sysmodules now?
* totk mods will lose the ability to...put every file in the romfs on sd card. There will be some unknown maximum filecount for totk mods.
On the bright side, implementing the transient memory stealing should improve compatibility for some mods which strictly add files?
* fs.mitm: skeleton the use of special allocation in romfs build
* pm: add api for ams.mitm to steal application memory
* pm/mitm: okay, that api won't work, try a different one
* romfs: revert memory usage increases; we'll handle torture games case-by-case.
* pm/romfs: first (broken?) pass at dynamic heap.
I cannot wait to figure out all the ways this is wrong.
* Release the dynamic heap a little more eagerly
* romfs: animal crossing is also not a nice game
* romfs: fix issues in close-during-build
* romfs: zelda is a blight upon this earth
This is needed for Animal Crossing 2.0.0, which has >99000 fucking files.
We now do several passes over dir/file tables instead of one pass,
doing entire hash tables before we touch dir/file tables. Thus we
no longer need to simultaneously allocate hash table and dir/file table space.
In addition, we now do repeated passes building a segment of hash tables
at a time, when insufficient memory is available. Similar is also now the
case for file/dir tables, we try 0x40000 work buffer and divide by 2
until we successfully alloc. We don't allow a work buffer <0x4000, for
write/perf reasons. If a game triggers that, let me know I guess.
Hard to imagine a worse torture-test for this code than animal crossing.
* sf: Begin experimenting with new interface declaration format
* sf: convert fs interfaces to new format
* sf: finish conversion of libstrat to new definitions
* sf: convert loader to new format
* sf: convert spl to new format
* sf: update ncm for new format
* sf: convert pm to new format
* sf: convert ro/sm to new format
* sf: update fatal for new format
* sf: support building dmnt under new scheme
* sf: update ams.mitm for new format
* sf: correct invocation def for pointer holder
* fs: correct 10.x+ user bindings for Get*SpaceSize
* ams: update to build with gcc10/c++20
* remove mno-outline-atomics
* ams: take care of most TODO C++20s
* fusee/sept: update for gcc10
* whoosh, your code now uses pre-compiled headers
* make: dependency fixes
This was tested using `https://github.com/node-dot-cpp/alloc-test` plus a few other by-hand tests.
It seems to work for the case we care about (sysmodules without thread cache-ing).
External users are advised to build with assertions on and contact SciresM if you find issues.
This is a lot of code to have gotten right in one go, and it was written mostly after midnight while sick, so there are probably un-noticed issues.
hexkyz