- display internal and sd card storage sizes.
- removed battery info.
- removed current time info.
- fix dumping save to sd card due to not opening the file with append.
- change all sizes to display GB instead of GiB.
- change progress bar units to 1000 rather than 1024.
- all info text, such as sizes and timestamps now use the info text colouring.
- shorten the ncm content type names.
more menus will use it soon.
need to have a way to show the spinner when loading, then revert to the default icon if
failed to load, or an icon doesn't exist.
otherwise, the user may think that the icon is still loading and wait for it.
- updated libhaze to 81154c1.
- increase ftpsrv stack size as it would crash when modifying custom mounts.
- fix warning for unused log data in haze.
- fix eof read for nsp/xci source by instead returning 0 for bytes read, rather than error.
- add support for lstat the root of a mount.
- handle zero size reads when reading games via devoptab.
it seems that curl does not like long blocking in the r/w callbacks.
blocking for too seems to cause a deadlock as the server stops send/recv anymore data.
to fix this, i now use curls pause api.
this api is not thread safe, so it's a little more involved than it needs to be.
however this fixes the deadlock as curls pause actually reduces the download/upload speed
to the minimum. it also reduces exit latency as now exiting is handled in the progress callback
as well, which is called far more often than r/w.
- enable sftp by default.
- add more descriptive stdio errors.
- disable devoptab timeout by default.
- handle errors for devoptab seek.
- add r/d/w progress events for threaded_file_transfer
- remove system album from file browser, only show sd card.
- do not clear game selection in games menu. useful for selecting games to backup, then delete after.
- change smb2 r/w to only send max amount, matches nfs behaviour. not sure if its needed as smb probably handles it for us.
this improves network fs speed by disabling stat for http entriely, and only enabling file stat for everything else.
this can be overriden in the config.
read still needs some work. opening a file should open a thread where data is read/written async.
this avoids the huge number of roundtrips per r/w.
eg, a read currently has to REST, open the data socket, send the file data, send ABOR to close and then close the data socket.
using the thread approach would just send the file over the already open data socket.
writes will also benefit from the above, just instead of REST it would be APPE.
seeks would need to do the above ABORT, close and re-open. however most reads in sphaira are not random access. so this isn't an issue.
fs path was changed to 255 in the past because that is the real path size that fs can handle.
however, this restriction does not apply to nfs, samba, http - and it is very easy to exceed this 255 path length.
to fix this, i have increased the path len to 1024. this allows fs to continue to work as the buffer is
big enough, but also gives network mounts enough space to work with.
Changelog:
- re-enable use in release build.
- remove ftpsrv and untitled from builtin ghdl options, as both packages are available in the appstore.
- add image viewer (png, jpg, bmp)
- add music player (bfstm, bfwav, mp3, wav, ogg)
- add idv3 tag parsing support for mp3.
- add "decyption" of GTA Vice City mp3.
- add usbdvd support for music playback and file browsing.
- add nsz export support (solid, block, ldm).
- add xcz export support (same as above).
- add nro fs proper mount support (romfs, nacp, icon).
- add program nca fs support.
- add bfsar fs support.
- re-write the usb protocol, still wip. replaces tinfoil protocol.
- all threads are now create with pre-emptive support with the proper affinity mask set.
- fix oob crash in libpulsar when a bfwav was opened that had more than 2 channels.
- bump yyjson version.
- bump usbhsfs version.
- disable nvjpg.
- add support for theme music of any supported playback type (bfstm, bfwav, mp3, wav, ogg).
- add support for setting background music.
- add async exit to blocking threads (download, nxlink, ftpsrv) to reduce exit time.
- add support for dumping to pc via usb.
- add null, deflate, zstd hash options, mainly used for benchmarking.
- add sidebar slider (currently unused).
- file_viwer can now be used with any filesystem.
- filebrowser will only ever stat file once. previously it would keep stat'ing until it succeeded.
- disabled themezer due to the api breaking and i am not willing to keep maintaining it.
- disable zlt handling in usbds as it's not needed for my api's because the size is always known.
- remove usbds enums and GetSpeed() as i pr'd it to libnx.
- added support for mounting nca's from any source, including files, memory, nsps, xcis etc.
- split the lru cache into it's own header as it's now used in multiple places (nsz, all mounted options).
- add support for fetching and decrypting es personalised tickets.
- fix es common ticket converting where i forgot to also convert the cert chain as well.
- remove the download default music option.
- improve performance of libpulsar when opening a bfsar by remove the large setvbuf option. instead, use the default 1k buffer and handle large buffers manually in sphaira by using a lru cache (todo: just write my own bfsar parser).
- during app init and exit, load times have been halved as i now load/exit async. timestamps have also been added to measure how long everything takes.
- download now async loads / exits the etag json file to improve init times.
- add custom zip io to dumper to support writing a zip to any dest (such as usb).
- dumper now returns a proper error if the transfer was cancelled by the user.
- fatfs mount now sets the timestamp for files.
- fatfs mount handles folders with the archive bit by reporting them as a file.
- ftpsrv config is async loaded to speed up load times.
- nxlink now tries attempt to connect/accept by handling blocking rather than just bailing out.
- added support for minini floats.
- thread_file_transfer now spawns 3 threads rather than 2, to have the middle thread be a optional processor (mainly used for compressing/decompressing).
- added spinner to progress box, taken from nvg demo.
- progress box disables sleep mode on init.
- add gamecard detection to game menu to detect a refresh.
- handle xci that have the key area prepended.
- change gamecard mount fs to use the xci mount code instead of native fs, that way we can see all the partitions rather than just secure.
- reformat the ghdl entries to show the timestamp first.
- support for exporting saves to pc via usb.
- zip fs now uses lru cache.
- fixed fs real path length not actually being 0x301, but instead 255. fixes#204
- file picker inherits from file browser now so there's a lot less duplicated code.
- file browser now saves the last highlighted file.
- fix bug in file browser where the new file path could be empty (ie not containing a /).
- added support for viewing qlaunch romfs.
- moved fs mount options to the top of the list (may revert).
from my notes:
this converts the passphrase array into an ascii string
well, turns out std::transform does not actually increase the std::string size when pushing the new ascii characters
basically, std::string (normally) has small string optimisation. For this, they have a small array which it uses instead of allocating memory.
std::transform casts the std::string itr to a void*. sooo it was basically manually writing to the class, which just so happened to be the SSO buffer. however after 32chars, it runs out...and starts overwriting the stack. so this was a stack overflow lmao
only reason i found this is because i changed some optimisation flags, so the compiler ended up inlining std::transform, which detected the buffer overflow. and then i realised how fucked it was lol.
really, these functions should be passed by && and using std::forward on assignment.
however, this means writing a lot of extra code for every single class, as well as explicitly calling move
in a lot of cases.
In the case of std::string, passing by value and calling std::move is the correct approach, especially if the
string is created as an rvalue, then it is only created once and moved into the dest.
Whereas with a const ref, the string is created and then copied into the dst, basically creating 2 copies.
The same thing happens std::function, and well any object.
However, accepting everything by value sucks if you call a constructor from within a constructor, as now you need to create 2 impls
that accept by value and the other by rvalue. All of this extra code to have a more efficent impl just isn't worth it when, going by
the benchmarks, makes no measurable difference (i count anything within >= 1ms as measurable).
the default options where to always verify the nca content, this was for brick protection.
however, users seem to not care about this protection and get frustrated with the default values,
seemingly unable to find the install menu to disable said options.
this change skips verify by default. less issues for me, less unhappy users, win win.
- nca::ParseControl now gets the nacp base on the language.
- xci returns a more descriptive error and logs info during install.
- replace volatile with atmoic, label atmoic methods with volatile as per the standard.
- s2s usb install skips verifying the content as its being installed from another switch.
- fix game / save menu bug where it would only load the nacp if it has started yet, now it will force load even if its in progress.
this fixes save file zips having empty names.
- game menu saves the application type, to be later used for displaying if its a gamecard (inserted or not), launched etc.
i am unsure how this happens, as i thought the profile uid was the same as the account uid, but apparently this can differ.
on the same switch in sysmmc, the uid's match, so everything works.
again on the same switch, created in emummc, same account, the uid differs...
i performed the same test but on another 2 switches, and the uid's all match, so i am not sure what causes them to change.
in any case, using the uid from the account is the intended behaviour anyway, so this commit fixes that.
This reverts commit 8e67e5f0fc.
While MTP install may not work for most people, i guess it's better to have it as an option still.
Who knows, someone may figure out why it randomly freezes on windows when installing heavily compressed nsz files.
- dumped nsp now have the tik/cert at the end of the file table, rather than the beginning.
- dumped nsp patches the ticket if needed (no personalised dumping yet).
- installing titles will now patch the ticket, performing personalised -> common convert if needed, as well as fixing bad common tickets.
- yati no longer tries to install ncas if they already exist.
- ticket only option now actually works.
- fixed some translations.
- removed unused error codes.
i've been trying to track down this bug for a while. i still don't understand why it happens, however i have managed to
reproduce it an narrow down the crash, and thus fix it.
the bug was caused calling nvgDeleteImage() inside ~LazyImage() on image 43. this would only trigger once games/saves menu
had been opened at least once also.
i can only assume that the image fd was still refrenced in deko3d when drawing, as it would only ever crash on the visible images.
destroying fb resources before the calls to nvg delete seems to fix the issue.
maybe the explicit call to waitIdle is what fixes it? or clearing the cmd buf? who knows...
* Add a new ThemeEntryID for split-screen and selected items and modify the theme.
* Adjust the position of the left side split-screen in the filebrowser menu.
* Add new strings and update Korean and Japanese translations.
* fix ja.json.
---------
Co-authored-by: ITotalJustice <47043333+ITotalJustice@users.noreply.github.com>
the issue with block installs was that i was not tracking the ncz block offset in between transfers.
this resulted in the block size being used for each transfer, rather then size-offset.
for blocks that were always compressed, this silently worked as zstd stream can handle multiple frames.
however, if there existed compressed and uncompressed blocks, then this bug would be exposed.
thanks to Marulv for reporting the bug.
the fix for file based emummc is to simply sleep between fs r/w to the sd card.
the performance impact is minimal, even with the reduced buffer size.
the above *only* applies for when using file based emummc. not affecting using partition or sysmmc.
- fix usb using the wrong year when polling the timestamp.
- fs file/dir has been re-written to allow for simplified calling and remove the need of manually closing.
- add SetSize for stdio by using ftruncate.
- don't truncate the file when opened in stdio.
- add getcount api for stdio.
- display file/dir count in filebrowser for non-native fs.
- allow hash to be used on non-native fs.
- slightly optimise nro parsing by manually calculating nro size rather than doing an os call.
- slightly optimise nro parsing by keeping the fs struct alive between calls, rather than creating a new one on the stack.
- fix filebrowser peeking into zip files that are stored on non-sd fs.
- set the timestamp of a file moved to a stdio location (cut/paste).
- slightly optimise daybreak update folder detection by skipping opening/polling the dir size.
- set the fullpath of the file thats being hashed in progress box.
- build with c++26 and c23, fixes warnings due to this change.
- use #embed over romfs where applicable.
- load all configs upfront in the app menu, massively improves boot time
- enable boost mode during load/scan time in all (slow loading) menus, huge load time improvement.
- enable boost mode when exiting the app, to speed up closing all the menus and saving the config.
- reduce the size of the nro nacp when loading to just the title + author + display version.
- add option to enable boost mode for all progress bar menus, huge perf improvement again.
- remove unused launch_count var from the playlog file.
- display full path when dumping.
- optimise appstore unzip code by iterating through the zip rather than finding a specific file, reduces retroarch extract time from 52s to 26s.
overall, this commit has reduced boot time from 0.4s to 0.3s and massively increased load times of other menus.
(it also reduced the binary size by 4kb, so yay)
due to the previous commit, i broke dumping multiple games via usb as the stream offset wasn't reset.
because of this, the first transfer would complete, but the 2nd one would fail.
Some notes i made when adding stream support:
The tinfoil API makes it hard / impossible to multi thread the file upload because each data transfer passes the the file name and offset, meaning that it can and will change files and offset in the middle of a transfer
So that prevents the read thread from running freely in the background and the pull thread pulling data when requested.
The extension adds a flag to the usb header which if set, enables stream mode (same as ftp installs). This removes random access, but allows for multi threading as the data will be requested in order.
previous uploads were all single threaded, which meant that it only uploaded as fast as the slowest source.
usb transfer is still single threaded due it being random access for both files and data, making it
hard for the read thread to run freely.
turns out that the event is auto cleared when waited.
this meant that if sphaira handled the event before qlaunch got chance to handle it,
then qlaunch won't update when app records changed.
this can result in a gamecard not being mounted, deleted games still apearing, installed games
not being displayed etc...
- added support for custom upload locations, set in /config/sphaira/locations.ini
- add support for various auth options for download/upload (port, pub/priv key, user/pass, bearer).
- added more es commands.
- fixed usb install potential hang if the exit command is sent, but the client stops responding (timeout is now 3s).
- added multi select to the games menu.
- added game dumping.
- added switch2switch support by having a switch act as a usb client to transfer games.
- replace std::find with std::ranges (in a few places).
- fix rounding of icon in progress box being too round.
- fix file copy helper in progress box not updating the progress bar.
on fw 19 and below, loading from cache takes ~5ms, whereas manually loading takes ~20ms.
manually loading is still faster than relying on ns to load control from storage (~50ms).
- loading the control data is ran on its own thread, it does not block the main thread. allows for smooth scrolling like nintendos home menu.
- on fw20+, sphaira manually parses the control data, rather than using ns. manually parsing takes 20-40ms, which is faster than ms which can take 50-500ms.
- on fw19 and below, if the control data is not in ns cache, sphaira will manually parse the data as its twice as fast as ns. You can see how fast this is by loading the gamecard menu as that manually parses everything, and it loads the gamecard faster than the home menu
allows the user to enable installs for one config and disable it for the other.
by default, it will load the install option found in the config, if found.
otherwise, it will load from the new config option.
getting the list of title_ids is very fast (less than 1ms), however parsing the control info, such as title names
is very slow.
depending on how many games the user has, blocking until we read all control info can take several seconds...
we would only need to block if the user wants to sort by name.
normally, we lazy load the control data, so we don't suffer from slow load times at all.
i decided that its not worth slowing the whole system down just to give the option to sort by name.
some other changes:
- shorten the next page and prev page to just next/prev in themezer.
- remove misc shortcut name. the function itself still exists.
this crash was found by @WE1ZARD.
to trigger it, press L2 in the filebrowser whilst a hidden file exists and the hide hidden is enabled.
the was due to GetEntry(i) internally using m_entries_current, and the select all was using the index from m_entries_current.
this would result in an index that goes oob, and as its a write, it crashes.
Commit changes to language file (de.json), with many improvements and corrections.
The changes include:
- Modification and correction of existing translations for greater clarity and accuracy.
- Updated translations to match the terminology for homebrew and Nintendo Switch UI elements.
- Added new 'de' translation for the “ 12h time format” setting function.
this is already handled by the gpu, but cpu side still has to do some work.
this wasn't a performance issue (we only use 4%) but its a free win, so we might as well.
curl/libcurl does not send Accept-Encoding by default.
many servers support sending compressed versions of files, to speed up transfers.
this is ideal for the switch as its io is shit, but the cpu is mostly idle (4% cpu usage for sphaira).
github and appstore support sending gzip json files, themezer doesn't seem to.
the web browser on the switch is really bad, it shouldnt be used.
i am removing this menu because its another option that gets in the way of other options, and code bloat.
i added the irs menu when i wanted to mess around with the sensor on the joycon.
since then, i have used it a total of 0 times, and i don't think any users use it either.
- replace the python script with the one included with tinfoil, minor changes such as changing the supported extension,
removing unused imports.
- tested with the included script, fluffy and ns-usbloader on linux.
a user was unable to get it working on mac however...
- added build instructions to the readme, i think they're correct.
- added install instructions to the readme.
the previous commit changed usb transfers to always transfer to/from page aligned buffers.
i wanted to keep the commits seperate so that its easier revert or git bisect later on, if needed.
- gamecards now wait for an event to change, rather than polling each frame.
this reduces cpu load on core 3 slightly (3-4% less).
- my understanding of fsOpenFileSystemWithId() was wrong, i thought it used the app_id for the id param.
turns out it needs the program id (found in the nca header), this is why mounting some control ncas
would fail.
fs (and ncm) have a call in 17+ to get the program id, it does so by parsing the nca header.
in yati, we already have the header so we can avoid the call.
for the gamecard menu, we don't. so we can parse the nca header, or use the id offset (which we already have)
to form the program id.
- std::find_if in yati now takes args by ref rather than by value, avoid quite large copies.
- stream installs can now parse the control nca.
- if an nca is already installed, it is now skipped. this is regardless to whether it is not in ncm db.
- nca skipping is technically supported for stream installs, however it is disabled for now as there needs
to be a way to allow for the stream to continue reading and discarding data until the stream has finished.
currently, if a ftp (stream) install is skipped, it will close the progress box and cause spahira to hang.
this is because sphaira expects the stream to only be closed upon all data being read, so there's nothing more
to process.
- renamed the title_id field in nca header to program_id.
- fix ignore distribution bit doing nothing.
- fix yati failing to parse control nca causing the transfer to abort.
- yati now uses ncm rather than ns to get the latest app version.
- improve ui::list input handling (it handles directional buttons now).
- progress bar displays speed and time remaining.
- added gc menu (taken from my gc installer nx and gci).
@@ -40,7 +40,6 @@ MTP can be enabled via the Network menu.
Sphaira has file association support. Let's say your app supports loading .png files, then you could write an association file, then when using the file browser, clicking on a .png file will launch your app along with the .png file as argv[1]. This was primarly added for rom loading support for emulators / frontends such as RetroArch, MelonDS, mGBA etc.
```ini
[config]
path=/switch/your_app.nro
supported_extensions=jpg|png|mp4|mp3
```
@@ -49,17 +48,60 @@ The `path` field is optional. If left out, it will use the name of the ini to fi
See `assets/romfs/assoc/` for more examples of file assoc entries.
## Installing (applications)
Sphaira can install applications (nsp, xci, nsz, xcz) from various sources (sd card, gamecard, ftp, usb).
For informantion about the install options, [see the wiki](https://github.com/ITotalJustice/sphaira/wiki/Install).
### Usb (install)
The USB protocol is the same as tinfoil, so tools such as [ns-usbloader](https://github.com/developersu/ns-usbloader) and [fluffy](https://github.com/fourminute/Fluffy) should work with sphaira. You may also use the provided python script found [here](tools/usb_install_pc.py).
### Ftp (install)
Once you have connected your ftp client to your switch, you can upload files to install into the `install` folder.
## Building from source
You will first need to install [devkitPro](https://devkitpro.org/wiki/Getting_Started).
database=Nintendo - Nintendo DS|Nintendo - Nintendo DS (Download Play)|Nintendo - Nintendo DSi|Nintendo - Nintendo DSi Decrypted|Nintendo - Nintendo DSi (Digital)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
