dmnt2: fix gdb register output

This also fixes ctrl-c break in gdbstub, and fixes crash on unknown monitor cmd.

.gitignore

@@ -91,7 +91,11 @@ dkms.conf
 **/out
 **/build
 **/build_nintendo_nx_arm64
+**/build_nintendo_nx_arm64_armv8a
 **/build_nintendo_nx_arm
+**/build_nintendo_nx_arm_armv8a
+**/build_nintendo_nx_arm_armv7a
+**/build_nintendo_nx_arm_armv4t
 **/build_nintendo_nx_x64
 **/build_nintendo_nx_x86
 

Makefile

@@ -112,6 +112,7 @@ dist-no-debug: all
 	mkdir -p atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/0100000000000042
 	mkdir -p atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/0100000000000420
 	mkdir -p atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000B240
+	mkdir -p atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000D609
 	mkdir -p atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000D623
 	cp stratosphere/boot2/boot2.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/0100000000000008/exefs.nsp
 	cp stratosphere/dmnt/dmnt.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000000D/exefs.nsp
@@ -125,6 +126,7 @@ dist-no-debug: all
 	cp stratosphere/pgl/pgl.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/0100000000000042/exefs.nsp
 	cp stratosphere/LogManager/LogManager.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/0100000000000420/exefs.nsp
 	cp stratosphere/htc/htc.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000B240/exefs.nsp
+	cp stratosphere/dmnt.gen2/dmnt.gen2.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000D609/exefs.nsp
 	cp stratosphere/TioServer/TioServer.nsp atmosphere-$(AMSVER)/stratosphere_romfs/atmosphere/contents/010000000000D623/exefs.nsp
 	@build_romfs atmosphere-$(AMSVER)/stratosphere_romfs atmosphere-$(AMSVER)/atmosphere/stratosphere.romfs
 	rm -r atmosphere-$(AMSVER)/stratosphere_romfs

docs/changelog.md

@@ -1,4 +1,59 @@
 # Changelog
+## 1.2.2
++ A number of fixes were made to Atmosphère's implementation of the new "sprofile" service added in 13.0.0.
+  + Nintendo is finally transmitting data over the internet to certain consoles, which has allowed for validating our service implementation.
+    + Unfortunately, there were several problems, and if your console began trying to use the new services atmosphere would show a fatal error with code 0xCAF6 (sprofile::ResultInvalidState()).
+  + With actual test data in hand, a test program was written and it was verified that our implementation can successfully import/access profile data now.
+    + Hopefully there are no more issues, and I sincerely apologize for anyone who got an 0xCAF6 fatal due to this.
++ A number of minor improvements were made to `mesosphère`, including:
+  + KThread::GetContextForSchedulerLoop was implemented in assembly (using static assertions to verify offset-of-context-in-struct is correct).
+    + This saves an unnecessary function call in the middle of the scheduler hot loop, replacing it with an addition instruction, which should improve microperformance.
+  + Mesosphere's hardware maintenance instructions were audited via a script and now directly match Nintendo's kernels.
+    + Notably, this inserts a missing instruction synchronization barrier when validating that slab heaps may be constructed.
+    + This missing ISB could cause an abort on certain (see: particularly sensitive) hardware on boot if the relevant codepath was speculatively executed (it normally only executes on game launch...)
+  + The SVC handlers for performing light IPC (normally unused) from 32-bit process were fixed in Mesosphere.
+  + A bug was fixed that would cause the register x27 to be overwritten with the contents of x26 when returning from a user exception handler.
+  + A bug was fixed that would cause the kernel to use the userland stack pointer instead of the kernel stack pointer while generating an error report for a kernel abort.
++ General system stability improvements to enhance the user's experience.
+## 1.2.1
++ Support was implemented for 13.1.0.
+  + `mesosphère` was updated to reflect the kernel behavioral changes made in 13.1.0.
+    + KScheduler now issues a data memory barrier when unlocking the scheduler lock and when early-returning due to top-thread-is-current during scheduling.
+  + `erpt` was  updated to reflect the latest official behaviors.
+    + The new service added in 13.0.0 ("sprofile") was revised, and the data formats it expects was changed.
+      + This still appears to be (possibly(?)) untestable due to data not being transmitted yet, but I have greater confidence things will go smoothly than I did when 1.1.0 released.
++ A number of improvements were made to `mesosphère`, including:
+  + A build target was created to build targeting the qemu `virt` board.
+    + This facilitates writing unit tests for the kernel (and other atmosphere components) and running them under PC.
+      + **Please Note**: Official system software will not work at all under this, and the Atmosphère project has zero interest in attempting to run official software of any kind. This is unit testing machinery, and explicitly not more than that.
+    + This should hopefully allow us to have greater confidence that all of atmosphere's components work the way they're theoretically supposed to in the future.
+    + **Please Note**: If you are a developer who is familiar with the Horizon operating system (or capable of becoming familiar), I would greatly appreciate help writing tests and improving the testing framework.
+      + Please contact `SciresM#0524` if you are capable and interested.
+        + Really, if you are actually a developer who would like to help me get this off the ground, I would deeply appreciate it.
+        + That said, if you are not a developer but want to be one, this probably isn't the best opportunity; I expect it to be highly technical.
+          + Consider the ReSwitched discord's #hack-n-all channel for your educational purposes.
+      + We are (at least for now) using [catch2](https://github.com/catchorg/Catch2) for unit tests.
+  + Almost all virtual calls in the kernel are now resolved statically.
+    + This eliminates substantial virtual call overhead, and should lead to improved kernel microperformance in pretty much every function.
+  + The remaining red black tree find operations which weren't using the optimized "find key" variant are now using the optimized version.
+  + Custom assembly was written to improve tick-to-timespan conversion.
+    + This works around gcc emitting suboptimal assembly at -Os (it emits good assembly at -O3, clang is fine at both -O3 and -Os).
+  + KThread and KSession structures were updated to optimize member layout, saving 0x10 bytes per KThread/KSession object.
+  + Rather than unnecessarily zero-ing all data in kernel objects only to overwrite members later, we now only initialize the members we need to in kernel object constructors.
+    + This is what Nintendo was doing already.
+  + A set of custom optimized atomic primitives were implemented and are used in place of std::atomic<>
+    + This works around a gcc bug which downgrades specified memory order to seq_cst, and introduces clrex in places where it is appropriate.
+    + This should strictly improve microperformance of many system calls.
+  + An compile-time toggleable extension was added to support 40-bit physical addresses in MapRange capabilities (using currently reserved bits).
+  + A number of minor bugs were fixed, including:
+    + Initial cache management now better reflects official behavior.
+      + This fixes an issue that caused certain hardware with cache sensitivity to produce cryptic kernel panics during boot.
+    + Incorrect logic when checking thread priority capabilities was fixed to reflect official behavior.
+    + The scheduler was updated to reflect latest official behavior, and a number of minor bugs involving clz/ctz were fixed.
+    + Accesses to the processes local region were fixed to properly use kernel linear region, not userland pointers.
+    + The cache SVCs exposed for 32-bit processes now better reflect official core mask request semantics.
+    + A bug was fixed that could cause a kernel panic if SvcArbitrateLock was called on a thread with exactly one reference in the middle of handling a user-mode exception.
++ General system stability improvements to enhance the user's experience.
 ## 1.2.0
 + `boot` was updated to reflect the latest official behavior for display/battery management.
   + This should fix any issues that might result from running older releases on the OLED model, if you're somehow in a position to do so.

emummc/.gitrepo

@@ -6,7 +6,7 @@
 [subrepo]
 	remote = https://github.com/m4xw/emuMMC
 	branch = develop
-	commit = f66087313546161a000ee196a788f0626caf80fa
-	parent = 38f9a76ba028995ed3274da3a45b0254f09d1f59
-	method = rebase
+	commit = a9d56959460fc794ce2cb6405402c25a3e89c47f
+	parent = ff719641396c635b735873fb2b020c910f768a04
+	method = merge
 	cmdver = 0.4.1

emummc/README.md

@@ -1,21 +1,21 @@
 # emuMMC
-*A SDMMC driver replacement for Nintendo's Filesystem Services, by **m4xw***
+*A SDMMC driver replacement for Nintendo's Filesystem Services, by **m4xw***  
 
 ### Supported Horizon Versions
-**1.0.0 - 13.0.0**
+**1.0.0 - 13.1.0**
 
 ## Features
-* Arbitrary SDMMC backend selection
+* Arbitrary SDMMC backend selection  
   **This allows loading eMMC from SD or even SD from eMMC**
-* On the fly hooking / patching, fully self-infesting
+* On the fly hooking / patching, fully self-infesting  
   **Only one payload required for all versions!**
-* File-based SDMMC backend support (from SD)
+* File-based SDMMC backend support (from SD)  
   **This allows loading eMMC images from hekate-backups (split or not)**
-* SDMMC device based sector offset (*currently eMMC only*)
+* SDMMC device based sector offset (*currently eMMC only*)  
   **Raw partition support for eMMC from SD with less performance overhead**
-* Full support for `/Nintendo` folder redirection to a arbitrary path
+* Full support for `/Nintendo` folder redirection to a arbitrary path  
   **No 8 char length restriction!**
-* exosphere based context configuration
+* exosphere based context configuration  
   **This includes full support for multiple emuMMC images**
 
 ## Compiling

emummc/source/FS/FS_offsets.c

@@ -57,6 +57,8 @@
 #include "offsets/1203_exfat.h"
 #include "offsets/1300.h"
 #include "offsets/1300_exfat.h"
+#include "offsets/1310.h"
+#include "offsets/1310_exfat.h"
 #include "../utils/fatal.h"
 
 #define GET_OFFSET_STRUCT_NAME(vers) g_offsets##vers
@@ -125,6 +127,8 @@ DEFINE_OFFSET_STRUCT(_1203);
 DEFINE_OFFSET_STRUCT(_1203_EXFAT);
 DEFINE_OFFSET_STRUCT(_1300);
 DEFINE_OFFSET_STRUCT(_1300_EXFAT);
+DEFINE_OFFSET_STRUCT(_1310);
+DEFINE_OFFSET_STRUCT(_1310_EXFAT);
 
 const fs_offsets_t *get_fs_offsets(enum FS_VER version) {
     switch (version) {
@@ -210,6 +214,10 @@ const fs_offsets_t *get_fs_offsets(enum FS_VER version) {
             return &(GET_OFFSET_STRUCT_NAME(_1300));
         case FS_VER_13_0_0_EXFAT:
             return &(GET_OFFSET_STRUCT_NAME(_1300_EXFAT));
+        case FS_VER_13_1_0:
+            return &(GET_OFFSET_STRUCT_NAME(_1310));
+        case FS_VER_13_1_0_EXFAT:
+            return &(GET_OFFSET_STRUCT_NAME(_1310_EXFAT));
         default:
             fatal_abort(Fatal_UnknownVersion);
     }

emummc/source/FS/FS_versions.h

@@ -83,6 +83,9 @@ enum FS_VER
     FS_VER_13_0_0,
     FS_VER_13_0_0_EXFAT,
 
+    FS_VER_13_1_0,
+    FS_VER_13_1_0_EXFAT,
+
     FS_VER_MAX,
 };
 

emummc/source/FS/offsets/1310.h

@@ -0,0 +1,59 @@
+/*
+ * Copyright (c) 2019 m4xw <m4x@m4xw.net>
+ * Copyright (c) 2019 Atmosphere-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __FS_1310_H__
+#define __FS_1310_H__
+
+// Accessor vtable getters
+#define FS_OFFSET_1310_SDMMC_ACCESSOR_GC   0x158C20
+#define FS_OFFSET_1310_SDMMC_ACCESSOR_SD   0x15AA30
+#define FS_OFFSET_1310_SDMMC_ACCESSOR_NAND 0x159150
+
+// Hooks
+#define FS_OFFSET_1310_SDMMC_WRAPPER_READ  0x1545C0
+#define FS_OFFSET_1310_SDMMC_WRAPPER_WRITE 0x154680
+#define FS_OFFSET_1310_RTLD                0x688
+#define FS_OFFSET_1310_RTLD_DESTINATION    ((uintptr_t)(INT64_C(-0x3C)))
+
+#define FS_OFFSET_1310_CLKRST_SET_MIN_V_CLK_RATE 0x1537C0
+
+// Misc funcs
+#define FS_OFFSET_1310_LOCK_MUTEX          0x29690
+#define FS_OFFSET_1310_UNLOCK_MUTEX        0x296E0
+
+#define FS_OFFSET_1310_SDMMC_WRAPPER_CONTROLLER_OPEN  0x1544A0
+#define FS_OFFSET_1310_SDMMC_WRAPPER_CONTROLLER_CLOSE 0x154530
+
+// Misc Data
+#define FS_OFFSET_1310_SD_MUTEX            0xE133E8
+#define FS_OFFSET_1310_NAND_MUTEX          0xE0E768
+#define FS_OFFSET_1310_ACTIVE_PARTITION    0xE0E7A8
+#define FS_OFFSET_1310_SDMMC_DAS_HANDLE    0xDF6E18
+
+// NOPs
+#define FS_OFFSET_1310_SD_DAS_INIT         0x27744
+
+// Nintendo Paths
+#define FS_OFFSET_1310_NINTENDO_PATHS \
+{ \
+	{.opcode_reg = 3, .adrp_offset = 0x0006EBE0, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 3, .adrp_offset = 0x0007BEEC, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 4, .adrp_offset = 0x00082294, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 4, .adrp_offset = 0x0009422C, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 0, .adrp_offset = 0, .add_rel_offset = 0}, \
+}
+
+#endif // __FS_1310_H__

emummc/source/FS/offsets/1310_exfat.h

@@ -0,0 +1,59 @@
+/*
+ * Copyright (c) 2019 m4xw <m4x@m4xw.net>
+ * Copyright (c) 2019 Atmosphere-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __FS_1310_EXFAT_H__
+#define __FS_1310_EXFAT_H__
+
+// Accessor vtable getters
+#define FS_OFFSET_1310_EXFAT_SDMMC_ACCESSOR_GC   0x158C20
+#define FS_OFFSET_1310_EXFAT_SDMMC_ACCESSOR_SD   0x15AA30
+#define FS_OFFSET_1310_EXFAT_SDMMC_ACCESSOR_NAND 0x159150
+
+// Hooks
+#define FS_OFFSET_1310_EXFAT_SDMMC_WRAPPER_READ  0x1545C0
+#define FS_OFFSET_1310_EXFAT_SDMMC_WRAPPER_WRITE 0x154680
+#define FS_OFFSET_1310_EXFAT_RTLD                0x688
+#define FS_OFFSET_1310_EXFAT_RTLD_DESTINATION    ((uintptr_t)(INT64_C(-0x3C)))
+
+#define FS_OFFSET_1310_EXFAT_CLKRST_SET_MIN_V_CLK_RATE 0x1537C0
+
+// Misc funcs
+#define FS_OFFSET_1310_EXFAT_LOCK_MUTEX          0x29690
+#define FS_OFFSET_1310_EXFAT_UNLOCK_MUTEX        0x296E0
+
+#define FS_OFFSET_1310_EXFAT_SDMMC_WRAPPER_CONTROLLER_OPEN  0x1544A0
+#define FS_OFFSET_1310_EXFAT_SDMMC_WRAPPER_CONTROLLER_CLOSE 0x154530
+
+// Misc Data
+#define FS_OFFSET_1310_EXFAT_SD_MUTEX            0xE203E8
+#define FS_OFFSET_1310_EXFAT_NAND_MUTEX          0xE1B768
+#define FS_OFFSET_1310_EXFAT_ACTIVE_PARTITION    0xE1B7A8
+#define FS_OFFSET_1310_EXFAT_SDMMC_DAS_HANDLE    0xE03E18
+
+// NOPs
+#define FS_OFFSET_1310_EXFAT_SD_DAS_INIT         0x27744
+
+// Nintendo Paths
+#define FS_OFFSET_1310_EXFAT_NINTENDO_PATHS \
+{ \
+	{.opcode_reg = 3, .adrp_offset = 0x0006EBE0, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 3, .adrp_offset = 0x0007BEEC, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 4, .adrp_offset = 0x00082294, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 4, .adrp_offset = 0x0009422C, .add_rel_offset = 0x00000004}, \
+	{.opcode_reg = 0, .adrp_offset = 0, .add_rel_offset = 0}, \
+}
+
+#endif // __FS_1310_EXFAT_H__

fusee/program/source/fusee_stratosphere.cpp

@@ -150,6 +150,9 @@ namespace ams::nxboot {
             FsVersion_13_0_0,
             FsVersion_13_0_0_Exfat,
 
+            FsVersion_13_1_0,
+            FsVersion_13_1_0_Exfat,
+
             FsVersion_Count,
         };
 
@@ -215,6 +218,9 @@ namespace ams::nxboot {
 
             { 0x7D, 0x20, 0x05, 0x47, 0x17, 0x8A, 0x83, 0x6A }, /* FsVersion_13_0_0 */
             { 0x51, 0xEB, 0xFA, 0x9C, 0xCF, 0x66, 0xC0, 0x9E }, /* FsVersion_13_0_0_Exfat */
+
+            { 0x91, 0xBA, 0x65, 0xA2, 0x1C, 0x1D, 0x50, 0xAE }, /* FsVersion_13_1_0 */
+            { 0x76, 0x38, 0x27, 0xEE, 0x9C, 0x20, 0x7E, 0x5B }, /* FsVersion_13_1_0_Exfat */
         };
 
         const InitialProcessBinaryHeader *FindInitialProcessBinary(const pkg2::Package2Header *header, const u8 *data, ams::TargetFirmware target_firmware) {
@@ -603,6 +609,11 @@ namespace ams::nxboot {
                     AddPatch(fs_meta, 0x159119, NogcPatch0, sizeof(NogcPatch0));
                     AddPatch(fs_meta, 0x1426D0, NogcPatch1, sizeof(NogcPatch1));
                     break;
+                case FsVersion_13_1_0:
+                case FsVersion_13_1_0_Exfat:
+                    AddPatch(fs_meta, 0x1590B9, NogcPatch0, sizeof(NogcPatch0));
+                    AddPatch(fs_meta, 0x142670, NogcPatch1, sizeof(NogcPatch1));
+                    break;
                 default:
                     break;
             }

libraries/.gitrepo

@@ -6,7 +6,7 @@
 [subrepo]
 	remote = https://github.com/Atmosphere-NX/Atmosphere-libs
 	branch = master
-	commit = cf765c0946cc5c828364ae6bfccddc4041304f28
-	parent = 8634ea0f7c4f0e68adf2dfaaddc6ae1e225c4fc2
+	commit = ceff2f3712d66904747f77619cd6dc12a5dcea8b
+	parent = 6cf5205a28cfeb30930fd1cfd3c8369cd13edfa3
 	method = merge
 	cmdver = 0.4.1

libraries/config/arch/armv4t/arch.mk

@@ -0,0 +1,11 @@
+ifeq ($(strip $(DEVKITPRO)),)
+$(error "Please set DEVKITPRO in your environment. export DEVKITPRO=<path to>/devkitpro")
+endif
+
+include $(DEVKITPRO)/devkitARM/base_rules
+
+export ATMOSPHERE_DEFINES  += -DATMOSPHERE_ARCH_ARM_V4T
+export ATMOSPHERE_SETTINGS +=
+export ATMOSPHERE_CFLAGS   +=
+export ATMOSPHERE_CXXFLAGS +=
+export ATMOSPHERE_ASFLAGS  +=

libraries/config/arch/armv8a/arch.mk

@@ -0,0 +1,11 @@
+ifeq ($(strip $(DEVKITPRO)),)
+$(error "Please set DEVKITPRO in your environment. export DEVKITPRO=<path to>/devkitpro")
+endif
+
+include $(DEVKITPRO)/devkitA64/base_rules
+
+export ATMOSPHERE_DEFINES  += -DATMOSPHERE_ARCH_ARM_V8A
+export ATMOSPHERE_SETTINGS +=
+export ATMOSPHERE_CFLAGS   +=
+export ATMOSPHERE_CXXFLAGS +=
+export ATMOSPHERE_ASFLAGS  +=

libraries/config/board/qemu/virt/board.mk

@@ -0,0 +1,5 @@
+export ATMOSPHERE_DEFINES  += -DATMOSPHERE_BOARD_QEMU_VIRT -D__SWITCH__
+export ATMOSPHERE_SETTINGS +=
+export ATMOSPHERE_CFLAGS   +=
+export ATMOSPHERE_CXXFLAGS +=
+export ATMOSPHERE_ASFLAGS  +=

libraries/config/common.mk

@@ -39,6 +39,9 @@ export ATMOSPHERE_ARCH_NAME  := arm64
 export ATMOSPHERE_BOARD_NAME := nintendo_nx
 export ATMOSPHERE_OS_NAME    := horizon
 
+export ATMOSPHERE_SUB_ARCH_DIR  = armv8a
+export ATMOSPHERE_SUB_ARCH_NAME = armv8a
+
 export ATMOSPHERE_CPU_EXTENSIONS := arm_crypto_extension aarch64_crypto_extension
 else ifeq ($(ATMOSPHERE_CPU),arm7tdmi)
 export ATMOSPHERE_ARCH_DIR   := arm
@@ -49,9 +52,30 @@ export ATMOSPHERE_ARCH_NAME  := arm
 export ATMOSPHERE_BOARD_NAME := nintendo_nx
 export ATMOSPHERE_OS_NAME    := horizon
 
+export ATMOSPHERE_SUB_ARCH_DIR  = armv4t
+export ATMOSPHERE_SUB_ARCH_NAME = armv4t
+
 export ATMOSPHERE_CPU_EXTENSIONS :=
 endif
 
+else ifeq ($(ATMOSPHERE_BOARD),qemu-virt)
+
+
+ifeq ($(ATMOSPHERE_CPU),arm-cortex-a57)
+export ATMOSPHERE_ARCH_DIR   := arm64
+export ATMOSPHERE_BOARD_DIR  := qemu/virt
+export ATMOSPHERE_OS_DIR     := horizon
+
+export ATMOSPHERE_ARCH_NAME  := arm64
+export ATMOSPHERE_BOARD_NAME := qemu_virt
+export ATMOSPHERE_OS_NAME    := horizon
+
+export ATMOSPHERE_SUB_ARCH_DIR  = armv8a
+export ATMOSPHERE_SUB_ARCH_NAME = armv8a
+
+export ATMOSPHERE_CPU_EXTENSIONS := arm_crypto_extension aarch64_crypto_extension
+endif
+
 endif
 
 ifeq ($(ATMOSPHERE_CPU),arm-cortex-a57)
@@ -70,14 +94,27 @@ export ATMOSPHERE_BOARD_MAKE_DIR := $(ATMOSPHERE_CONFIG_MAKE_DIR)/board/$(ATMOSP
 export ATMOSPHERE_OS_MAKE_DIR    := $(ATMOSPHERE_CONFIG_MAKE_DIR)/os/$(ATMOSPHERE_OS_DIR)
 export ATMOSPHERE_CPU_MAKE_DIR   := $(ATMOSPHERE_ARCH_MAKE_DIR)/cpu/$(ATMOSPHERE_CPU_DIR)
 
+ifneq ($(strip $(ATMOSPHERE_SUB_ARCH_NAME)),)
+export ATMOSPHERE_LIBRARY_DIR := lib_$(ATMOSPHERE_BOARD_NAME)_$(ATMOSPHERE_ARCH_NAME)_$(ATMOSPHERE_SUB_ARCH_NAME)
+export ATMOSPHERE_BUILD_DIR := build_$(ATMOSPHERE_BOARD_NAME)_$(ATMOSPHERE_ARCH_NAME)_$(ATMOSPHERE_SUB_ARCH_NAME)
+else
 export ATMOSPHERE_LIBRARY_DIR := lib_$(ATMOSPHERE_BOARD_NAME)_$(ATMOSPHERE_ARCH_NAME)
 export ATMOSPHERE_BUILD_DIR := build_$(ATMOSPHERE_BOARD_NAME)_$(ATMOSPHERE_ARCH_NAME)
+endif
 
 include $(ATMOSPHERE_ARCH_MAKE_DIR)/arch.mk
 include $(ATMOSPHERE_BOARD_MAKE_DIR)/board.mk
 include $(ATMOSPHERE_OS_MAKE_DIR)/os.mk
 include $(ATMOSPHERE_CPU_MAKE_DIR)/cpu.mk
 
+
+ifneq ($(strip $(ATMOSPHERE_SUB_ARCH_NAME)),)
+export ATMOSPHERE_SUB_ARCH_MAKE_DIR  := $(ATMOSPHERE_CONFIG_MAKE_DIR)/arch/$(ATMOSPHERE_SUB_ARCH_DIR)
+
+include $(ATMOSPHERE_SUB_ARCH_MAKE_DIR)/arch.mk
+endif
+
+
 #---------------------------------------------------------------------------------
 # get atmosphere git revision information
 #---------------------------------------------------------------------------------
@@ -110,12 +147,20 @@ DATA         := data
 INCLUDES     := include
 
 GENERAL_SOURCE_DIRS=$1 $(foreach d,$(filter-out $1/arch $1/board $1/os $1/cpu $1,$(wildcard $1/*)),$(if $(wildcard $d/.),$(filter-out $d,$(call GENERAL_SOURCE_DIRS,$d)) $d,))
+
 SPECIFIC_SOURCE_DIRS=$(if $(wildcard $1/$2/$3/.*),$1/$2/$3 $(call DIR_WILDCARD,$1/$2/$3),$(if $(wildcard $1/$2/generic/.*), $1/$2/generic $(call DIR_WILDCARD,$1/$2/generic),))
+
+ifneq ($(strip $(ATMOSPHERE_SUB_ARCH_NAME)),)
+SPECIFIC_SOURCE_DIRS_ARCH=$(if $(wildcard $1/$2/$3/.*),$1/$2/$3 $(call DIR_WILDCARD,$1/$2/$3),$(if $(wildcard $1/$2/$4/.*),$1/$2/$4 $(call DIR_WILDCARD,$1/$2/$4),$(if $(wildcard $1/$2/generic/.*), $1/$2/generic $(call DIR_WILDCARD,$1/$2/generic),)))
+else
+SPECIFIC_SOURCE_DIRS_ARCH=$(if $(wildcard $1/$2/$3/.*),$1/$2/$3 $(call DIR_WILDCARD,$1/$2/$3),$(if $(wildcard $1/$2/generic/.*), $1/$2/generic $(call DIR_WILDCARD,$1/$2/generic),))
+endif
+
 UNFILTERED_SOURCE_DIRS=$1 $(foreach d,$(wildcard $1/*),$(if $(wildcard $d/.),$(call DIR_WILDCARD,$d) $d,))
 
 ALL_SOURCE_DIRS=$(foreach d,$(call GENERAL_SOURCE_DIRS,$1), \
                     $d \
-                    $(call SPECIFIC_SOURCE_DIRS,$d,arch,$(ATMOSPHERE_ARCH_DIR)) \
+                    $(call SPECIFIC_SOURCE_DIRS_ARCH,$d,arch,$(ATMOSPHERE_ARCH_DIR),$(ATMOSPHERE_SUB_ARCH_DIR)) \
                     $(call SPECIFIC_SOURCE_DIRS,$d,board,$(ATMOSPHERE_BOARD_DIR)) \
                     $(call SPECIFIC_SOURCE_DIRS,$d,os,$(ATMOSPHERE_OS_DIR)) \
                     $(call SPECIFIC_SOURCE_DIRS,$d,cpu,$(ATMOSPHERE_ARCH_DIR)/$(ATMOSPHERE_CPU_DIR)) \
@@ -132,7 +177,7 @@ FIND_SOURCE_FILES=$(foreach dir,$1,$(filter-out $(notdir $(wildcard $(dir)/*.arc
                                                     $(notdir $(wildcard $(dir)/*.os.*.$2)) \
                                                     $(notdir $(wildcard $(dir)/.cpu.*.$2)), \
                                                 $(notdir $(wildcard $(dir)/*.$2)))) \
-                  $(foreach dir,$1,$(call FIND_SPECIFIC_SOURCE_FILES,$(dir),arch,$(ATMOSPHERE_ARCH_NAME),$2)) \
+                  $(foreach dir,$1,$(call FIND_SPECIFIC_SOURCE_FILES_EX,$(dir),arch,$(ATMOSPHERE_ARCH_NAME) $(ATMOSPHERE_SUB_ARCH_NAME),$2)) \
                   $(foreach dir,$1,$(call FIND_SPECIFIC_SOURCE_FILES,$(dir),board,$(ATMOSPHERE_BOARD_NAME),$2)) \
                   $(foreach dir,$1,$(call FIND_SPECIFIC_SOURCE_FILES,$(dir),os,$(ATMOSPHERE_OS_NAME),$2)) \
                   $(foreach dir,$1,$(call FIND_SPECIFIC_SOURCE_FILES_EX,$(dir),cpu,$(ATMOSPHERE_CPU_NAME) $(ATMOSPHERE_CPU_EXTENSIONS),$2))

libraries/libmesosphere/Makefile

@@ -104,6 +104,18 @@ $(eval $(call ATMOSPHERE_ADD_TARGET, audit, $(TARGET)_audit.a, \
 	ATMOSPHERE_BUILD_SETTINGS="-DMESOSPHERE_BUILD_FOR_AUDITING" \
 ))
 
+$(eval $(call ATMOSPHERE_ADD_TARGET, qemu_virt_release, $(TARGET)_qemu_virt.a, \
+	ATMOSPHERE_BUILD_SETTINGS="" \
+))
+
+$(eval $(call ATMOSPHERE_ADD_TARGET, qemu_virt_debug, $(TARGET)_qemu_virt_debug.a, \
+	ATMOSPHERE_BUILD_SETTINGS="-DMESOSPHERE_BUILD_FOR_DEBUGGING" \
+))
+
+$(eval $(call ATMOSPHERE_ADD_TARGET, qemu_virt_audit, $(TARGET)_qemu_virt_audit.a, \
+	ATMOSPHERE_BUILD_SETTINGS="-DMESOSPHERE_BUILD_FOR_AUDITING" \
+))
+
 #---------------------------------------------------------------------------------
 
 -include $(ATMOSPHERE_BOARD_NAME)_$(ATMOSPHERE_ARCH_NAME).mk

libraries/libmesosphere/include/mesosphere.hpp

@@ -93,3 +93,4 @@
 
 /* Deferred includes. */
 #include <mesosphere/kern_k_auto_object_impls.hpp>
+#include <mesosphere/kern_k_scheduler_impls.hpp>

libraries/libmesosphere/include/mesosphere/arch/arm64/init/kern_k_init_page_table.hpp

@@ -279,7 +279,7 @@ namespace ams::kern::arch::arm64::init {
 
                 /* Invalidate the entire tlb. */
                 cpu::DataSynchronizationBarrierInnerShareable();
-                cpu::InvalidateEntireTlb();
+                cpu::InvalidateEntireTlbInnerShareable();
 
                 /* Copy data, if we should. */
                 const u64 negative_block_size_for_mask = static_cast<u64>(-static_cast<s64>(block_size));
@@ -350,7 +350,6 @@ namespace ams::kern::arch::arm64::init {
                     /* If we don't already have an L2 table, we need to make a new one. */
                     if (!l1_entry->IsTable()) {
                         KPhysicalAddress new_table = AllocateNewPageTable(allocator);
-                        ClearNewPageTable(new_table);
                         *l1_entry = L1PageTableEntry(PageTableEntry::TableTag{}, new_table, attr.IsPrivilegedExecuteNever());
                         cpu::DataSynchronizationBarrierInnerShareable();
                     }
@@ -361,12 +360,12 @@ namespace ams::kern::arch::arm64::init {
                     if (util::IsAligned(GetInteger(virt_addr), L2ContiguousBlockSize) && util::IsAligned(GetInteger(phys_addr), L2ContiguousBlockSize) && size >= L2ContiguousBlockSize) {
                         for (size_t i = 0; i < L2ContiguousBlockSize / L2BlockSize; i++) {
                             l2_entry[i] = L2PageTableEntry(PageTableEntry::BlockTag{}, phys_addr, attr, PageTableEntry::SoftwareReservedBit_None, true);
-                            cpu::DataSynchronizationBarrierInnerShareable();
 
                             virt_addr += L2BlockSize;
                             phys_addr += L2BlockSize;
                             size      -= L2BlockSize;
                         }
+                        cpu::DataSynchronizationBarrierInnerShareable();
                         continue;
                     }
 
@@ -384,7 +383,6 @@ namespace ams::kern::arch::arm64::init {
                     /* If we don't already have an L3 table, we need to make a new one. */
                     if (!l2_entry->IsTable()) {
                         KPhysicalAddress new_table = AllocateNewPageTable(allocator);
-                        ClearNewPageTable(new_table);
                         *l2_entry = L2PageTableEntry(PageTableEntry::TableTag{}, new_table, attr.IsPrivilegedExecuteNever());
                         cpu::DataSynchronizationBarrierInnerShareable();
                     }
@@ -395,12 +393,12 @@ namespace ams::kern::arch::arm64::init {
                     if (util::IsAligned(GetInteger(virt_addr), L3ContiguousBlockSize) && util::IsAligned(GetInteger(phys_addr), L3ContiguousBlockSize) && size >= L3ContiguousBlockSize) {
                         for (size_t i = 0; i < L3ContiguousBlockSize / L3BlockSize; i++) {
                             l3_entry[i] = L3PageTableEntry(PageTableEntry::BlockTag{}, phys_addr, attr, PageTableEntry::SoftwareReservedBit_None, true);
-                            cpu::DataSynchronizationBarrierInnerShareable();
 
                             virt_addr += L3BlockSize;
                             phys_addr += L3BlockSize;
                             size      -= L3BlockSize;
                         }
+                        cpu::DataSynchronizationBarrierInnerShareable();
                         continue;
                     }
 

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_assembly_offsets.h

@@ -19,6 +19,9 @@
 /* TODO: Different header for this? */
 #define AMS_KERN_NUM_SUPERVISOR_CALLS 0xC0
 
+/* ams::kern::KThread, https://github.com/Atmosphere-NX/Atmosphere/blob/master/libraries/libmesosphere/include/mesosphere/kern_k_thread.hpp */
+#define THREAD_THREAD_CONTEXT 0xD0
+
 /* ams::kern::KThread::StackParameters, https://github.com/Atmosphere-NX/Atmosphere/blob/master/libraries/libmesosphere/include/mesosphere/kern_k_thread.hpp */
 #define THREAD_STACK_PARAMETERS_SIZE                    0x30
 #define THREAD_STACK_PARAMETERS_SVC_PERMISSION          0x00

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_cpu.hpp

@@ -30,6 +30,8 @@ namespace ams::kern::arch::arm64::cpu {
 
 #if defined(ATMOSPHERE_BOARD_NINTENDO_NX)
     constexpr inline size_t NumCores = 4;
+#elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+    constexpr inline size_t NumCores = 4;
 #else
     #error "Unknown Board for cpu::NumCores"
 #endif
@@ -50,10 +52,19 @@ namespace ams::kern::arch::arm64::cpu {
         __asm__ __volatile__("dmb sy" ::: "memory");
     }
 
+    ALWAYS_INLINE void DataMemoryBarrierInnerShareable() {
+        __asm__ __volatile__("dmb ish" ::: "memory");
+    }
+
     ALWAYS_INLINE void InstructionMemoryBarrier() {
         __asm__ __volatile__("isb" ::: "memory");
     }
 
+    ALWAYS_INLINE void EnsureInstructionConsistencyInnerShareable() {
+        DataSynchronizationBarrierInnerShareable();
+        InstructionMemoryBarrier();
+    }
+
     ALWAYS_INLINE void EnsureInstructionConsistency() {
         DataSynchronizationBarrier();
         InstructionMemoryBarrier();
@@ -171,7 +182,6 @@ namespace ams::kern::arch::arm64::cpu {
     NOINLINE void SynchronizeAllCores();
 
     /* Cache management helpers. */
-    void ClearPageToZeroImpl(void *);
     void StoreEntireCacheForInit();
     void FlushEntireCacheForInit();
 
@@ -184,10 +194,16 @@ namespace ams::kern::arch::arm64::cpu {
 
     void InvalidateEntireInstructionCache();
 
-    ALWAYS_INLINE void ClearPageToZero(void *page) {
+    ALWAYS_INLINE void ClearPageToZero(void * const page) {
         MESOSPHERE_ASSERT(util::IsAligned(reinterpret_cast<uintptr_t>(page), PageSize));
         MESOSPHERE_ASSERT(page != nullptr);
-        ClearPageToZeroImpl(page);
+
+        uintptr_t cur = reinterpret_cast<uintptr_t>(__builtin_assume_aligned(page, PageSize));
+        const uintptr_t last = cur + PageSize - DataCacheLineSize;
+
+        for (/* ... */; cur <= last; cur += DataCacheLineSize) {
+            __asm__ __volatile__("dc zva, %[cur]" :: [cur]"r"(cur) : "memory");
+        }
     }
 
     ALWAYS_INLINE void InvalidateTlbByAsid(u32 asid) {
@@ -207,6 +223,11 @@ namespace ams::kern::arch::arm64::cpu {
         EnsureInstructionConsistency();
     }
 
+    ALWAYS_INLINE void InvalidateEntireTlbInnerShareable() {
+        __asm__ __volatile__("tlbi vmalle1is" ::: "memory");
+        EnsureInstructionConsistencyInnerShareable();
+    }
+
     ALWAYS_INLINE void InvalidateEntireTlbDataOnly() {
         __asm__ __volatile__("tlbi vmalle1is" ::: "memory");
         DataSynchronizationBarrier();

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_interrupt_manager.hpp

@@ -105,35 +105,48 @@ namespace ams::kern::arch::arm64 {
             Result UnbindLocal(s32 irq);
             Result ClearGlobal(s32 irq);
             Result ClearLocal(s32 irq);
-        public:
-            static ALWAYS_INLINE u32 DisableInterrupts() {
+        private:
+            [[nodiscard]] static ALWAYS_INLINE u32 GetInterruptsEnabledState() {
                 u64 intr_state;
-                __asm__ __volatile__("mrs %[intr_state], daif\n"
-                                     "msr daifset, #2"
+                __asm__ __volatile__("mrs   %[intr_state], daif\n"
+                                     "ubfx  %[intr_state], %[intr_state], #7, #1"
                                      : [intr_state]"=r"(intr_state)
                                      :: "memory");
                 return intr_state;
             }
+        public:
+            static ALWAYS_INLINE void EnableInterrupts() {
+                __asm__ __volatile__("msr   daifclr, #2" ::: "memory");
+            }
 
-            static ALWAYS_INLINE u32 EnableInterrupts() {
-                u64 intr_state;
-                __asm__ __volatile__("mrs %[intr_state], daif\n"
-                                     "msr daifclr, #2"
-                                     : [intr_state]"=r"(intr_state)
-                                     :: "memory");
+            static ALWAYS_INLINE void DisableInterrupts() {
+                __asm__ __volatile__("msr   daifset, #2" ::: "memory");
+            }
+
+            [[nodiscard]] static ALWAYS_INLINE u32 GetInterruptsEnabledStateAndDisableInterrupts() {
+                const auto intr_state = GetInterruptsEnabledState();
+                DisableInterrupts();
+                return intr_state;
+            }
+
+            [[nodiscard]] static ALWAYS_INLINE u32 GetInterruptsEnabledStateAndEnableInterrupts() {
+                const auto intr_state = GetInterruptsEnabledState();
+                EnableInterrupts();
                 return intr_state;
             }
 
             static ALWAYS_INLINE void RestoreInterrupts(u32 intr_state) {
-                u64 cur_state;
-                __asm__ __volatile__("mrs %[cur_state], daif" : [cur_state]"=r"(cur_state));
-                __asm__ __volatile__("msr daif, %[intr_state]" :: [intr_state]"r"((cur_state & ~0x80ul) | (intr_state & 0x80)));
+                u64 tmp;
+                __asm__ __volatile__("mrs   %[tmp], daif\n"
+                                     "bfi   %[tmp], %[intr_state], #7, #1\n"
+                                     "msr daif, %[tmp]"
+                                     : [tmp]"=&r"(tmp)
+                                     : [intr_state]"r"(intr_state)
+                                     : "memory");
             }
 
             static ALWAYS_INLINE bool AreInterruptsEnabled() {
-                u64 intr_state;
-                __asm__ __volatile__("mrs %[intr_state], daif" : [intr_state]"=r"(intr_state));
-                return (intr_state & 0x80) == 0;
+                return GetInterruptsEnabledState() == 0;
             }
     };
 

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_interrupt_name.hpp

@@ -36,6 +36,10 @@ namespace ams::kern::arch::arm64 {
             KInterruptName_SecurePhysicalTimer    = 29,
             KInterruptName_NonSecurePhysicalTimer = 30,
             KInterruptName_LegacyNIrq             = 31,
+    #elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+            KInterruptName_VirtualTimer           = 27,
+            KInterruptName_SecurePhysicalTimer    = 29,
+            KInterruptName_NonSecurePhysicalTimer = 30,
     #endif
 
     #if defined(ATMOSPHERE_BOARD_NINTENDO_NX)

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_page_table.hpp

@@ -219,27 +219,27 @@ namespace ams::kern::arch::arm64 {
 
             Result ChangePermissions(KProcessAddress virt_addr, size_t num_pages, PageTableEntry entry_template, DisableMergeAttribute disable_merge_attr, bool refresh_mapping, PageLinkedList *page_list, bool reuse_ll);
 
-            static void PteDataSynchronizationBarrier() {
+            static ALWAYS_INLINE void PteDataSynchronizationBarrier() {
                 cpu::DataSynchronizationBarrierInnerShareable();
             }
 
-            static void ClearPageTable(KVirtualAddress table) {
+            static ALWAYS_INLINE void ClearPageTable(KVirtualAddress table) {
                 cpu::ClearPageToZero(GetVoidPointer(table));
             }
 
-            void OnTableUpdated() const {
+            ALWAYS_INLINE void OnTableUpdated() const {
                 cpu::InvalidateTlbByAsid(m_asid);
             }
 
-            void OnKernelTableUpdated() const {
+            ALWAYS_INLINE void OnKernelTableUpdated() const {
                 cpu::InvalidateEntireTlbDataOnly();
             }
 
-            void OnKernelTableSinglePageUpdated(KProcessAddress virt_addr) const {
+            ALWAYS_INLINE void OnKernelTableSinglePageUpdated(KProcessAddress virt_addr) const {
                 cpu::InvalidateTlbByVaDataOnly(virt_addr);
             }
 
-            void NoteUpdated() const {
+            ALWAYS_INLINE void NoteUpdated() const {
                 cpu::DataSynchronizationBarrier();
 
                 if (this->IsKernel()) {
@@ -249,7 +249,7 @@ namespace ams::kern::arch::arm64 {
                 }
             }
 
-            void NoteSingleKernelPageUpdated(KProcessAddress virt_addr) const {
+            ALWAYS_INLINE void NoteSingleKernelPageUpdated(KProcessAddress virt_addr) const {
                 MESOSPHERE_ASSERT(this->IsKernel());
 
                 cpu::DataSynchronizationBarrier();

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_slab_heap_impl.hpp

@@ -45,6 +45,7 @@ namespace ams::kern::arch::arm64 {
 
             /* Select L1 cache. */
             cpu::SetCsselrEl1(0);
+            cpu::InstructionMemoryBarrier();
 
             /* Check that the L1 cache is not direct-mapped. */
             return cpu::CacheSizeIdRegisterAccessor().GetAssociativity() != 0;

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_secure_monitor_base.hpp

@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_select_cpu.hpp>
+#include <mesosphere/kern_select_interrupt_manager.hpp>
+
+namespace ams::kern::arch::arm64::smc {
+
+    template<int SmcId, bool DisableInterrupt>
+    void SecureMonitorCall(u64 *buf) {
+        /* Load arguments into registers. */
+        register u64 x0 asm("x0") = buf[0];
+        register u64 x1 asm("x1") = buf[1];
+        register u64 x2 asm("x2") = buf[2];
+        register u64 x3 asm("x3") = buf[3];
+        register u64 x4 asm("x4") = buf[4];
+        register u64 x5 asm("x5") = buf[5];
+        register u64 x6 asm("x6") = buf[6];
+        register u64 x7 asm("x7") = buf[7];
+
+        /* Perform the call. */
+        if constexpr (DisableInterrupt) {
+            KScopedInterruptDisable di;
+
+            /* Backup the current thread pointer. */
+            const uintptr_t current_thread_pointer_value = cpu::GetCurrentThreadPointerValue();
+
+            __asm__ __volatile__("smc %c[smc_id]"
+                                : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
+                                : [smc_id]"i"(SmcId)
+                                : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
+                                );
+
+            /* Restore the current thread pointer into X18. */
+            cpu::SetCurrentThreadPointerValue(current_thread_pointer_value);
+        } else {
+            /* Backup the current thread pointer. */
+            const uintptr_t current_thread_pointer_value = cpu::GetCurrentThreadPointerValue();
+
+            __asm__ __volatile__("smc %c[smc_id]"
+                                : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
+                                : [smc_id]"i"(SmcId)
+                                : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
+                                );
+
+            /* Restore the current thread pointer into X18. */
+            cpu::SetCurrentThreadPointerValue(current_thread_pointer_value);
+        }
+
+        /* Store arguments to output. */
+        buf[0] = x0;
+        buf[1] = x1;
+        buf[2] = x2;
+        buf[3] = x3;
+        buf[4] = x4;
+        buf[5] = x5;
+        buf[6] = x6;
+        buf[7] = x7;
+    }
+
+    enum PsciFunction {
+        PsciFunction_CpuSuspend = 0xC4000001,
+        PsciFunction_CpuOff     = 0x84000002,
+        PsciFunction_CpuOn      = 0xC4000003,
+    };
+
+    template<int SmcId, bool DisableInterrupt>
+    u64 PsciCall(PsciFunction function, u64 x1 = 0, u64 x2 = 0, u64 x3 = 0, u64 x4 = 0, u64 x5 = 0, u64 x6 = 0, u64 x7 = 0) {
+        ams::svc::lp64::SecureMonitorArguments args = { { function, x1, x2, x3, x4, x5, x6, x7 } };
+
+        SecureMonitorCall<SmcId, DisableInterrupt>(args.r);
+
+        return args.r[0];
+    }
+
+    template<int SmcId, bool DisableInterrupt>
+    u64 CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg) {
+        return PsciCall<SmcId, DisableInterrupt>(PsciFunction_CpuOn, core_id, entrypoint, arg);
+    }
+
+}

libraries/libmesosphere/include/mesosphere/board/generic/kern_k_device_page_table.hpp

@@ -44,13 +44,13 @@ namespace ams::kern::board::generic {
                 return ams::kern::svc::ResultNotImplemented();
             }
 
-            Result ALWAYS_INLINE Map(size_t *out_mapped_size, const KPageGroup &pg, KDeviceVirtualAddress device_address, ams::svc::MemoryPermission device_perm, bool refresh_mappings) {
-                MESOSPHERE_UNUSED(out_mapped_size, pg, device_address, device_perm, refresh_mappings);
+            Result ALWAYS_INLINE Map(KProcessPageTable *page_table, KProcessAddress process_address, size_t size, KDeviceVirtualAddress device_address, ams::svc::MemoryPermission device_perm, bool is_aligned) {
+                MESOSPHERE_UNUSED(page_table, process_address, size, device_address, device_perm, is_aligned);
                 return ams::kern::svc::ResultNotImplemented();
             }
 
-            Result ALWAYS_INLINE Unmap(const KPageGroup &pg, KDeviceVirtualAddress device_address) {
-                MESOSPHERE_UNUSED(pg, device_address);
+            Result ALWAYS_INLINE Unmap(KProcessPageTable *page_table, KProcessAddress process_address, size_t size, KDeviceVirtualAddress device_address) {
+                MESOSPHERE_UNUSED(page_table, process_address, size, device_address);
                 return ams::kern::svc::ResultNotImplemented();
             }
 

libraries/libmesosphere/include/mesosphere/board/nintendo/nx/kern_k_memory_layout.hpp

@@ -15,9 +15,12 @@
  */
 #pragma once
 #include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_typed_address.hpp>
 
 namespace ams::kern {
 
+    constexpr inline KPhysicalAddress MainMemoryAddress = 0x80000000;
+
     constexpr inline size_t MainMemorySize    = 4_GB;
     constexpr inline size_t MainMemorySizeMax = 8_GB;
 

libraries/libmesosphere/include/mesosphere/board/nintendo/nx/kern_k_system_control.hpp

@@ -15,23 +15,17 @@
  */
 #pragma once
 #include <mesosphere/kern_common.hpp>
-
-namespace ams::kern {
-
-    struct InitialProcessBinaryLayout;
-
-}
+#include <mesosphere/kern_k_system_control_base.hpp>
 
 namespace ams::kern::board::nintendo::nx {
 
-    class KSystemControl {
+    class KSystemControl : public KSystemControlBase {
         public:
-            class Init {
+            class Init : public KSystemControlBase::Init {
                 public:
                     /* Initialization. */
+                    static size_t GetRealMemorySize();
                     static size_t GetIntendedMemorySize();
-                    static KPhysicalAddress GetKernelPhysicalBaseAddress(uintptr_t base_address);
-                    static void GetInitialProcessBinaryLayout(InitialProcessBinaryLayout *out);
                     static bool ShouldIncreaseThreadResourceLimit();
                     static void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg);
                     static size_t GetApplicationPoolSize();
@@ -40,7 +34,7 @@ namespace ams::kern::board::nintendo::nx {
                     static u8 GetDebugLogUartPort();
 
                     /* Randomness. */
-                    static void GenerateRandomBytes(void *dst, size_t size);
+                    static void GenerateRandom(u64 *dst, size_t count);
                     static u64  GenerateRandomRange(u64 min, u64 max);
             };
         public:
@@ -50,7 +44,7 @@ namespace ams::kern::board::nintendo::nx {
             static NOINLINE u32 GetCreateProcessMemoryPool();
 
             /* Randomness. */
-            static void GenerateRandomBytes(void *dst, size_t size);
+            static void GenerateRandom(u64 *dst, size_t count);
             static u64  GenerateRandomRange(u64 min, u64 max);
             static u64  GenerateRandomU64();
 
@@ -58,23 +52,12 @@ namespace ams::kern::board::nintendo::nx {
             static void ReadWriteRegisterPrivileged(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value);
             static Result ReadWriteRegister(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value);
 
-            static ALWAYS_INLINE u32 ReadRegisterPrivileged(ams::svc::PhysicalAddress address) {
-                u32 v;
-                ReadWriteRegisterPrivileged(std::addressof(v), address, 0x00000000u, 0);
-                return v;
-            }
-
-            static ALWAYS_INLINE void WriteRegisterPrivileged(ams::svc::PhysicalAddress address, u32 value) {
-                u32 v;
-                ReadWriteRegisterPrivileged(std::addressof(v), address, 0xFFFFFFFFu, value);
-            }
-
             /* Power management. */
             static void SleepSystem();
             static NORETURN void StopSystem(void *arg = nullptr);
 
             /* User access. */
-            static void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
+            static void CallSecureMonitorFromUserImpl(ams::svc::lp64::SecureMonitorArguments *args);
 
             /* Secure Memory. */
             static size_t CalculateRequiredSecureMemorySize(size_t size, u32 pool);

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_cpu_map.hpp

@@ -0,0 +1,33 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+
+namespace ams::kern::board::qemu::virt::impl::cpu {
+
+    /* Virtual to Physical core map. */
+    constexpr inline const s32 VirtualToPhysicalCoreMap[BITSIZEOF(u64)] = {
+        0, 1, 2, 3, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 3,
+    };
+
+}

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_k_memory_layout.hpp

@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_typed_address.hpp>
+
+namespace ams::kern {
+
+    constexpr inline KPhysicalAddress MainMemoryAddress = 0x40000000;
+
+    constexpr inline size_t MainMemorySize    = 4_GB;
+    constexpr inline size_t MainMemorySizeMax = 8_GB;
+
+}

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_k_memory_region_device_types.inc

@@ -0,0 +1,20 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* All architectures must define NumBoardDeviceRegions. */
+constexpr inline const auto NumBoardDeviceRegions = 0;
+                                                                                                /* UNUSED: .Derive(NumBoardDeviceRegions, 0); */
+

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_k_system_control.hpp

@@ -0,0 +1,28 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_system_control_base.hpp>
+
+namespace ams::kern::board::qemu::virt {
+
+    class KSystemControl : public KSystemControlBase {
+        public:
+            /* User access. */
+            static void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
+    };
+
+}

libraries/libmesosphere/include/mesosphere/kern_build_config.hpp

@@ -38,4 +38,19 @@
 /* at the cost of storing class tokens inside the class object. */
 /* However, as of (10/16/2021) KAutoObject has an unused class member */
 /* of the right side, and so this does not actually cost any space. */
-#define MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST
+#define MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST
+
+/* NOTE: This enables usage of KDebug handles as parameter for svc::GetInfo */
+/* calls which require a process parameter. This enables a debugger to obtain */
+/* address space/layout information, for example. However, it changes abi, and so */
+/* this define allows toggling the extension. */
+#define MESOSPHERE_ENABLE_GET_INFO_OF_DEBUG_PROCESS
+
+/* NOTE: This uses currently-reserved bits inside the MapRange capability */
+/* in order to support large physical addresses (40-bit instead of 36). */
+/* This is toggleable in order to disable it if N ever uses those bits. */
+#if defined(ATMOSPHERE_BOARD_NINTENDO_NX)
+//#define MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES
+#else
+#define MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES
+#endif

libraries/libmesosphere/include/mesosphere/kern_debug_log.hpp

@@ -43,6 +43,8 @@ namespace ams::kern {
 
     #ifdef ATMOSPHERE_BOARD_NINTENDO_NX
         #define MESOSPHERE_DEBUG_LOG_USE_UART
+    #elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+        #define MESOSPHERE_DEBUG_LOG_USE_SEMIHOSTING
     #else
         #error "Unknown board for Default Debug Log Source"
     #endif

libraries/libmesosphere/include/mesosphere/kern_k_capabilities.hpp

@@ -82,7 +82,11 @@ namespace ams::kern {
                 DEFINE_FIELD(Index, Mask,    3);
             };
 
+            #if defined(MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES)
+            static constexpr u64 PhysicalMapAllowedMask = (1ul << 40) - 1;
+            #else
             static constexpr u64 PhysicalMapAllowedMask = (1ul << 36) - 1;
+            #endif
 
             struct MapRange {
                 using IdBits = Field<0, CapabilityId<CapabilityType::MapRange> + 1>;
@@ -94,9 +98,15 @@ namespace ams::kern {
             struct MapRangeSize {
                 using IdBits = Field<0, CapabilityId<CapabilityType::MapRange> + 1>;
 
-                DEFINE_FIELD(Pages,    IdBits,   20);
+                DEFINE_FIELD(Pages, IdBits, 20);
+
+                #if defined(MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES)
+                DEFINE_FIELD(AddressHigh, Pages,        4);
+                DEFINE_FIELD(Normal,      AddressHigh,  1, bool);
+                #else
                 DEFINE_FIELD(Reserved, Pages,     4);
                 DEFINE_FIELD(Normal,   Reserved,  1, bool);
+                #endif
             };
 
             struct MapIoPage {

libraries/libmesosphere/include/mesosphere/kern_k_dynamic_resource_manager.hpp

@@ -46,7 +46,7 @@ namespace ams::kern {
                 return m_slab_heap->Allocate(m_page_allocator);
             }
 
-            void Free(T *t) const {
+            ALWAYS_INLINE void Free(T *t) const {
                 m_slab_heap->Free(t);
             }
     };

libraries/libmesosphere/include/mesosphere/kern_k_memory_layout.hpp

@@ -20,6 +20,8 @@
 
 #if defined(ATMOSPHERE_BOARD_NINTENDO_NX)
     #include <mesosphere/board/nintendo/nx/kern_k_memory_layout.hpp>
+#elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+    #include <mesosphere/board/qemu/virt/kern_k_memory_layout.hpp>
 #else
     #error "Unknown board for KMemoryLayout"
 #endif
@@ -210,13 +212,17 @@ namespace ams::kern {
             static NOINLINE auto GetKernelPageTableHeapRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramKernelPtHeap); }
             static NOINLINE auto GetKernelInitPageTableRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramKernelInitPt); }
 
-            static NOINLINE auto GetKernelPoolManagementRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramPoolManagement); }
             static NOINLINE auto GetKernelPoolPartitionRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramPoolPartition); }
+            static NOINLINE auto GetKernelPoolManagementRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramPoolManagement); }
             static NOINLINE auto GetKernelSystemPoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramSystemPool); }
             static NOINLINE auto GetKernelSystemNonSecurePoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramSystemNonSecurePool); }
             static NOINLINE auto GetKernelAppletPoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramAppletPool); }
             static NOINLINE auto GetKernelApplicationPoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramApplicationPool); }
 
+            static NOINLINE bool HasKernelSystemNonSecurePoolRegion() { return GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramSystemNonSecurePool) != nullptr; }
+            static NOINLINE bool HasKernelAppletPoolRegion() { return GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramAppletPool) != nullptr; }
+            static NOINLINE bool HasKernelApplicationPoolRegion() { return GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramApplicationPool) != nullptr; }
+
             static NOINLINE auto GetKernelTraceBufferRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_KernelTraceBuffer); }
     };
 

libraries/libmesosphere/include/mesosphere/kern_k_page_bitmap.hpp

@@ -48,39 +48,20 @@ namespace ams::kern {
                     }
 
                     size_t SelectRandomBit(u64 bitmap) {
-                        u64 selected     = 0;
+                        u64 selected = 0;
 
-                        u64 cur_num_bits = BITSIZEOF(bitmap) / 2;
-                        u64 cur_mask     = (1ull << cur_num_bits) - 1;
+                        for (size_t cur_num_bits = BITSIZEOF(bitmap) / 2; cur_num_bits != 0; cur_num_bits /= 2) {
+                            const u64 high = (bitmap >> cur_num_bits);
+                            const u64 low  = (bitmap & (~(UINT64_C(0xFFFFFFFFFFFFFFFF) << cur_num_bits)));
 
-                        while (cur_num_bits) {
-                            const u64 low  = (bitmap >> 0) & cur_mask;
-                            const u64 high = (bitmap >> cur_num_bits) & cur_mask;
-
-                            bool choose_low;
-                            if (high == 0) {
-                                /* If only low val is set, choose low. */
-                                choose_low = true;
-                            } else if (low == 0) {
-                                /* If only high val is set, choose high. */
-                                choose_low = false;
-                            } else {
-                                /* If both are set, choose random. */
-                                choose_low = this->GenerateRandomBit();
-                            }
-
-                            /* If we chose low, proceed with low. */
-                            if (choose_low) {
-                                bitmap    = low;
-                                selected += 0;
-                            } else {
+                            /* Choose high if we have high and (don't have low or select high randomly). */
+                            if (high && (low == 0 || this->GenerateRandomBit())) {
                                 bitmap    = high;
                                 selected += cur_num_bits;
+                            } else {
+                                bitmap    = low;
+                                selected += 0;
                             }
-
-                            /* Proceed. */
-                            cur_num_bits /= 2;
-                            cur_mask >>= cur_num_bits;
                         }
 
                         return selected;

libraries/libmesosphere/include/mesosphere/kern_k_scheduler.hpp

@@ -211,18 +211,6 @@ namespace ams::kern {
             static consteval bool ValidateAssemblyOffsets();
     };
 
-    consteval bool KScheduler::ValidateAssemblyOffsets() {
-        static_assert(AMS_OFFSETOF(KScheduler, m_state.needs_scheduling)        == KSCHEDULER_NEEDS_SCHEDULING);
-        static_assert(AMS_OFFSETOF(KScheduler, m_state.interrupt_task_runnable) == KSCHEDULER_INTERRUPT_TASK_RUNNABLE);
-        static_assert(AMS_OFFSETOF(KScheduler, m_state.highest_priority_thread) == KSCHEDULER_HIGHEST_PRIORITY_THREAD);
-        static_assert(AMS_OFFSETOF(KScheduler, m_state.idle_thread_stack)       == KSCHEDULER_IDLE_THREAD_STACK);
-        static_assert(AMS_OFFSETOF(KScheduler, m_state.prev_thread)             == KSCHEDULER_PREVIOUS_THREAD);
-        static_assert(AMS_OFFSETOF(KScheduler, m_state.interrupt_task_manager)  == KSCHEDULER_INTERRUPT_TASK_MANAGER);
-
-        return true;
-    }
-    static_assert(KScheduler::ValidateAssemblyOffsets());
-
     class KScopedSchedulerLock : KScopedLock<KScheduler::LockType> {
         public:
             explicit ALWAYS_INLINE KScopedSchedulerLock() : KScopedLock(KScheduler::s_scheduler_lock) { /* ... */ }

libraries/libmesosphere/include/mesosphere/kern_k_scheduler_impls.hpp

@@ -0,0 +1,43 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_scheduler.hpp>
+#include <mesosphere/kern_select_interrupt_manager.hpp>
+
+namespace ams::kern {
+
+    /* NOTE: This header is included after all main headers. */
+    consteval bool KScheduler::ValidateAssemblyOffsets() {
+        static_assert(AMS_OFFSETOF(KScheduler, m_state.needs_scheduling)        == KSCHEDULER_NEEDS_SCHEDULING);
+        static_assert(AMS_OFFSETOF(KScheduler, m_state.interrupt_task_runnable) == KSCHEDULER_INTERRUPT_TASK_RUNNABLE);
+        static_assert(AMS_OFFSETOF(KScheduler, m_state.highest_priority_thread) == KSCHEDULER_HIGHEST_PRIORITY_THREAD);
+        static_assert(AMS_OFFSETOF(KScheduler, m_state.idle_thread_stack)       == KSCHEDULER_IDLE_THREAD_STACK);
+        static_assert(AMS_OFFSETOF(KScheduler, m_state.prev_thread)             == KSCHEDULER_PREVIOUS_THREAD);
+        static_assert(AMS_OFFSETOF(KScheduler, m_state.interrupt_task_manager)  == KSCHEDULER_INTERRUPT_TASK_MANAGER);
+
+        return true;
+    }
+    static_assert(KScheduler::ValidateAssemblyOffsets());
+
+    ALWAYS_INLINE void KScheduler::RescheduleOtherCores(u64 cores_needing_scheduling) {
+        if (const u64 core_mask = cores_needing_scheduling & ~(1ul << m_core_id); core_mask != 0) {
+            cpu::DataSynchronizationBarrier();
+            Kernel::GetInterruptManager().SendInterProcessorInterrupt(KInterruptName_Scheduler, core_mask);
+        }
+    }
+
+}

libraries/libmesosphere/include/mesosphere/kern_k_scheduler_lock.hpp

@@ -74,6 +74,9 @@ namespace ams::kern {
 
                 /* Release an instance of the lock. */
                 if ((--m_lock_count) == 0) {
+                    /* Perform a memory barrier here. */
+                    cpu::DataMemoryBarrierInnerShareable();
+
                     /* We're no longer going to hold the lock. Take note of what cores need scheduling. */
                     const u64 cores_needing_scheduling = SchedulerType::UpdateHighestPriorityThreads();
 

libraries/libmesosphere/include/mesosphere/kern_k_system_control_base.hpp

@@ -0,0 +1,110 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_spin_lock.hpp>
+
+namespace ams::kern {
+
+    struct InitialProcessBinaryLayout;
+
+}
+
+namespace ams::kern {
+
+    class KSystemControlBase {
+        protected:
+            /* Nintendo uses std::mt19937_t for randomness. */
+            /* To save space (and because mt19337_t isn't secure anyway), */
+            /* We will use TinyMT. */
+            static constinit inline bool         s_initialized_random_generator;
+            static constinit inline util::TinyMT s_random_generator{util::ConstantInitialize};
+            static constinit inline KSpinLock    s_random_lock;
+        public:
+            class Init {
+                public:
+                    /* Initialization. */
+                    static size_t GetRealMemorySize();
+                    static size_t GetIntendedMemorySize();
+                    static KPhysicalAddress GetKernelPhysicalBaseAddress(KPhysicalAddress base_address);
+                    static void GetInitialProcessBinaryLayout(InitialProcessBinaryLayout *out);
+                    static bool ShouldIncreaseThreadResourceLimit();
+                    static void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg);
+                    static size_t GetApplicationPoolSize();
+                    static size_t GetAppletPoolSize();
+                    static size_t GetMinimumNonSecureSystemPoolSize();
+                    static u8 GetDebugLogUartPort();
+
+                    /* Randomness. */
+                    static void GenerateRandom(u64 *dst, size_t count);
+                    static u64  GenerateRandomRange(u64 min, u64 max);
+            };
+        public:
+            /* Initialization. */
+            static NOINLINE void InitializePhase1(bool skip_target_system = false);
+            static NOINLINE void InitializePhase2();
+            static NOINLINE u32 GetCreateProcessMemoryPool();
+
+            /* Randomness. */
+            static void GenerateRandom(u64 *dst, size_t count);
+            static u64  GenerateRandomRange(u64 min, u64 max);
+            static u64  GenerateRandomU64();
+
+            /* Register access Access. */
+            static Result ReadWriteRegister(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value);
+            static void ReadWriteRegisterPrivileged(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value);
+
+            static u32 ReadRegisterPrivileged(ams::svc::PhysicalAddress address);
+            static void WriteRegisterPrivileged(ams::svc::PhysicalAddress address, u32 value);
+
+            /* Power management. */
+            static void SleepSystem();
+            static NORETURN void StopSystem(void *arg = nullptr);
+
+            /* User access. */
+            #if defined(ATMOSPHERE_ARCH_ARM64)
+            static void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
+            #endif
+
+            /* Secure Memory. */
+            static size_t CalculateRequiredSecureMemorySize(size_t size, u32 pool);
+            static Result AllocateSecureMemory(KVirtualAddress *out, size_t size, u32 pool);
+            static void FreeSecureMemory(KVirtualAddress address, size_t size, u32 pool);
+        protected:
+            template<typename F>
+            static ALWAYS_INLINE u64 GenerateUniformRange(u64 min, u64 max, F f) {
+                /* Handle the case where the difference is too large to represent. */
+                if (max == std::numeric_limits<u64>::max() && min == std::numeric_limits<u64>::min()) {
+                    return f();
+                }
+
+                /* Iterate until we get a value in range. */
+                const u64 range_size    = ((max + 1) - min);
+                const u64 effective_max = (std::numeric_limits<u64>::max() / range_size) * range_size;
+                while (true) {
+                    if (const u64 rnd = f(); rnd < effective_max) {
+                        return min + (rnd % range_size);
+                    }
+                }
+            }
+
+            /* User access. */
+            #if defined(ATMOSPHERE_ARCH_ARM64)
+            static void CallSecureMonitorFromUserImpl(ams::svc::lp64::SecureMonitorArguments *args);
+            #endif
+    };
+
+}

libraries/libmesosphere/include/mesosphere/kern_k_target_system.hpp

@@ -21,6 +21,7 @@ namespace ams::kern {
 
     class KTargetSystem {
         private:
+            friend class KSystemControlBase;
             friend class KSystemControl;
         private:
             static inline constinit bool s_is_debug_mode;

libraries/libmesosphere/include/mesosphere/kern_k_thread.hpp

@@ -405,8 +405,6 @@ namespace ams::kern {
             constexpr ThreadState GetState() const { return static_cast<ThreadState>(m_thread_state & ThreadState_Mask); }
             constexpr ThreadState GetRawState() const { return m_thread_state; }
 
-            NOINLINE KThreadContext *GetContextForSchedulerLoop();
-
             constexpr uintptr_t GetConditionVariableKey() const { return m_condvar_key; }
             constexpr uintptr_t GetAddressArbiterKey() const { return m_condvar_key; }
 
@@ -624,9 +622,7 @@ namespace ams::kern {
             void OnTimer();
             void DoWorkerTaskImpl();
         public:
-            static constexpr bool IsConditionVariableThreadTreeValid() {
-                return ConditionVariableThreadTreeTraits::IsValid();
-            }
+            static consteval bool IsKThreadStructurallyValid();
 
             static KThread *GetThreadFromId(u64 thread_id);
             static Result GetThreadList(s32 *out_num_threads, ams::kern::svc::KUserPointer<u64 *> out_thread_ids, s32 max_out_count);
@@ -634,7 +630,18 @@ namespace ams::kern {
             using ConditionVariableThreadTreeType = ConditionVariableThreadTree;
     };
     static_assert(alignof(KThread) == 0x10);
-    static_assert(KThread::IsConditionVariableThreadTreeValid());
+
+    consteval bool KThread::IsKThreadStructurallyValid() {
+        /* Check that the condition variable tree is valid. */
+        static_assert(ConditionVariableThreadTreeTraits::IsValid());
+
+        /* Check that the assembly offsets are valid. */
+        static_assert(AMS_OFFSETOF(KThread, m_thread_context) == THREAD_THREAD_CONTEXT);
+
+        return true;
+    }
+
+    static_assert(KThread::IsKThreadStructurallyValid());
 
     class KScopedDisableDispatch {
         public:

libraries/libmesosphere/include/mesosphere/kern_select_cpu.hpp

@@ -39,6 +39,16 @@
 
     }
 
+#elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+
+    #include <mesosphere/board/qemu/virt/kern_cpu_map.hpp>
+
+    namespace ams::kern::cpu {
+
+        using namespace ams::kern::board::qemu::virt::impl::cpu;
+
+    }
+
 #else
     #error "Unknown board for CPU Map"
 #endif

libraries/libmesosphere/include/mesosphere/kern_select_interrupt_manager.hpp

@@ -41,7 +41,7 @@ namespace ams::kern {
         private:
             u32 m_prev_intr_state;
         public:
-            ALWAYS_INLINE KScopedInterruptDisable() : m_prev_intr_state(KInterruptManager::DisableInterrupts()) { /* ... */ }
+            ALWAYS_INLINE KScopedInterruptDisable() : m_prev_intr_state(KInterruptManager::GetInterruptsEnabledStateAndDisableInterrupts()) { /* ... */ }
             ALWAYS_INLINE ~KScopedInterruptDisable() { KInterruptManager::RestoreInterrupts(m_prev_intr_state); }
     };
 
@@ -51,7 +51,7 @@ namespace ams::kern {
         private:
             u32 m_prev_intr_state;
         public:
-            ALWAYS_INLINE KScopedInterruptEnable() : m_prev_intr_state(KInterruptManager::EnableInterrupts()) { /* ... */ }
+            ALWAYS_INLINE KScopedInterruptEnable() : m_prev_intr_state(KInterruptManager::GetInterruptsEnabledStateAndEnableInterrupts()) { /* ... */ }
             ALWAYS_INLINE ~KScopedInterruptEnable() { KInterruptManager::RestoreInterrupts(m_prev_intr_state); }
     };
 

libraries/libmesosphere/include/mesosphere/kern_select_system_control.hpp

@@ -15,6 +15,7 @@
  */
 #pragma once
 #include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_system_control_base.hpp>
 
 #ifdef ATMOSPHERE_BOARD_NINTENDO_NX
     #include <mesosphere/board/nintendo/nx/kern_k_system_control.hpp>
@@ -23,6 +24,28 @@
         using ams::kern::board::nintendo::nx::KSystemControl;
     }
 
+#elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+    #include <mesosphere/board/qemu/virt/kern_k_system_control.hpp>
+
+    namespace ams::kern {
+        using ams::kern::board::qemu::virt::KSystemControl;
+    }
+
 #else
     #error "Unknown board for KSystemControl"
 #endif
+
+namespace ams::kern {
+
+    ALWAYS_INLINE u32 KSystemControlBase::ReadRegisterPrivileged(ams::svc::PhysicalAddress address) {
+        u32 v;
+        KSystemControl::ReadWriteRegisterPrivileged(std::addressof(v), address, 0x00000000u, 0);
+        return v;
+    }
+
+    ALWAYS_INLINE void KSystemControlBase::WriteRegisterPrivileged(ams::svc::PhysicalAddress address, u32 value) {
+        u32 v;
+        KSystemControl::ReadWriteRegisterPrivileged(std::addressof(v), address, 0xFFFFFFFFu, value);
+    }
+
+}

libraries/libmesosphere/source/arch/arm64/kern_cpu.cpp

@@ -76,7 +76,7 @@ namespace ams::kern::arch::arm64::cpu {
                     } else {
                         m_counter = cpu::GetPerformanceCounter(m_which);
                     }
-                    DataMemoryBarrier();
+                    DataMemoryBarrierInnerShareable();
                     m_done = true;
                     return nullptr;
                 }

libraries/libmesosphere/source/arch/arm64/kern_cpu_asm.s

@@ -61,139 +61,3 @@ _ZN3ams4kern4arch5arm643cpu23SynchronizeAllCoresImplEPii:
 5:
     stlr wzr, [x0]
     ret
-
-
-/* ams::kern::arch::arm64::cpu::ClearPageToZero(void *) */
-.section    .text._ZN3ams4kern4arch5arm643cpu19ClearPageToZeroImplEPv, "ax", %progbits
-.global     _ZN3ams4kern4arch5arm643cpu19ClearPageToZeroImplEPv
-.type       _ZN3ams4kern4arch5arm643cpu19ClearPageToZeroImplEPv, %function
-_ZN3ams4kern4arch5arm643cpu19ClearPageToZeroImplEPv:
-    /* Efficiently clear the page using dc zva. */
-    dc      zva, x0
-    add     x8, x0, #0x040
-    dc      zva, x8
-    add     x8, x0, #0x080
-    dc      zva, x8
-    add     x8, x0, #0x0c0
-    dc      zva, x8
-    add     x8, x0, #0x100
-    dc      zva, x8
-    add     x8, x0, #0x140
-    dc      zva, x8
-    add     x8, x0, #0x180
-    dc      zva, x8
-    add     x8, x0, #0x1c0
-    dc      zva, x8
-    add     x8, x0, #0x200
-    dc      zva, x8
-    add     x8, x0, #0x240
-    dc      zva, x8
-    add     x8, x0, #0x280
-    dc      zva, x8
-    add     x8, x0, #0x2c0
-    dc      zva, x8
-    add     x8, x0, #0x300
-    dc      zva, x8
-    add     x8, x0, #0x340
-    dc      zva, x8
-    add     x8, x0, #0x380
-    dc      zva, x8
-    add     x8, x0, #0x3c0
-    dc      zva, x8
-    add     x8, x0, #0x400
-    dc      zva, x8
-    add     x8, x0, #0x440
-    dc      zva, x8
-    add     x8, x0, #0x480
-    dc      zva, x8
-    add     x8, x0, #0x4c0
-    dc      zva, x8
-    add     x8, x0, #0x500
-    dc      zva, x8
-    add     x8, x0, #0x540
-    dc      zva, x8
-    add     x8, x0, #0x580
-    dc      zva, x8
-    add     x8, x0, #0x5c0
-    dc      zva, x8
-    add     x8, x0, #0x600
-    dc      zva, x8
-    add     x8, x0, #0x640
-    dc      zva, x8
-    add     x8, x0, #0x680
-    dc      zva, x8
-    add     x8, x0, #0x6c0
-    dc      zva, x8
-    add     x8, x0, #0x700
-    dc      zva, x8
-    add     x8, x0, #0x740
-    dc      zva, x8
-    add     x8, x0, #0x780
-    dc      zva, x8
-    add     x8, x0, #0x7c0
-    dc      zva, x8
-    add     x8, x0, #0x800
-    dc      zva, x8
-    add     x8, x0, #0x840
-    dc      zva, x8
-    add     x8, x0, #0x880
-    dc      zva, x8
-    add     x8, x0, #0x8c0
-    dc      zva, x8
-    add     x8, x0, #0x900
-    dc      zva, x8
-    add     x8, x0, #0x940
-    dc      zva, x8
-    add     x8, x0, #0x980
-    dc      zva, x8
-    add     x8, x0, #0x9c0
-    dc      zva, x8
-    add     x8, x0, #0xa00
-    dc      zva, x8
-    add     x8, x0, #0xa40
-    dc      zva, x8
-    add     x8, x0, #0xa80
-    dc      zva, x8
-    add     x8, x0, #0xac0
-    dc      zva, x8
-    add     x8, x0, #0xb00
-    dc      zva, x8
-    add     x8, x0, #0xb40
-    dc      zva, x8
-    add     x8, x0, #0xb80
-    dc      zva, x8
-    add     x8, x0, #0xbc0
-    dc      zva, x8
-    add     x8, x0, #0xc00
-    dc      zva, x8
-    add     x8, x0, #0xc40
-    dc      zva, x8
-    add     x8, x0, #0xc80
-    dc      zva, x8
-    add     x8, x0, #0xcc0
-    dc      zva, x8
-    add     x8, x0, #0xd00
-    dc      zva, x8
-    add     x8, x0, #0xd40
-    dc      zva, x8
-    add     x8, x0, #0xd80
-    dc      zva, x8
-    add     x8, x0, #0xdc0
-    dc      zva, x8
-    add     x8, x0, #0xe00
-    dc      zva, x8
-    add     x8, x0, #0xe40
-    dc      zva, x8
-    add     x8, x0, #0xe80
-    dc      zva, x8
-    add     x8, x0, #0xec0
-    dc      zva, x8
-    add     x8, x0, #0xf00
-    dc      zva, x8
-    add     x8, x0, #0xf40
-    dc      zva, x8
-    add     x8, x0, #0xf80
-    dc      zva, x8
-    add     x8, x0, #0xfc0
-    dc      zva, x8
-    ret

libraries/libmesosphere/source/arch/arm64/kern_exception_handlers.cpp

@@ -225,7 +225,7 @@ namespace ams::kern::arch::arm64 {
             if (AMS_UNLIKELY(GetCurrentThread().IsSingleStep())) {
                 GetCurrentThread().ClearSingleStep();
                 cpu::MonitorDebugSystemControlRegisterAccessor().SetSoftwareStep(false).Store();
-                cpu::EnsureInstructionConsistency();
+                cpu::InstructionMemoryBarrier();
             }
             #endif
 

libraries/libmesosphere/source/arch/arm64/kern_k_interrupt_manager.cpp

@@ -201,7 +201,7 @@ namespace ams::kern::arch::arm64 {
         if (user_mode) {
             KThread *cur_thread = GetCurrentThreadPointer();
             if (cur_thread->IsTerminationRequested()) {
-                KScopedInterruptEnable ei;
+                EnableInterrupts();
                 cur_thread->Exit();
             }
         }
@@ -212,13 +212,14 @@ namespace ams::kern::arch::arm64 {
 
         R_UNLESS(KInterruptController::IsGlobal(irq) || KInterruptController::IsLocal(irq), svc::ResultOutOfRange());
 
-        KScopedInterruptDisable di;
-
         if (KInterruptController::IsGlobal(irq)) {
+            KScopedInterruptDisable di;
             KScopedSpinLock lk(this->GetGlobalInterruptLock());
             return this->BindGlobal(handler, irq, core_id, priority, manual_clear, level);
         } else {
             MESOSPHERE_ASSERT(core_id == GetCurrentCoreId());
+
+            KScopedInterruptDisable di;
             return this->BindLocal(handler, irq, priority, manual_clear);
         }
     }
@@ -228,13 +229,16 @@ namespace ams::kern::arch::arm64 {
 
         R_UNLESS(KInterruptController::IsGlobal(irq) || KInterruptController::IsLocal(irq), svc::ResultOutOfRange());
 
-        KScopedInterruptDisable di;
 
         if (KInterruptController::IsGlobal(irq)) {
+            KScopedInterruptDisable di;
+
             KScopedSpinLock lk(this->GetGlobalInterruptLock());
             return this->UnbindGlobal(irq);
         } else {
             MESOSPHERE_ASSERT(core_id == GetCurrentCoreId());
+
+            KScopedInterruptDisable di;
             return this->UnbindLocal(irq);
         }
     }
@@ -244,13 +248,15 @@ namespace ams::kern::arch::arm64 {
 
         R_UNLESS(KInterruptController::IsGlobal(irq) || KInterruptController::IsLocal(irq), svc::ResultOutOfRange());
 
-        KScopedInterruptDisable di;
 
         if (KInterruptController::IsGlobal(irq)) {
+            KScopedInterruptDisable di;
             KScopedSpinLock lk(this->GetGlobalInterruptLock());
             return this->ClearGlobal(irq);
         } else {
             MESOSPHERE_ASSERT(core_id == GetCurrentCoreId());
+
+            KScopedInterruptDisable di;
             return this->ClearLocal(irq);
         }
     }

libraries/libmesosphere/source/arch/arm64/kern_k_page_table.cpp

@@ -169,10 +169,10 @@ namespace ams::kern::arch::arm64 {
         m_manager = std::addressof(Kernel::GetSystemPageTableManager());
 
         /* Allocate a page for ttbr. */
+        /* NOTE: It is a postcondition of page table manager allocation that the page is all-zero. */
         const u64 asid_tag = (static_cast<u64>(m_asid) << 48ul);
         const KVirtualAddress page = m_manager->Allocate();
         MESOSPHERE_ASSERT(page != Null<KVirtualAddress>);
-        cpu::ClearPageToZero(GetVoidPointer(page));
         m_ttbr = GetInteger(KPageTableBase::GetLinearMappedPhysicalAddress(page)) | asid_tag;
 
         /* Initialize the base page table. */
@@ -1058,7 +1058,7 @@ namespace ams::kern::arch::arm64 {
         auto sw_reserved_bits = PageTableEntry::EncodeSoftwareReservedBits(head_entry->IsHeadMergeDisabled(), head_entry->IsHeadAndBodyMergeDisabled(), tail_entry->IsTailMergeDisabled());
 
         /* Merge! */
-        PteDataSynchronizationBarrier();
+        /* NOTE: As of 13.1.0, Nintendo does not do: PteDataSynchronizationBarrier(); */
         *l1_entry = L1PageTableEntry(PageTableEntry::BlockTag{}, phys_addr, PageTableEntry(entry_template), sw_reserved_bits, false);
 
         /* Note that we updated. */

libraries/libmesosphere/source/arch/arm64/svc/kern_svc_exception_asm.s

@@ -28,7 +28,7 @@ _ZN3ams4kern3svc25CallReturnFromException64Ev:
     stp     x20, x21, [sp, #(EXCEPTION_CONTEXT_X20_X21)]
     stp     x22, x23, [sp, #(EXCEPTION_CONTEXT_X22_X23)]
     stp     x24, x25, [sp, #(EXCEPTION_CONTEXT_X24_X25)]
-    stp     x26, x26, [sp, #(EXCEPTION_CONTEXT_X26_X27)]
+    stp     x26, x27, [sp, #(EXCEPTION_CONTEXT_X26_X27)]
     stp     x28, x29, [sp, #(EXCEPTION_CONTEXT_X28_X29)]
 
     /* Call ams::kern::arch::arm64::ReturnFromException(result). */

libraries/libmesosphere/source/arch/arm64/svc/kern_svc_light_ipc_asm.s

@@ -50,10 +50,6 @@ _ZN3ams4kern3svc26CallSendSyncRequestLight64Ev:
 .global     _ZN3ams4kern3svc32CallSendSyncRequestLight64From32Ev
 .type       _ZN3ams4kern3svc32CallSendSyncRequestLight64From32Ev, %function
 _ZN3ams4kern3svc32CallSendSyncRequestLight64From32Ev:
-    /* Load x4-x7 from where the svc handler stores them. */
-    ldp     x4, x5, [sp, #(8 * 0)]
-    ldp     x6, x7, [sp, #(8 * 2)]
-
     /* Allocate space for the light ipc data. */
     sub     sp, sp, #(4 * 8)
 
@@ -78,13 +74,8 @@ _ZN3ams4kern3svc32CallSendSyncRequestLight64From32Ev:
     /* Free the stack space for the light ipc data. */
     add     sp, sp, #(4 * 8)
 
-    /* Save x4-x7 to where the svc handler stores them. */
-    stp     x4, x5, [sp, #(8 * 0)]
-    stp     x6, x7, [sp, #(8 * 2)]
-
     ret
 
-
 /* ams::kern::svc::CallReplyAndReceiveLight64() */
 .section    .text._ZN3ams4kern3svc26CallReplyAndReceiveLight64Ev, "ax", %progbits
 .global     _ZN3ams4kern3svc26CallReplyAndReceiveLight64Ev
@@ -121,10 +112,6 @@ _ZN3ams4kern3svc26CallReplyAndReceiveLight64Ev:
 .global     _ZN3ams4kern3svc32CallReplyAndReceiveLight64From32Ev
 .type       _ZN3ams4kern3svc32CallReplyAndReceiveLight64From32Ev, %function
 _ZN3ams4kern3svc32CallReplyAndReceiveLight64From32Ev:
-    /* Load x4-x7 from where the svc handler stores them. */
-    ldp     x4, x5, [sp, #(8 * 0)]
-    ldp     x6, x7, [sp, #(8 * 2)]
-
     /* Allocate space for the light ipc data. */
     sub     sp, sp, #(4 * 8)
 
@@ -149,8 +136,4 @@ _ZN3ams4kern3svc32CallReplyAndReceiveLight64From32Ev:
     /* Free the stack space for the light ipc data. */
     add     sp, sp, #(4 * 8)
 
-    /* Save x4-x7 to where the svc handler stores them. */
-    stp     x4, x5, [sp, #(8 * 0)]
-    stp     x6, x7, [sp, #(8 * 2)]
-
     ret

libraries/libmesosphere/source/board/nintendo/nx/kern_k_device_page_table.cpp

@@ -656,9 +656,8 @@ namespace ams::kern::board::nintendo::nx {
         MESOSPHERE_ASSERT(IsValidPhysicalAddress(table_phys_addr));
         Kernel::GetSystemPageTableManager().Open(table_virt_addr, 1);
 
-        /* Clear the page and save it. */
+        /* Save the page. Note that it is a pre-condition that the page is cleared, when allocated from the system page table manager. */
         /* NOTE: Nintendo does not check the result of StoreDataCache. */
-        cpu::ClearPageToZero(GetVoidPointer(table_virt_addr));
         cpu::StoreDataCache(GetVoidPointer(table_virt_addr), PageDirectorySize);
         g_reserved_table_phys_addr = table_phys_addr;
 
@@ -1130,7 +1129,7 @@ namespace ams::kern::board::nintendo::nx {
             size_t cur_size;
             {
                 /* Get the current contiguous range. */
-                KPageTableBase::MemoryRange contig_range = {};
+                KPageTableBase::MemoryRange contig_range = { .address = Null<KPhysicalAddress>, .size = 0 };
                 R_TRY(page_table->OpenMemoryRangeForMapDeviceAddressSpace(std::addressof(contig_range), process_address + mapped_size, size - mapped_size, ConvertToKMemoryPermission(device_perm), is_aligned));
 
                 /* Ensure we close the range when we're done. */
@@ -1288,7 +1287,7 @@ namespace ams::kern::board::nintendo::nx {
         MESOSPHERE_ASSERT(((device_address + size - 1) & ~DeviceVirtualAddressMask) == 0);
 
         /* We need to traverse the ranges that make up our mapping, to make sure they're all good. Start by getting a contiguous range. */
-        KPageTableBase::MemoryRange contig_range = {};
+        KPageTableBase::MemoryRange contig_range = { .address = Null<KPhysicalAddress>, .size = 0 };
         if (R_FAILED(page_table->OpenMemoryRangeForUnmapDeviceAddressSpace(std::addressof(contig_range), process_address, size))) {
             return false;
         }

libraries/libmesosphere/source/board/nintendo/nx/kern_k_sleep_manager.cpp

@@ -73,7 +73,7 @@ namespace ams::kern::board::nintendo::nx {
 
         void PowerOnCpu(int core_id, KPhysicalAddress entry_phys_addr, u64 context_id) {
             /* Request the secure monitor power on the core. */
-            smc::CpuOn(cpu::MultiprocessorAffinityRegisterAccessor().GetCpuOnArgument() | core_id, GetInteger(entry_phys_addr), context_id);
+            ::ams::kern::arch::arm64::smc::CpuOn<smc::SmcId_Supervisor, true>(cpu::MultiprocessorAffinityRegisterAccessor().GetCpuOnArgument() | core_id, GetInteger(entry_phys_addr), context_id);
         }
 
         void WaitOtherCpuPowerOff() {
@@ -341,7 +341,9 @@ namespace ams::kern::board::nintendo::nx {
 
             /* Restore pmu registers. */
             cpu::SetPmUserEnrEl0(0);
-            cpu::PerformanceMonitorsControlRegisterAccessor().SetEventCounterReset(true).SetCycleCounterReset(true).Store();
+            cpu::PerformanceMonitorsControlRegisterAccessor(0).SetEventCounterReset(true).SetCycleCounterReset(true).Store();
+            cpu::EnsureInstructionConsistency();
+
             cpu::SetPmOvsClrEl0(static_cast<u64>(static_cast<u32>(~u32())));
             cpu::SetPmIntEnClrEl1(static_cast<u64>(static_cast<u32>(~u32())));
             cpu::SetPmCntEnClrEl0(static_cast<u64>(static_cast<u32>(~u32())));

libraries/libmesosphere/source/board/nintendo/nx/kern_k_system_control.cpp

@@ -21,8 +21,7 @@ namespace ams::kern::board::nintendo::nx {
 
     namespace {
 
-        constexpr uintptr_t DramPhysicalAddress = 0x80000000;
-        constexpr size_t SecureAlignment        = 128_KB;
+        constexpr size_t SecureAlignment = 128_KB;
 
         /* Global variables for panic. */
         constinit bool g_call_smc_on_panic;
@@ -38,22 +37,6 @@ namespace ams::kern::board::nintendo::nx {
         constinit KPhysicalAddress g_secure_region_phys_addr = Null<KPhysicalAddress>;
         constinit size_t g_secure_region_size = 0;
 
-        /* Global variables for randomness. */
-        /* Nintendo uses std::mt19937_t for randomness. */
-        /* To save space (and because mt19337_t isn't secure anyway), */
-        /* We will use TinyMT. */
-        constinit bool         g_initialized_random_generator;
-        constinit util::TinyMT g_random_generator{util::ConstantInitialize};
-        constinit KSpinLock    g_random_lock;
-
-        ALWAYS_INLINE size_t GetRealMemorySizeForInit() {
-            /* TODO: Move this into a header for the MC in general. */
-            constexpr u32 MemoryControllerConfigurationRegister = 0x70019050;
-            u32 config_value;
-            MESOSPHERE_INIT_ABORT_UNLESS(smc::init::ReadWriteRegister(&config_value, MemoryControllerConfigurationRegister, 0, 0));
-            return static_cast<size_t>(config_value & 0x3FFF) << 20;
-        }
-
         ALWAYS_INLINE util::BitPack32 GetKernelConfigurationForInit() {
             u64 value = 0;
             smc::init::GetConfig(&value, 1, smc::ConfigItem::KernelConfiguration);
@@ -86,7 +69,7 @@ namespace ams::kern::board::nintendo::nx {
 
         ALWAYS_INLINE u64 GenerateRandomU64ForInit() {
             u64 value;
-            smc::init::GenerateRandomBytes(&value, sizeof(value));
+            smc::init::GenerateRandomBytes(std::addressof(value), sizeof(value));
             return value;
         }
 
@@ -96,27 +79,6 @@ namespace ams::kern::board::nintendo::nx {
             return value;
         }
 
-        ALWAYS_INLINE u64 GenerateRandomU64FromGenerator() {
-            return g_random_generator.GenerateRandomU64();
-        }
-
-        template<typename F>
-        ALWAYS_INLINE u64 GenerateUniformRange(u64 min, u64 max, F f) {
-            /* Handle the case where the difference is too large to represent. */
-            if (max == std::numeric_limits<u64>::max() && min == std::numeric_limits<u64>::min()) {
-                return f();
-            }
-
-            /* Iterate until we get a value in range. */
-            const u64 range_size    = ((max + 1) - min);
-            const u64 effective_max = (std::numeric_limits<u64>::max() / range_size) * range_size;
-            while (true) {
-                if (const u64 rnd = f(); rnd < effective_max) {
-                    return min + (rnd % range_size);
-                }
-            }
-        }
-
         ALWAYS_INLINE u64 GetConfigU64(smc::ConfigItem which) {
             u64 value;
             smc::GetConfig(&value, 1, which);
@@ -324,6 +286,14 @@ namespace ams::kern::board::nintendo::nx {
     }
 
     /* Initialization. */
+    size_t KSystemControl::Init::GetRealMemorySize() {
+        /* TODO: Move this into a header for the MC in general. */
+        constexpr u32 MemoryControllerConfigurationRegister = 0x70019050;
+        u32 config_value;
+        MESOSPHERE_INIT_ABORT_UNLESS(smc::init::ReadWriteRegister(&config_value, MemoryControllerConfigurationRegister, 0, 0));
+        return static_cast<size_t>(config_value & 0x3FFF) << 20;
+    }
+
     size_t KSystemControl::Init::GetIntendedMemorySize() {
         switch (GetKernelConfigurationForInit().Get<smc::KernelConfiguration::MemorySize>()) {
             case smc::MemorySize_4GB:
@@ -336,23 +306,6 @@ namespace ams::kern::board::nintendo::nx {
         }
     }
 
-    KPhysicalAddress KSystemControl::Init::GetKernelPhysicalBaseAddress(uintptr_t base_address) {
-        const size_t real_dram_size     = GetRealMemorySizeForInit();
-        const size_t intended_dram_size = KSystemControl::Init::GetIntendedMemorySize();
-        if (intended_dram_size * 2 < real_dram_size) {
-            return base_address;
-        } else {
-            return base_address + ((real_dram_size - intended_dram_size) / 2);
-        }
-    }
-
-    void KSystemControl::Init::GetInitialProcessBinaryLayout(InitialProcessBinaryLayout *out) {
-        *out = {
-            .address = GetInteger(GetKernelPhysicalBaseAddress(DramPhysicalAddress)) + GetIntendedMemorySize() - KTraceBufferSize - InitialProcessBinarySizeMax,
-            ._08     = 0,
-        };
-    }
-
     bool KSystemControl::Init::ShouldIncreaseThreadResourceLimit() {
         return GetKernelConfigurationForInit().Get<smc::KernelConfiguration::IncreaseThreadResourceLimit>();
     }
@@ -424,17 +377,17 @@ namespace ams::kern::board::nintendo::nx {
     }
 
     void KSystemControl::Init::CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg) {
-        smc::init::CpuOn(core_id, entrypoint, arg);
+        MESOSPHERE_INIT_ABORT_UNLESS((::ams::kern::arch::arm64::smc::CpuOn<smc::SmcId_Supervisor, false>(core_id, entrypoint, arg)) == 0);
     }
 
     /* Randomness for Initialization. */
-    void KSystemControl::Init::GenerateRandomBytes(void *dst, size_t size) {
-        MESOSPHERE_INIT_ABORT_UNLESS(size <= 0x38);
-        smc::init::GenerateRandomBytes(dst, size);
+    void KSystemControl::Init::GenerateRandom(u64 *dst, size_t count) {
+        MESOSPHERE_INIT_ABORT_UNLESS(count <= 7);
+        smc::init::GenerateRandomBytes(dst, count * sizeof(u64));
     }
 
     u64 KSystemControl::Init::GenerateRandomRange(u64 min, u64 max) {
-        return GenerateUniformRange(min, max, GenerateRandomU64ForInit);
+        return KSystemControlBase::GenerateUniformRange(min, max, GenerateRandomU64ForInit);
     }
 
     /* System Initialization. */
@@ -443,8 +396,8 @@ namespace ams::kern::board::nintendo::nx {
         {
             u64 seed;
             smc::GenerateRandomBytes(std::addressof(seed), sizeof(seed));
-            g_random_generator.Initialize(reinterpret_cast<u32*>(std::addressof(seed)), sizeof(seed) / sizeof(u32));
-            g_initialized_random_generator = true;
+            s_random_generator.Initialize(reinterpret_cast<const u32*>(std::addressof(seed)), sizeof(seed) / sizeof(u32));
+            s_initialized_random_generator = true;
         }
 
         /* Set IsDebugMode. */
@@ -483,25 +436,8 @@ namespace ams::kern::board::nintendo::nx {
             smc::ConfigureCarveout(0, carveout.GetAddress(), carveout.GetSize());
         }
 
-        /* System ResourceLimit initialization. */
-        {
-            /* Construct the resource limit object. */
-            KResourceLimit &sys_res_limit = Kernel::GetSystemResourceLimit();
-            KAutoObject::Create<KResourceLimit>(std::addressof(sys_res_limit));
-            sys_res_limit.Initialize();
-
-            /* Set the initial limits. */
-            const auto [total_memory_size, kernel_memory_size] = KMemoryLayout::GetTotalAndKernelMemorySizes();
-            const auto &slab_counts = init::GetSlabResourceCounts();
-            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_PhysicalMemoryMax,      total_memory_size));
-            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_ThreadCountMax,         slab_counts.num_KThread));
-            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_EventCountMax,          slab_counts.num_KEvent));
-            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_TransferMemoryCountMax, slab_counts.num_KTransferMemory));
-            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_SessionCountMax,        slab_counts.num_KSession));
-
-            /* Reserve system memory. */
-            MESOSPHERE_ABORT_UNLESS(sys_res_limit.Reserve(ams::svc::LimitableResource_PhysicalMemoryMax, kernel_memory_size));
-        }
+        /* Initialize the system resource limit (and potentially other things). */
+        KSystemControlBase::InitializePhase1(true);
     }
 
     void KSystemControl::InitializePhase2() {
@@ -520,11 +456,8 @@ namespace ams::kern::board::nintendo::nx {
             g_secure_applet_memory_address = KMemoryLayout::GetLinearVirtualAddress(secure_applet_memory_phys_addr);
         }
 
-        /* Initialize KTrace. */
-        if constexpr (IsKTraceEnabled) {
-            const auto &ktrace = KMemoryLayout::GetKernelTraceBufferRegion();
-            KTrace::Initialize(ktrace.GetAddress(), ktrace.GetSize());
-        }
+        /* Initialize KTrace (and potentially other init). */
+        KSystemControlBase::InitializePhase2();
     }
 
     u32 KSystemControl::GetCreateProcessMemoryPool() {
@@ -546,29 +479,29 @@ namespace ams::kern::board::nintendo::nx {
     }
 
     /* Randomness. */
-    void KSystemControl::GenerateRandomBytes(void *dst, size_t size) {
-        MESOSPHERE_INIT_ABORT_UNLESS(size <= 0x38);
-        smc::GenerateRandomBytes(dst, size);
+    void KSystemControl::GenerateRandom(u64 *dst, size_t count) {
+        MESOSPHERE_INIT_ABORT_UNLESS(count <= 7);
+        smc::GenerateRandomBytes(dst, count * sizeof(u64));
     }
 
     u64 KSystemControl::GenerateRandomRange(u64 min, u64 max) {
         KScopedInterruptDisable intr_disable;
-        KScopedSpinLock lk(g_random_lock);
+        KScopedSpinLock lk(s_random_lock);
 
 
-        if (AMS_LIKELY(g_initialized_random_generator)) {
-            return GenerateUniformRange(min, max, GenerateRandomU64FromGenerator);
+        if (AMS_LIKELY(s_initialized_random_generator)) {
+            return KSystemControlBase::GenerateUniformRange(min, max, [] ALWAYS_INLINE_LAMBDA () -> u64 { return s_random_generator.GenerateRandomU64(); });
         } else {
-            return GenerateUniformRange(min, max, GenerateRandomU64FromSmc);
+            return KSystemControlBase::GenerateUniformRange(min, max, GenerateRandomU64FromSmc);
         }
     }
 
     u64 KSystemControl::GenerateRandomU64() {
         KScopedInterruptDisable intr_disable;
-        KScopedSpinLock lk(g_random_lock);
+        KScopedSpinLock lk(s_random_lock);
 
-        if (AMS_LIKELY(g_initialized_random_generator)) {
-            return GenerateRandomU64FromGenerator();
+        if (AMS_LIKELY(s_initialized_random_generator)) {
+            return s_random_generator.GenerateRandomU64();
         } else {
             return GenerateRandomU64FromSmc();
         }
@@ -672,52 +605,18 @@ namespace ams::kern::board::nintendo::nx {
     }
 
     /* User access. */
-    void KSystemControl::CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args) {
-        /* Get the function id for the current call. */
-        u64 function_id = args->r[0];
-
-        /* We'll need to map in pages if arguments are pointers. Prepare page groups to do so. */
-        auto &page_table = GetCurrentProcess().GetPageTable();
-        auto *bim = page_table.GetBlockInfoManager();
-
-        constexpr size_t MaxMappedRegisters = 7;
-        std::array<KPageGroup, MaxMappedRegisters> page_groups = { KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), };
-
-        for (size_t i = 0; i < MaxMappedRegisters; i++) {
-            const size_t reg_id = i + 1;
-            if (function_id & (1ul << (8 + reg_id))) {
-                /* Create and open a new page group for the address. */
-                KVirtualAddress virt_addr = args->r[reg_id];
-
-                if (R_SUCCEEDED(page_table.MakeAndOpenPageGroup(std::addressof(page_groups[i]), util::AlignDown(GetInteger(virt_addr), PageSize), 1, KMemoryState_None, KMemoryState_None, KMemoryPermission_UserReadWrite, KMemoryPermission_UserReadWrite, KMemoryAttribute_None, KMemoryAttribute_None))) {
-                    /* Translate the virtual address to a physical address. */
-                    const auto it = page_groups[i].begin();
-                    MESOSPHERE_ASSERT(it != page_groups[i].end());
-                    MESOSPHERE_ASSERT(it->GetNumPages() == 1);
-
-                    args->r[reg_id] = GetInteger(it->GetAddress()) | (GetInteger(virt_addr) & (PageSize - 1));
-                } else {
-                    /* If we couldn't map, we should clear the address. */
-                    args->r[reg_id] = 0;
-                }
-            }
-        }
-
+    void KSystemControl::CallSecureMonitorFromUserImpl(ams::svc::lp64::SecureMonitorArguments *args) {
         /* Invoke the secure monitor. */
-        smc::CallSecureMonitorFromUser(args);
-
-        /* Make sure that we close any pages that we opened. */
-        for (size_t i = 0; i < MaxMappedRegisters; i++) {
-            page_groups[i].Close();
-        }
+        return smc::CallSecureMonitorFromUser(args);
     }
 
     /* Secure Memory. */
     size_t KSystemControl::CalculateRequiredSecureMemorySize(size_t size, u32 pool) {
         if (pool == KMemoryManager::Pool_Applet) {
             return 0;
+        } else {
+            return KSystemControlBase::CalculateRequiredSecureMemorySize(size, pool);
         }
-        return size;
     }
 
     Result KSystemControl::AllocateSecureMemory(KVirtualAddress *out, size_t size, u32 pool) {

libraries/libmesosphere/source/board/nintendo/nx/kern_secure_monitor.cpp

@@ -20,18 +20,27 @@ namespace ams::kern::board::nintendo::nx::smc {
 
     namespace {
 
-        struct SecureMonitorArguments {
-            u64 x[8];
-        };
-
         enum UserFunctionId : u32 {
-            UserFunctionId_SetConfig = 0xC3000401,
+            UserFunctionId_SetConfig                       = 0xC3000401,
+            UserFunctionId_GetConfigUser                   = 0xC3000002,
+            UserFunctionId_GetResult                       = 0xC3000003,
+            UserFunctionId_GetResultData                   = 0xC3000404,
+            UserFunctionId_ModularExponentiate             = 0xC3000E05,
+            UserFunctionId_GenerateRandomBytes             = 0xC3000006,
+            UserFunctionId_GenerateAesKek                  = 0xC3000007,
+            UserFunctionId_LoadAesKey                      = 0xC3000008,
+            UserFunctionId_ComputeAes                      = 0xC3000009,
+            UserFunctionId_GenerateSpecificAesKey          = 0xC300000A,
+            UserFunctionId_ComputeCmac                     = 0xC300040B,
+            UserFunctionId_ReencryptDeviceUniqueData       = 0xC300D60C,
+            UserFunctionId_DecryptDeviceUniqueData         = 0xC300100D,
+            UserFunctionId_ModularExponentiateByStorageKey = 0xC300060F,
+            UserFunctionId_PrepareEsDeviceUniqueKey        = 0xC3000610,
+            UserFunctionId_LoadPreparedAesKey              = 0xC3000011,
+            UserFunctionId_PrepareEsCommonTitleKey         = 0xC3000012,
         };
 
         enum FunctionId : u32 {
-            FunctionId_CpuSuspend          = 0xC4000001,
-            FunctionId_CpuOff              = 0x84000002,
-            FunctionId_CpuOn               = 0xC4000003,
             FunctionId_GetConfig           = 0xC3000004,
             FunctionId_GenerateRandomBytes = 0xC3000005,
             FunctionId_Panic               = 0xC3000006,
@@ -42,171 +51,60 @@ namespace ams::kern::board::nintendo::nx::smc {
             FunctionId_SetConfig           = 0xC3000409,
         };
 
-        void CallPrivilegedSecureMonitorFunction(SecureMonitorArguments &args) {
-            /* Load arguments into registers. */
-            register u64 x0 asm("x0") = args.x[0];
-            register u64 x1 asm("x1") = args.x[1];
-            register u64 x2 asm("x2") = args.x[2];
-            register u64 x3 asm("x3") = args.x[3];
-            register u64 x4 asm("x4") = args.x[4];
-            register u64 x5 asm("x5") = args.x[5];
-            register u64 x6 asm("x6") = args.x[6];
-            register u64 x7 asm("x7") = args.x[7];
-
-            /* Actually make the call. */
-            {
-                /* Disable interrupts while making the call. */
-                KScopedInterruptDisable intr_disable;
-
-                {
-                    /* Backup the current thread pointer. */
-                    const uintptr_t current_thread_pointer_value = cpu::GetCurrentThreadPointerValue();
-
-                    __asm__ __volatile__("smc #1"
-                                        : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
-                                        :
-                                        : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
-                                        );
-
-                    /* Restore the current thread pointer into X18. */
-                    cpu::SetCurrentThreadPointerValue(current_thread_pointer_value);
-
-                    /* Store arguments to output. */
-                    args.x[0] = x0;
-                    args.x[1] = x1;
-                    args.x[2] = x2;
-                    args.x[3] = x3;
-                    args.x[4] = x4;
-                    args.x[5] = x5;
-                    args.x[6] = x6;
-                    args.x[7] = x7;
-                }
-            }
-        }
-
-        void CallUserSecureMonitorFunction(ams::svc::lp64::SecureMonitorArguments *args) {
-            /* Load arguments into registers. */
-            register u64 x0 asm("x0") = args->r[0];
-            register u64 x1 asm("x1") = args->r[1];
-            register u64 x2 asm("x2") = args->r[2];
-            register u64 x3 asm("x3") = args->r[3];
-            register u64 x4 asm("x4") = args->r[4];
-            register u64 x5 asm("x5") = args->r[5];
-            register u64 x6 asm("x6") = args->r[6];
-            register u64 x7 asm("x7") = args->r[7];
-
-            /* Actually make the call. */
-            {
-                /* Disable interrupts while making the call. */
-                KScopedInterruptDisable intr_disable;
-
-                {
-                    /* Backup the current thread pointer. */
-                    const uintptr_t current_thread_pointer_value = cpu::GetCurrentThreadPointerValue();
-
-                    __asm__ __volatile__("smc #0"
-                                        : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
-                                        :
-                                        : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
-                                        );
-
-                    /* Restore the current thread pointer into X18. */
-                    cpu::SetCurrentThreadPointerValue(current_thread_pointer_value);
-
-                    /* Store arguments to output. */
-                    args->r[0] = x0;
-                    args->r[1] = x1;
-                    args->r[2] = x2;
-                    args->r[3] = x3;
-                    args->r[4] = x4;
-                    args->r[5] = x5;
-                    args->r[6] = x6;
-                    args->r[7] = x7;
-                }
-            }
-        }
-
-        void CallPrivilegedSecureMonitorFunctionForInit(SecureMonitorArguments &args) {
-            /* Load arguments into registers. */
-            register u64 x0 asm("x0") = args.x[0];
-            register u64 x1 asm("x1") = args.x[1];
-            register u64 x2 asm("x2") = args.x[2];
-            register u64 x3 asm("x3") = args.x[3];
-            register u64 x4 asm("x4") = args.x[4];
-            register u64 x5 asm("x5") = args.x[5];
-            register u64 x6 asm("x6") = args.x[6];
-            register u64 x7 asm("x7") = args.x[7];
-
-            /* Actually make the call. */
-            __asm__ __volatile__("smc #1"
-                                : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
-                                :
-                                : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
-                                );
-
-            /* Store arguments to output. */
-            args.x[0] = x0;
-            args.x[1] = x1;
-            args.x[2] = x2;
-            args.x[3] = x3;
-            args.x[4] = x4;
-            args.x[5] = x5;
-            args.x[6] = x6;
-            args.x[7] = x7;
-        }
-
         /* Global lock for generate random bytes. */
-        KSpinLock g_generate_random_lock;
+        constinit KSpinLock g_generate_random_lock;
 
     }
 
     /* SMC functionality needed for init. */
     namespace init {
 
-        void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg) {
-            SecureMonitorArguments args = { FunctionId_CpuOn, core_id, entrypoint, arg };
-            CallPrivilegedSecureMonitorFunctionForInit(args);
-        }
-
         void GetConfig(u64 *out, size_t num_qwords, ConfigItem config_item) {
-            SecureMonitorArguments args = { FunctionId_GetConfig, static_cast<u32>(config_item) };
-            CallPrivilegedSecureMonitorFunctionForInit(args);
-            MESOSPHERE_INIT_ABORT_UNLESS((static_cast<SmcResult>(args.x[0]) == SmcResult::Success));
+            ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_GetConfig, static_cast<u32>(config_item) } };
+
+            ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, false>(args.r);
+            MESOSPHERE_INIT_ABORT_UNLESS((static_cast<SmcResult>(args.r[0]) == SmcResult::Success));
+
             for (size_t i = 0; i < num_qwords && i < 7; i++) {
-                out[i] = args.x[1 + i];
+                out[i] = args.r[1 + i];
             }
         }
 
         void GenerateRandomBytes(void *dst, size_t size) {
             /* Call SmcGenerateRandomBytes() */
-            SecureMonitorArguments args = { FunctionId_GenerateRandomBytes, size };
-            MESOSPHERE_INIT_ABORT_UNLESS(size <= sizeof(args) - sizeof(args.x[0]));
+            ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_GenerateRandomBytes, size } };
+            MESOSPHERE_INIT_ABORT_UNLESS(size <= sizeof(args) - sizeof(args.r[0]));
 
-            CallPrivilegedSecureMonitorFunctionForInit(args);
-            MESOSPHERE_INIT_ABORT_UNLESS((static_cast<SmcResult>(args.x[0]) == SmcResult::Success));
+            ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, false>(args.r);
+            MESOSPHERE_INIT_ABORT_UNLESS((static_cast<SmcResult>(args.r[0]) == SmcResult::Success));
 
             /* Copy output. */
-            std::memcpy(dst, std::addressof(args.x[1]), size);
+            std::memcpy(dst, std::addressof(args.r[1]), size);
         }
 
         bool ReadWriteRegister(u32 *out, u64 address, u32 mask, u32 value) {
-            SecureMonitorArguments args = { FunctionId_ReadWriteRegister, address, mask, value };
-            CallPrivilegedSecureMonitorFunctionForInit(args);
-            *out = args.x[1];
-            return static_cast<SmcResult>(args.x[0]) == SmcResult::Success;
+            ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_ReadWriteRegister, address, mask, value } };
+
+            ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, false>(args.r);
+            MESOSPHERE_INIT_ABORT_UNLESS((static_cast<SmcResult>(args.r[0]) == SmcResult::Success));
+
+            *out = args.r[1];
+
+            return static_cast<SmcResult>(args.r[0]) == SmcResult::Success;
         }
 
     }
 
     bool TryGetConfig(u64 *out, size_t num_qwords, ConfigItem config_item) {
-        SecureMonitorArguments args = { FunctionId_GetConfig, static_cast<u32>(config_item) };
-        CallPrivilegedSecureMonitorFunction(args);
-        if (static_cast<SmcResult>(args.x[0]) != SmcResult::Success) {
+        ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_GetConfig, static_cast<u32>(config_item) } };
+
+        ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, true>(args.r);
+        if (AMS_UNLIKELY(static_cast<SmcResult>(args.r[0]) != SmcResult::Success)) {
             return false;
         }
 
         for (size_t i = 0; i < num_qwords && i < 7; i++) {
-            out[i] = args.x[1 + i];
+            out[i] = args.r[1 + i];
         }
 
         return true;
@@ -217,55 +115,58 @@ namespace ams::kern::board::nintendo::nx::smc {
     }
 
     bool SetConfig(ConfigItem config_item, u64 value) {
-        SecureMonitorArguments args = { FunctionId_SetConfig, static_cast<u32>(config_item), 0, value };
-        CallPrivilegedSecureMonitorFunction(args);
-        return static_cast<SmcResult>(args.x[0]) == SmcResult::Success;
+        ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_SetConfig, static_cast<u32>(config_item), 0, value } };
+
+        ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, true>(args.r);
+
+        return static_cast<SmcResult>(args.r[0]) == SmcResult::Success;
     }
 
     bool ReadWriteRegister(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value) {
-        SecureMonitorArguments args = { FunctionId_ReadWriteRegister, address, mask, value };
-        CallPrivilegedSecureMonitorFunction(args);
-        *out = static_cast<u32>(args.x[1]);
-        return static_cast<SmcResult>(args.x[0]) == SmcResult::Success;
+        ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_ReadWriteRegister, address, mask, value } };
+
+        ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, true>(args.r);
+
+        *out = static_cast<u32>(args.r[1]);
+        return static_cast<SmcResult>(args.r[0]) == SmcResult::Success;
     }
 
     void ConfigureCarveout(size_t which, uintptr_t address, size_t size) {
-        SecureMonitorArguments args = { FunctionId_ConfigureCarveout, static_cast<u64>(which), static_cast<u64>(address), static_cast<u64>(size) };
-        CallPrivilegedSecureMonitorFunction(args);
-        MESOSPHERE_ABORT_UNLESS((static_cast<SmcResult>(args.x[0]) == SmcResult::Success));
-    }
+        ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_ConfigureCarveout, static_cast<u64>(which), static_cast<u64>(address), static_cast<u64>(size) } };
 
-    void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg) {
-        SecureMonitorArguments args = { FunctionId_CpuOn, core_id, static_cast<u64>(entrypoint), static_cast<u64>(arg) };
-        CallPrivilegedSecureMonitorFunction(args);
-        MESOSPHERE_ABORT_UNLESS((static_cast<SmcResult>(args.x[0]) == SmcResult::Success));
+        ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, true>(args.r);
+
+        MESOSPHERE_ABORT_UNLESS((static_cast<SmcResult>(args.r[0]) == SmcResult::Success));
     }
 
     void GenerateRandomBytes(void *dst, size_t size) {
         /* Setup for call. */
-        SecureMonitorArguments args = { FunctionId_GenerateRandomBytes, size };
-        MESOSPHERE_ABORT_UNLESS(size <= sizeof(args) - sizeof(args.x[0]));
+        ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_GenerateRandomBytes, size } };
+        MESOSPHERE_ABORT_UNLESS(size <= sizeof(args) - sizeof(args.r[0]));
 
         /* Make call. */
         {
             KScopedInterruptDisable intr_disable;
             KScopedSpinLock lk(g_generate_random_lock);
-            CallPrivilegedSecureMonitorFunction(args);
+
+            ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, true>(args.r);
         }
-        MESOSPHERE_ABORT_UNLESS((static_cast<SmcResult>(args.x[0]) == SmcResult::Success));
+        MESOSPHERE_ABORT_UNLESS((static_cast<SmcResult>(args.r[0]) == SmcResult::Success));
 
         /* Copy output. */
-        std::memcpy(dst, std::addressof(args.x[1]), size);
+        std::memcpy(dst, std::addressof(args.r[1]), size);
     }
 
     void NORETURN Panic(u32 color) {
-        SecureMonitorArguments args = { FunctionId_Panic, color };
-        CallPrivilegedSecureMonitorFunction(args);
+        ams::svc::lp64::SecureMonitorArguments args = { { FunctionId_Panic, color } };
+
+        ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_Supervisor, true>(args.r);
+
         AMS_INFINITE_LOOP();
     }
 
     void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args) {
-        CallUserSecureMonitorFunction(args);
+        ::ams::kern::arch::arm64::smc::SecureMonitorCall<SmcId_User, true>(args->r);
     }
 
 }

libraries/libmesosphere/source/board/nintendo/nx/kern_secure_monitor.hpp

@@ -15,10 +15,16 @@
  */
 #pragma once
 #include <mesosphere.hpp>
+#include <mesosphere/arch/arm64/kern_secure_monitor_base.hpp>
 
 namespace ams::kern::board::nintendo::nx::smc {
 
     /* Types. */
+    enum SmcId {
+        SmcId_User       = 0,
+        SmcId_Supervisor = 1,
+    };
+
     enum MemorySize {
         MemorySize_4GB = 0,
         MemorySize_6GB = 1,
@@ -105,15 +111,12 @@ namespace ams::kern::board::nintendo::nx::smc {
 
     bool SetConfig(ConfigItem config_item, u64 value);
 
-    void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg);
-
     void NORETURN Panic(u32 color);
 
     void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
 
     namespace init {
 
-        void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg);
         void GetConfig(u64 *out, size_t num_qwords, ConfigItem config_item);
         void GenerateRandomBytes(void *dst, size_t size);
         bool ReadWriteRegister(u32 *out, u64 address, u32 mask, u32 value);

libraries/libmesosphere/source/board/qemu/virt/kern_k_system_control.cpp

@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <mesosphere.hpp>
+#include "kern_secure_monitor.hpp"
+
+namespace ams::kern::board::qemu::virt {
+
+    /* User access. */
+    void KSystemControl::CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args) {
+        /* Invoke the secure monitor. */
+        return smc::CallSecureMonitorFromUser(args);
+    }
+
+}

libraries/libmesosphere/source/board/qemu/virt/kern_secure_monitor.cpp

@@ -0,0 +1,71 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <mesosphere.hpp>
+#include "kern_secure_monitor.hpp"
+
+namespace ams::kern::board::qemu::virt::smc {
+
+    namespace {
+
+        enum UserFunctionId : u32 {
+            UserFunctionId_SetConfig                       = 0xC3000401,
+            UserFunctionId_GetConfig                       = 0xC3000002,
+            UserFunctionId_GetResult                       = 0xC3000003,
+            UserFunctionId_GetResultData                   = 0xC3000404,
+            UserFunctionId_ModularExponentiate             = 0xC3000E05,
+            UserFunctionId_GenerateRandomBytes             = 0xC3000006,
+            UserFunctionId_GenerateAesKek                  = 0xC3000007,
+            UserFunctionId_LoadAesKey                      = 0xC3000008,
+            UserFunctionId_ComputeAes                      = 0xC3000009,
+            UserFunctionId_GenerateSpecificAesKey          = 0xC300000A,
+            UserFunctionId_ComputeCmac                     = 0xC300040B,
+            UserFunctionId_ReencryptDeviceUniqueData       = 0xC300D60C,
+            UserFunctionId_DecryptDeviceUniqueData         = 0xC300100D,
+            UserFunctionId_ModularExponentiateByStorageKey = 0xC300060F,
+            UserFunctionId_PrepareEsDeviceUniqueKey        = 0xC3000610,
+            UserFunctionId_LoadPreparedAesKey              = 0xC3000011,
+            UserFunctionId_PrepareEsCommonTitleKey         = 0xC3000012,
+        };
+
+    }
+
+    void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args) {
+        MESOSPHERE_LOG("Received SMC [%p %p %p %p %p %p %p %p] from %s\n", reinterpret_cast<void *>(args->r[0]), reinterpret_cast<void *>(args->r[1]), reinterpret_cast<void *>(args->r[2]), reinterpret_cast<void *>(args->r[3]), reinterpret_cast<void *>(args->r[4]), reinterpret_cast<void *>(args->r[5]), reinterpret_cast<void *>(args->r[6]), reinterpret_cast<void *>(args->r[7]), GetCurrentProcess().GetName());
+
+        switch (args->r[0]) {
+            case UserFunctionId_GetConfig:
+                {
+                    switch (static_cast<ConfigItem>(args->r[1])) {
+                        case ConfigItem::ExosphereApiVersion:
+                            args->r[1] = (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MAJOR & 0xFF) << 56) |
+                                         (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MINOR & 0xFF) << 48) |
+                                         (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MICRO & 0xFF) << 40) |
+                                         (static_cast<u64>(13)                                      << 32) |
+                                         (static_cast<u64>(GetTargetFirmware())                     <<  0);
+                            break;
+                        default:
+                            MESOSPHERE_PANIC("Unhandled GetConfig\n");
+                    }
+
+                    args->r[0] = static_cast<u64>(SmcResult::Success);
+                }
+                break;
+            default:
+                MESOSPHERE_PANIC("Unhandled SMC [%p %p %p %p %p %p %p %p]", reinterpret_cast<void *>(args->r[0]), reinterpret_cast<void *>(args->r[1]), reinterpret_cast<void *>(args->r[2]), reinterpret_cast<void *>(args->r[3]), reinterpret_cast<void *>(args->r[4]), reinterpret_cast<void *>(args->r[5]), reinterpret_cast<void *>(args->r[6]), reinterpret_cast<void *>(args->r[7]));
+        }
+    }
+
+}

libraries/libmesosphere/source/board/qemu/virt/kern_secure_monitor.hpp

@@ -0,0 +1,68 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere.hpp>
+
+namespace ams::kern::board::qemu::virt::smc {
+
+    enum class ConfigItem : u32 {
+        /* Standard config items. */
+        DisableProgramVerification  = 1,
+        DramId                      = 2,
+        SecurityEngineIrqNumber     = 3,
+        Version                     = 4,
+        HardwareType                = 5,
+        IsRetail                    = 6,
+        IsRecoveryBoot              = 7,
+        DeviceId                    = 8,
+        BootReason                  = 9,
+        MemoryMode                  = 10,
+        IsDebugMode                 = 11,
+        KernelConfiguration         = 12,
+        IsChargerHiZModeEnabled     = 13,
+        IsQuest                     = 14,
+        RegulatorType               = 15,
+        DeviceUniqueKeyGeneration   = 16,
+        Package2Hash                = 17,
+
+        /* Extension config items for exosphere. */
+        ExosphereApiVersion          = 65000,
+        ExosphereNeedsReboot         = 65001,
+        ExosphereNeedsShutdown       = 65002,
+        ExosphereGitCommitHash       = 65003,
+        ExosphereHasRcmBugPatch      = 65004,
+        ExosphereBlankProdInfo       = 65005,
+        ExosphereAllowCalWrites      = 65006,
+        ExosphereEmummcType          = 65007,
+        ExospherePayloadAddress      = 65008,
+        ExosphereLogConfiguration    = 65009,
+        ExosphereForceEnableUsb30    = 65010,
+        ExosphereSupportedHosVersion = 65011,
+    };
+
+    enum class SmcResult {
+        Success               = 0,
+        NotImplemented        = 1,
+        InvalidArgument       = 2,
+        InProgress            = 3,
+        NoAsyncOperation      = 4,
+        InvalidAsyncOperation = 5,
+        NotPermitted          = 6,
+    };
+
+    void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
+
+}

libraries/libmesosphere/source/kern_debug_log.cpp

@@ -32,6 +32,9 @@ namespace ams::kern {
                 return;
             }
 
+            #if defined(MESOSPHERE_DEBUG_LOG_USE_SEMIHOSTING)
+            KDebugLogImpl::PutStringBySemihosting(str);
+            #else
             while (*str) {
                 /* Get a character. */
                 const char c = *(str++);
@@ -44,6 +47,7 @@ namespace ams::kern {
             }
 
             KDebugLogImpl::Flush();
+            #endif
         }
 
         #if defined(MESOSPHERE_ENABLE_DEBUG_PRINT)
@@ -54,6 +58,11 @@ namespace ams::kern {
                 return ResultSuccess();
             }
 
+            #if defined(MESOSPHERE_DEBUG_LOG_USE_SEMIHOSTING)
+            /* TODO: should we do this properly? */
+            KDebugLogImpl::PutStringBySemihosting(user_str.GetUnsafePointer());
+            MESOSPHERE_UNUSED(len);
+            #else
             for (size_t i = 0; i < len; ++i) {
                 /* Get a character. */
                 char c;
@@ -67,6 +76,7 @@ namespace ams::kern {
             }
 
             KDebugLogImpl::Flush();
+            #endif
 
             return ResultSuccess();
         }

libraries/libmesosphere/source/kern_debug_log_impl.arch.arm64.s

@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* ams::kern::KDebugLogImpl::PutStringBySemihosting(const char *str) */
+.section    .text._ZN3ams4kern13KDebugLogImpl22PutStringBySemihostingEPKc, "ax", %progbits
+.global     _ZN3ams4kern13KDebugLogImpl22PutStringBySemihostingEPKc
+.type       _ZN3ams4kern13KDebugLogImpl22PutStringBySemihostingEPKc, %function
+.balign 0x10
+_ZN3ams4kern13KDebugLogImpl22PutStringBySemihostingEPKc:
+    mov x1, x0
+    mov x0, #0x4
+    hlt #0xF000
+    ret
+

libraries/libmesosphere/source/kern_debug_log_impl.board.qemu_virt.cpp

@@ -0,0 +1,50 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <mesosphere.hpp>
+#include "kern_debug_log_impl.hpp"
+
+namespace ams::kern {
+
+#if defined(MESOSPHERE_DEBUG_LOG_USE_SEMIHOSTING)
+
+    bool KDebugLogImpl::Initialize() {
+        return true;
+    }
+
+    void KDebugLogImpl::PutChar(char c) {
+        /* TODO */
+        AMS_UNUSED(c);
+    }
+
+    void KDebugLogImpl::Flush() {
+        /* ... */
+    }
+
+    void KDebugLogImpl::Save() {
+        /* ... */
+    }
+
+    void KDebugLogImpl::Restore() {
+        /* ... */
+    }
+
+#else
+
+    #error "Unknown Debug device!"
+
+#endif
+
+}

libraries/libmesosphere/source/kern_debug_log_impl.hpp

@@ -21,6 +21,7 @@ namespace ams::kern {
     class KDebugLogImpl {
         public:
             static NOINLINE bool Initialize();
+            static NOINLINE void PutStringBySemihosting(const char *s);
             static NOINLINE void PutChar(char c);
             static NOINLINE void Flush();
 

libraries/libmesosphere/source/kern_k_capabilities.cpp

@@ -117,11 +117,15 @@ namespace ams::kern {
     }
 
     Result KCapabilities::MapRange(const util::BitPack32 cap, const util::BitPack32 size_cap, KProcessPageTable *page_table) {
+        /* Get/validate address/size */
+        #if defined(MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES)
+        const u64 phys_addr    = static_cast<u64>(cap.Get<MapRange::Address>() | (size_cap.Get<MapRangeSize::AddressHigh>() << MapRange::Address::Count)) * PageSize;
+        #else
+        const u64 phys_addr    = static_cast<u64>(cap.Get<MapRange::Address>()) * PageSize;
+
         /* Validate reserved bits are unused. */
         R_UNLESS(size_cap.Get<MapRangeSize::Reserved>() == 0, svc::ResultOutOfRange());
-
-        /* Get/validate address/size */
-        const u64 phys_addr    = cap.Get<MapRange::Address>() * PageSize;
+        #endif
         const size_t num_pages = size_cap.Get<MapRangeSize::Pages>();
         const size_t size      = num_pages * PageSize;
         R_UNLESS(phys_addr == GetInteger(KPhysicalAddress(phys_addr)),    svc::ResultInvalidAddress());

libraries/libmesosphere/source/kern_k_condition_variable.cpp

@@ -250,7 +250,7 @@ namespace ams::kern {
                 {
                     const u32 has_waiter_flag = 1;
                     WriteToUser(key, std::addressof(has_waiter_flag));
-                    cpu::DataMemoryBarrier();
+                    cpu::DataMemoryBarrierInnerShareable();
                 }
 
                 /* Write the value to userspace. */

libraries/libmesosphere/source/kern_k_memory_block_manager.cpp

@@ -19,7 +19,7 @@ namespace ams::kern {
 
     namespace {
 
-        constexpr std::tuple<KMemoryState, const char *> MemoryStateNames[] = {
+        constexpr const std::pair<KMemoryState, const char *> MemoryStateNames[] = {
             {KMemoryState_Free               , "----- Free -----"},
             {KMemoryState_Io                 , "Io              "},
             {KMemoryState_Static             , "Static          "},
@@ -41,6 +41,7 @@ namespace ams::kern {
             {KMemoryState_Kernel             , "Kernel          "},
             {KMemoryState_GeneratedCode      , "GeneratedCode   "},
             {KMemoryState_CodeOut            , "CodeOut         "},
+            {KMemoryState_Coverage           , "Coverage        "},
         };
 
         constexpr const char *GetMemoryStateName(KMemoryState state) {

libraries/libmesosphere/source/kern_k_memory_layout.board.nintendo_nx.cpp

@@ -19,7 +19,6 @@ namespace ams::kern {
 
     namespace {
 
-        constexpr uintptr_t DramPhysicalAddress = 0x80000000;
         constexpr size_t ReservedEarlyDramSize  = 0x60000;
 
         constexpr size_t CarveoutAlignment      = 0x20000;
@@ -100,7 +99,7 @@ namespace ams::kern {
 
         void SetupDramPhysicalMemoryRegions() {
             const size_t intended_memory_size                   = KSystemControl::Init::GetIntendedMemorySize();
-            const KPhysicalAddress physical_memory_base_address = KSystemControl::Init::GetKernelPhysicalBaseAddress(DramPhysicalAddress);
+            const KPhysicalAddress physical_memory_base_address = KSystemControl::Init::GetKernelPhysicalBaseAddress(ams::kern::MainMemoryAddress);
 
             /* Insert blocks into the tree. */
             MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(GetInteger(physical_memory_base_address), intended_memory_size,  KMemoryRegionType_Dram));
@@ -173,16 +172,21 @@ namespace ams::kern {
                 InsertPoolPartitionRegionIntoBothTrees(unsafe_system_pool_start, unsafe_system_pool_size, KMemoryRegionType_DramSystemNonSecurePool, KMemoryRegionType_VirtualDramSystemNonSecurePool, cur_pool_attr);
                 total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(unsafe_system_pool_size);
 
-                /* Insert the pool management region. */
+                /* Determine final total overhead size. */
                 total_overhead_size += KMemoryManager::CalculateManagementOverheadSize((unsafe_system_pool_start - pool_partitions_start) - total_overhead_size);
-                const uintptr_t pool_management_start = unsafe_system_pool_start - total_overhead_size;
+
+                /* NOTE: Nintendo's kernel has layout [System, Management] but we have [Management, System]. This ensures the four UserPool regions are contiguous. */
+
+                /* Insert the system pool. */
+                const uintptr_t system_pool_start = pool_partitions_start + total_overhead_size;
+                const size_t    system_pool_size  = unsafe_system_pool_start - system_pool_start;
+                InsertPoolPartitionRegionIntoBothTrees(system_pool_start, system_pool_size, KMemoryRegionType_DramSystemPool, KMemoryRegionType_VirtualDramSystemPool, cur_pool_attr);
+
+                /* Insert the pool management region. */
+                const uintptr_t pool_management_start = pool_partitions_start;
                 const size_t    pool_management_size  = total_overhead_size;
                 u32 pool_management_attr = 0;
                 InsertPoolPartitionRegionIntoBothTrees(pool_management_start, pool_management_size, KMemoryRegionType_DramPoolManagement, KMemoryRegionType_VirtualDramPoolManagement, pool_management_attr);
-
-                /* Insert the system pool. */
-                const uintptr_t system_pool_size = pool_management_start - pool_partitions_start;
-                InsertPoolPartitionRegionIntoBothTrees(pool_partitions_start, system_pool_size, KMemoryRegionType_DramSystemPool, KMemoryRegionType_VirtualDramSystemPool, cur_pool_attr);
             } else {
                 /* On < 5.0.0, setup a legacy 2-pool layout for backwards compatibility. */
 
@@ -249,14 +253,18 @@ namespace ams::kern {
                     total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(second_application_pool_size);
                 }
 
-                /* Insert the secure pool. */
-                InsertPoolPartitionRegionIntoBothTrees(pool_partitions_start, secure_pool_size, KMemoryRegionType_DramSystemPool, KMemoryRegionType_VirtualDramSystemPool, cur_pool_attr);
-
-                /* Insert the pool management region. */
+                /* Validate the true overhead size. */
                 MESOSPHERE_INIT_ABORT_UNLESS(total_overhead_size <= approximate_total_overhead_size);
 
-                const uintptr_t pool_management_start = pool_partitions_start + secure_pool_size;
-                const size_t    pool_management_size  = unsafe_memory_start - pool_management_start;
+                /* NOTE: Nintendo's kernel has layout [System, Management] but we have [Management, System]. This ensures the UserPool regions are contiguous. */
+
+                /* Insert the secure pool. */
+                const uintptr_t secure_pool_start = unsafe_memory_start - secure_pool_size;
+                InsertPoolPartitionRegionIntoBothTrees(secure_pool_start, secure_pool_size, KMemoryRegionType_DramSystemPool, KMemoryRegionType_VirtualDramSystemPool, cur_pool_attr);
+
+                /* Insert the pool management region. */
+                const uintptr_t pool_management_start = pool_partitions_start;
+                const size_t    pool_management_size  = secure_pool_start - pool_management_start;
                 MESOSPHERE_INIT_ABORT_UNLESS(total_overhead_size <= pool_management_size);
 
                 u32 pool_management_attr = 0;

libraries/libmesosphere/source/kern_k_memory_layout.board.qemu_virt.cpp

@@ -0,0 +1,144 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <mesosphere.hpp>
+
+namespace ams::kern {
+
+    namespace {
+
+        constexpr size_t ReservedEarlyDramSize  = 0x00080000;
+
+        template<typename... T> requires (std::same_as<T, KMemoryRegionAttr> && ...)
+        constexpr ALWAYS_INLINE KMemoryRegionType GetMemoryRegionType(KMemoryRegionType base, T... attr) {
+            return util::FromUnderlying<KMemoryRegionType>(util::ToUnderlying(base) | (util::ToUnderlying<T>(attr) | ...));
+        }
+
+        void InsertPoolPartitionRegionIntoBothTrees(size_t start, size_t size, KMemoryRegionType phys_type, KMemoryRegionType virt_type, u32 &cur_attr) {
+            const u32 attr = cur_attr++;
+
+            MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(start, size, phys_type, attr));
+            const KMemoryRegion *phys = KMemoryLayout::GetPhysicalMemoryRegionTree().FindByTypeAndAttribute(phys_type, attr);
+            MESOSPHERE_INIT_ABORT_UNLESS(phys != nullptr);
+            MESOSPHERE_INIT_ABORT_UNLESS(phys->GetEndAddress() != 0);
+            MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetVirtualMemoryRegionTree().Insert(phys->GetPairAddress(), size, virt_type, attr));
+        }
+
+    }
+
+    namespace init {
+
+        void SetupDevicePhysicalMemoryRegions() {
+            MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(0x08000000, 0x10000,  GetMemoryRegionType(KMemoryRegionType_InterruptDistributor,  KMemoryRegionAttr_ShouldKernelMap)));
+            MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(0x08010000, 0x10000,  GetMemoryRegionType(KMemoryRegionType_InterruptCpuInterface, KMemoryRegionAttr_ShouldKernelMap)));
+        }
+
+        void SetupDramPhysicalMemoryRegions() {
+            const size_t intended_memory_size                   = KSystemControl::Init::GetIntendedMemorySize();
+            const KPhysicalAddress physical_memory_base_address = KSystemControl::Init::GetKernelPhysicalBaseAddress(ams::kern::MainMemoryAddress);
+
+            /* Insert blocks into the tree. */
+            MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(GetInteger(physical_memory_base_address), intended_memory_size,  KMemoryRegionType_Dram));
+            MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(GetInteger(physical_memory_base_address), ReservedEarlyDramSize, KMemoryRegionType_DramReservedEarly));
+
+            /* Insert the KTrace block at the end of Dram, if KTrace is enabled. */
+            static_assert(!IsKTraceEnabled || KTraceBufferSize > 0);
+            if constexpr (IsKTraceEnabled) {
+                const KPhysicalAddress ktrace_buffer_phys_addr = physical_memory_base_address + intended_memory_size - KTraceBufferSize;
+                MESOSPHERE_INIT_ABORT_UNLESS(KMemoryLayout::GetPhysicalMemoryRegionTree().Insert(GetInteger(ktrace_buffer_phys_addr), KTraceBufferSize, KMemoryRegionType_KernelTraceBuffer));
+            }
+        }
+
+        void SetupPoolPartitionMemoryRegions() {
+            /* Start by identifying the extents of the DRAM memory region. */
+            const auto dram_extents = KMemoryLayout::GetMainMemoryPhysicalExtents();
+            MESOSPHERE_INIT_ABORT_UNLESS(dram_extents.GetEndAddress() != 0);
+
+            /* Determine the end of the pool region. */
+            const uintptr_t pool_end = dram_extents.GetEndAddress() - KTraceBufferSize;
+
+            /* Find the start of the kernel DRAM region. */
+            const KMemoryRegion *kernel_dram_region = KMemoryLayout::GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramKernelBase);
+            MESOSPHERE_INIT_ABORT_UNLESS(kernel_dram_region != nullptr);
+
+            /* Find the start of the pool partitions region. */
+            const KMemoryRegion *pool_partitions_region = KMemoryLayout::GetPhysicalMemoryRegionTree().FindByTypeAndAttribute(KMemoryRegionType_DramPoolPartition, 0);
+            MESOSPHERE_INIT_ABORT_UNLESS(pool_partitions_region != nullptr);
+            const uintptr_t pool_partitions_start = pool_partitions_region->GetAddress();
+
+            /* Setup the pool partition layouts. */
+            /* Get Application and Applet pool sizes. */
+            const size_t application_pool_size       = KSystemControl::Init::GetApplicationPoolSize();
+            const size_t applet_pool_size            = KSystemControl::Init::GetAppletPoolSize();
+            const size_t unsafe_system_pool_min_size = KSystemControl::Init::GetMinimumNonSecureSystemPoolSize();
+
+            /* Decide on starting addresses for our pools. */
+            const uintptr_t application_pool_start   = pool_end - application_pool_size;
+            const uintptr_t applet_pool_start        = application_pool_start - applet_pool_size;
+            const uintptr_t unsafe_system_pool_start = util::AlignDown(applet_pool_start - unsafe_system_pool_min_size, PageSize);
+            const size_t    unsafe_system_pool_size  = applet_pool_start - unsafe_system_pool_start;
+
+            /* We want to arrange application pool depending on where the middle of dram is. */
+            const uintptr_t dram_midpoint = (dram_extents.GetAddress() + dram_extents.GetEndAddress()) / 2;
+            u32 cur_pool_attr = 0;
+            size_t total_overhead_size = 0;
+
+            /* Insert the application pool. */
+            if (application_pool_size > 0) {
+                if (dram_extents.GetEndAddress() <= dram_midpoint || dram_midpoint <= application_pool_start) {
+                    InsertPoolPartitionRegionIntoBothTrees(application_pool_start, application_pool_size, KMemoryRegionType_DramApplicationPool, KMemoryRegionType_VirtualDramApplicationPool, cur_pool_attr);
+                    total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(application_pool_size);
+                } else {
+                    const size_t first_application_pool_size  = dram_midpoint - application_pool_start;
+                    const size_t second_application_pool_size = application_pool_start + application_pool_size - dram_midpoint;
+                    InsertPoolPartitionRegionIntoBothTrees(application_pool_start, first_application_pool_size, KMemoryRegionType_DramApplicationPool, KMemoryRegionType_VirtualDramApplicationPool, cur_pool_attr);
+                    InsertPoolPartitionRegionIntoBothTrees(dram_midpoint, second_application_pool_size, KMemoryRegionType_DramApplicationPool, KMemoryRegionType_VirtualDramApplicationPool, cur_pool_attr);
+                    total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(first_application_pool_size);
+                    total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(second_application_pool_size);
+                }
+            }
+
+            /* Insert the applet pool. */
+            if (applet_pool_size > 0) {
+                InsertPoolPartitionRegionIntoBothTrees(applet_pool_start, applet_pool_size, KMemoryRegionType_DramAppletPool, KMemoryRegionType_VirtualDramAppletPool, cur_pool_attr);
+                total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(applet_pool_size);
+            }
+
+            /* Insert the nonsecure system pool. */
+            if (unsafe_system_pool_size > 0) {
+                InsertPoolPartitionRegionIntoBothTrees(unsafe_system_pool_start, unsafe_system_pool_size, KMemoryRegionType_DramSystemNonSecurePool, KMemoryRegionType_VirtualDramSystemNonSecurePool, cur_pool_attr);
+                total_overhead_size += KMemoryManager::CalculateManagementOverheadSize(unsafe_system_pool_size);
+            }
+
+            /* Determine final total overhead size. */
+            total_overhead_size += KMemoryManager::CalculateManagementOverheadSize((unsafe_system_pool_start - pool_partitions_start) - total_overhead_size);
+
+            /* NOTE: Nintendo's kernel has layout [System, Management] but we have [Management, System]. This ensures the four UserPool regions are contiguous. */
+
+            /* Insert the system pool. */
+            const uintptr_t system_pool_start = pool_partitions_start + total_overhead_size;
+            const size_t    system_pool_size  = unsafe_system_pool_start - system_pool_start;
+            InsertPoolPartitionRegionIntoBothTrees(system_pool_start, system_pool_size, KMemoryRegionType_DramSystemPool, KMemoryRegionType_VirtualDramSystemPool, cur_pool_attr);
+
+            /* Insert the pool management region. */
+            const uintptr_t pool_management_start = pool_partitions_start;
+            const size_t    pool_management_size  = total_overhead_size;
+            u32 pool_management_attr = 0;
+            InsertPoolPartitionRegionIntoBothTrees(pool_management_start, pool_management_size, KMemoryRegionType_DramPoolManagement, KMemoryRegionType_VirtualDramPoolManagement, pool_management_attr);
+        }
+
+    }
+
+}

libraries/libmesosphere/source/kern_k_process.cpp

@@ -238,7 +238,7 @@ namespace ams::kern {
         }
 
         /* Generate random entropy. */
-        KSystemControl::GenerateRandomBytes(m_entropy, sizeof(m_entropy));
+        KSystemControl::GenerateRandom(m_entropy, util::size(m_entropy));
 
         /* Clear remaining fields. */
         m_num_running_threads         = 0;

libraries/libmesosphere/source/kern_k_scheduler.cpp

@@ -79,13 +79,6 @@ namespace ams::kern {
         RescheduleCurrentCore();
     }
 
-    void KScheduler::RescheduleOtherCores(u64 cores_needing_scheduling) {
-        if (const u64 core_mask = cores_needing_scheduling & ~(1ul << m_core_id); core_mask != 0) {
-            cpu::DataSynchronizationBarrier();
-            Kernel::GetInterruptManager().SendInterProcessorInterrupt(KInterruptName_Scheduler, core_mask);
-        }
-    }
-
     u64 KScheduler::UpdateHighestPriorityThread(KThread *highest_thread) {
         if (KThread *prev_highest_thread = m_state.highest_priority_thread; AMS_LIKELY(prev_highest_thread != highest_thread)) {
             if (AMS_LIKELY(prev_highest_thread != nullptr)) {
@@ -254,9 +247,24 @@ namespace ams::kern {
 
         MESOSPHERE_KTRACE_THREAD_SWITCH(next_thread);
 
+        #if defined(MESOSPHERE_ENABLE_HARDWARE_SINGLE_STEP)
+        /* Ensure the single-step bit in mdscr reflects the correct single-step state for the new thread. */
+        cpu::MonitorDebugSystemControlRegisterAccessor().SetSoftwareStep(next_thread->IsSingleStep()).Store();
+        #endif
+
         /* Switch the current process, if we're switching processes. */
         if (KProcess *next_process = next_thread->GetOwnerProcess(); next_process != cur_process) {
             KProcess::Switch(cur_process, next_process);
+        } else {
+            /* The single-step bit set up above requires an instruction synchronization barrier, to ensure */
+            /* the state change takes before we actually perform a return which might break-to-step. */
+            /* KProcess::Switch performs an isb incidentally, and so when we're changing process we */
+            /* can piggy-back off of that isb to avoid unnecessarily emptying the pipeline twice. */
+            /* However, this means that when we're switching to thread in a different process, */
+            /* we must ensure that we still isb. In practice, gcc will deduplicate into a single isb. */
+            #if defined(MESOSPHERE_ENABLE_HARDWARE_SINGLE_STEP)
+            cpu::InstructionMemoryBarrier();
+            #endif
         }
 
         /* Set the new thread. */

libraries/libmesosphere/source/kern_k_system_control_base.cpp

@@ -0,0 +1,295 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <mesosphere.hpp>
+#if defined(ATMOSPHERE_ARCH_ARM64)
+#include <mesosphere/arch/arm64/kern_secure_monitor_base.hpp>
+#endif
+
+namespace ams::kern {
+
+    /* Initialization. */
+    size_t KSystemControlBase::Init::GetRealMemorySize() {
+        return ams::kern::MainMemorySize;
+    }
+
+    size_t KSystemControlBase::Init::GetIntendedMemorySize() {
+        return ams::kern::MainMemorySize;
+    }
+
+    KPhysicalAddress KSystemControlBase::Init::GetKernelPhysicalBaseAddress(KPhysicalAddress base_address) {
+        const size_t real_dram_size     = KSystemControl::Init::GetRealMemorySize();
+        const size_t intended_dram_size = KSystemControl::Init::GetIntendedMemorySize();
+        if (intended_dram_size * 2 < real_dram_size) {
+            return base_address;
+        } else {
+            return base_address + ((real_dram_size - intended_dram_size) / 2);
+        }
+    }
+
+    void KSystemControlBase::Init::GetInitialProcessBinaryLayout(InitialProcessBinaryLayout *out) {
+        *out = {
+            .address = GetInteger(KSystemControl::Init::GetKernelPhysicalBaseAddress(ams::kern::MainMemoryAddress)) + KSystemControl::Init::GetIntendedMemorySize() - KTraceBufferSize - InitialProcessBinarySizeMax,
+            ._08     = 0,
+        };
+    }
+
+
+    bool KSystemControlBase::Init::ShouldIncreaseThreadResourceLimit() {
+        return true;
+    }
+
+
+    size_t KSystemControlBase::Init::GetApplicationPoolSize() {
+        return 0;
+    }
+
+    size_t KSystemControlBase::Init::GetAppletPoolSize() {
+        return 0;
+    }
+
+    size_t KSystemControlBase::Init::GetMinimumNonSecureSystemPoolSize() {
+        return 0;
+    }
+
+    u8 KSystemControlBase::Init::GetDebugLogUartPort() {
+        return 0;
+    }
+
+    void KSystemControlBase::Init::CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg) {
+        #if defined(ATMOSPHERE_ARCH_ARM64)
+        MESOSPHERE_INIT_ABORT_UNLESS((::ams::kern::arch::arm64::smc::CpuOn<0, false>(core_id, entrypoint, arg)) == 0);
+        #else
+        AMS_INFINITE_LOOP();
+        #endif
+    }
+
+    /* Randomness for Initialization. */
+    void KSystemControlBase::Init::GenerateRandom(u64 *dst, size_t count) {
+        if (AMS_UNLIKELY(!s_initialized_random_generator)) {
+            const u64 seed = KHardwareTimer::GetTick();
+            s_random_generator.Initialize(reinterpret_cast<const u32*>(std::addressof(seed)), sizeof(seed) / sizeof(u32));
+            s_initialized_random_generator = true;
+        }
+
+        for (size_t i = 0; i < count; ++i) {
+            dst[i] = s_random_generator.GenerateRandomU64();
+        }
+    }
+
+    u64 KSystemControlBase::Init::GenerateRandomRange(u64 min, u64 max) {
+        if (AMS_UNLIKELY(!s_initialized_random_generator)) {
+            const u64 seed = KHardwareTimer::GetTick();
+            s_random_generator.Initialize(reinterpret_cast<const u32*>(std::addressof(seed)), sizeof(seed) / sizeof(u32));
+            s_initialized_random_generator = true;
+        }
+
+        return KSystemControlBase::GenerateUniformRange(min, max, [] ALWAYS_INLINE_LAMBDA () -> u64 { return s_random_generator.GenerateRandomU64(); });
+    }
+
+    /* System Initialization. */
+    void KSystemControlBase::InitializePhase1(bool skip_target_system) {
+        /* Initialize the rng, if we somehow haven't already. */
+        if (AMS_UNLIKELY(!s_initialized_random_generator)) {
+            const u64 seed = KHardwareTimer::GetTick();
+            s_random_generator.Initialize(reinterpret_cast<const u32*>(std::addressof(seed)), sizeof(seed) / sizeof(u32));
+            s_initialized_random_generator = true;
+        }
+
+        /* Configure KTargetSystem, if we haven't already by an implementation SystemControl. */
+        if (!skip_target_system) {
+            /* Set IsDebugMode. */
+            {
+                KTargetSystem::SetIsDebugMode(true);
+
+                /* If debug mode, we want to initialize uart logging. */
+                KTargetSystem::EnableDebugLogging(true);
+                KDebugLog::Initialize();
+            }
+
+            /* Set Kernel Configuration. */
+            {
+                KTargetSystem::EnableDebugMemoryFill(false);
+                KTargetSystem::EnableUserExceptionHandlers(true);
+                KTargetSystem::EnableDynamicResourceLimits(true);
+                KTargetSystem::EnableUserPmuAccess(false);
+            }
+
+            /* Set Kernel Debugging. */
+            {
+                /* NOTE: This is used to restrict access to SvcKernelDebug/SvcChangeKernelTraceState. */
+                /* Mesosphere may wish to not require this, as we'd ideally keep ProgramVerification enabled for userland. */
+                KTargetSystem::EnableKernelDebugging(true);
+            }
+        }
+
+        /* System ResourceLimit initialization. */
+        {
+            /* Construct the resource limit object. */
+            KResourceLimit &sys_res_limit = Kernel::GetSystemResourceLimit();
+            KAutoObject::Create<KResourceLimit>(std::addressof(sys_res_limit));
+            sys_res_limit.Initialize();
+
+            /* Set the initial limits. */
+            const auto [total_memory_size, kernel_memory_size] = KMemoryLayout::GetTotalAndKernelMemorySizes();
+            const auto &slab_counts = init::GetSlabResourceCounts();
+            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_PhysicalMemoryMax,      total_memory_size));
+            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_ThreadCountMax,         slab_counts.num_KThread));
+            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_EventCountMax,          slab_counts.num_KEvent));
+            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_TransferMemoryCountMax, slab_counts.num_KTransferMemory));
+            MESOSPHERE_R_ABORT_UNLESS(sys_res_limit.SetLimitValue(ams::svc::LimitableResource_SessionCountMax,        slab_counts.num_KSession));
+
+            /* Reserve system memory. */
+            MESOSPHERE_ABORT_UNLESS(sys_res_limit.Reserve(ams::svc::LimitableResource_PhysicalMemoryMax, kernel_memory_size));
+        }
+    }
+
+    void KSystemControlBase::InitializePhase2() {
+        /* Initialize KTrace. */
+        if constexpr (IsKTraceEnabled) {
+            const auto &ktrace = KMemoryLayout::GetKernelTraceBufferRegion();
+            KTrace::Initialize(ktrace.GetAddress(), ktrace.GetSize());
+        }
+    }
+
+    u32 KSystemControlBase::GetCreateProcessMemoryPool() {
+        return KMemoryManager::Pool_System;
+    }
+
+    /* Privileged Access. */
+    void KSystemControlBase::ReadWriteRegisterPrivileged(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value) {
+        /* TODO */
+        MESOSPHERE_UNUSED(out, address, mask, value);
+        MESOSPHERE_UNIMPLEMENTED();
+    }
+
+    Result KSystemControlBase::ReadWriteRegister(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value) {
+        MESOSPHERE_UNUSED(out, address, mask, value);
+        return svc::ResultNotImplemented();
+    }
+
+    /* Randomness. */
+    void KSystemControlBase::GenerateRandom(u64 *dst, size_t count) {
+        KScopedInterruptDisable intr_disable;
+        KScopedSpinLock lk(s_random_lock);
+
+        for (size_t i = 0; i < count; ++i) {
+            dst[i] = s_random_generator.GenerateRandomU64();
+        }
+    }
+
+    u64 KSystemControlBase::GenerateRandomRange(u64 min, u64 max) {
+        KScopedInterruptDisable intr_disable;
+        KScopedSpinLock lk(s_random_lock);
+
+        return KSystemControlBase::GenerateUniformRange(min, max, [] ALWAYS_INLINE_LAMBDA () -> u64 { return s_random_generator.GenerateRandomU64(); });
+    }
+
+    u64 KSystemControlBase::GenerateRandomU64() {
+        KScopedInterruptDisable intr_disable;
+        KScopedSpinLock lk(s_random_lock);
+
+        return s_random_generator.GenerateRandomU64();
+    }
+
+    void KSystemControlBase::SleepSystem() {
+        MESOSPHERE_LOG("SleepSystem() was called\n");
+    }
+
+    void KSystemControlBase::StopSystem(void *) {
+        MESOSPHERE_LOG("KSystemControlBase::StopSystem\n");
+        AMS_INFINITE_LOOP();
+    }
+
+    /* User access. */
+    #if defined(ATMOSPHERE_ARCH_ARM64)
+    void KSystemControlBase::CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args) {
+        /* Get the function id for the current call. */
+        u64 function_id = args->r[0];
+
+        /* We'll need to map in pages if arguments are pointers. Prepare page groups to do so. */
+        auto &page_table = GetCurrentProcess().GetPageTable();
+        auto *bim = page_table.GetBlockInfoManager();
+
+        constexpr size_t MaxMappedRegisters = 7;
+        std::array<KPageGroup, MaxMappedRegisters> page_groups = { KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), KPageGroup(bim), };
+
+        for (size_t i = 0; i < MaxMappedRegisters; i++) {
+            const size_t reg_id = i + 1;
+            if (function_id & (1ul << (8 + reg_id))) {
+                /* Create and open a new page group for the address. */
+                KVirtualAddress virt_addr = args->r[reg_id];
+
+                if (R_SUCCEEDED(page_table.MakeAndOpenPageGroup(std::addressof(page_groups[i]), util::AlignDown(GetInteger(virt_addr), PageSize), 1, KMemoryState_None, KMemoryState_None, KMemoryPermission_UserReadWrite, KMemoryPermission_UserReadWrite, KMemoryAttribute_None, KMemoryAttribute_None))) {
+                    /* Translate the virtual address to a physical address. */
+                    const auto it = page_groups[i].begin();
+                    MESOSPHERE_ASSERT(it != page_groups[i].end());
+                    MESOSPHERE_ASSERT(it->GetNumPages() == 1);
+
+                    args->r[reg_id] = GetInteger(it->GetAddress()) | (GetInteger(virt_addr) & (PageSize - 1));
+                } else {
+                    /* If we couldn't map, we should clear the address. */
+                    args->r[reg_id] = 0;
+                }
+            }
+        }
+
+        /* Invoke the secure monitor. */
+        KSystemControl::CallSecureMonitorFromUserImpl(args);
+
+        /* Make sure that we close any pages that we opened. */
+        for (size_t i = 0; i < MaxMappedRegisters; i++) {
+            page_groups[i].Close();
+        }
+    }
+
+    void KSystemControlBase::CallSecureMonitorFromUserImpl(ams::svc::lp64::SecureMonitorArguments *args) {
+        /* By default, we don't actually support secure monitor, so just set args to a failure code. */
+        args->r[0] = 1;
+    }
+    #endif
+
+    /* Secure Memory. */
+    size_t KSystemControlBase::CalculateRequiredSecureMemorySize(size_t size, u32 pool) {
+        MESOSPHERE_UNUSED(pool);
+        return size;
+    }
+
+    Result KSystemControlBase::AllocateSecureMemory(KVirtualAddress *out, size_t size, u32 pool) {
+        /* Ensure the size is aligned. */
+        constexpr size_t Alignment = PageSize;
+        R_UNLESS(util::IsAligned(size, Alignment), svc::ResultInvalidSize());
+
+        /* Allocate the memory. */
+        const size_t num_pages = size / PageSize;
+        const KPhysicalAddress paddr = Kernel::GetMemoryManager().AllocateAndOpenContinuous(num_pages, Alignment / PageSize, KMemoryManager::EncodeOption(static_cast<KMemoryManager::Pool>(pool), KMemoryManager::Direction_FromFront));
+        R_UNLESS(paddr != Null<KPhysicalAddress>, svc::ResultOutOfMemory());
+
+        *out = KPageTable::GetHeapVirtualAddress(paddr);
+        return ResultSuccess();
+    }
+
+    void KSystemControlBase::FreeSecureMemory(KVirtualAddress address, size_t size, u32 pool) {
+        /* Ensure the size is aligned. */
+        constexpr size_t Alignment = PageSize;
+        MESOSPHERE_UNUSED(pool);
+        MESOSPHERE_ABORT_UNLESS(util::IsAligned(GetInteger(address), Alignment));
+        MESOSPHERE_ABORT_UNLESS(util::IsAligned(size, Alignment));
+
+        /* Close the secure region's pages. */
+        Kernel::GetMemoryManager().Close(KPageTable::GetHeapPhysicalAddress(address), size / PageSize);
+    }
+
+}

libraries/libmesosphere/source/kern_k_thread.cpp

@@ -1315,10 +1315,6 @@ namespace ams::kern {
         }
     }
 
-    KThreadContext *KThread::GetContextForSchedulerLoop() {
-        return std::addressof(this->GetContext());
-    }
-
     KThread *KThread::GetThreadFromId(u64 thread_id) {
         /* Lock the list. */
         KThread::ListAccessor accessor;

libraries/libmesosphere/source/kern_kernel.cpp

@@ -139,14 +139,20 @@ namespace ams::kern {
         PrintMemoryRegion("        InitPageTable",  KMemoryLayout::GetKernelInitPageTableRegionPhysicalExtents());
         PrintMemoryRegion("    MemoryPoolRegion",   KMemoryLayout::GetKernelPoolPartitionRegionPhysicalExtents());
         if (GetTargetFirmware() >= TargetFirmware_5_0_0) {
-            PrintMemoryRegion("        System",         KMemoryLayout::GetKernelSystemPoolRegionPhysicalExtents());
             PrintMemoryRegion("        Management",     KMemoryLayout::GetKernelPoolManagementRegionPhysicalExtents());
-            PrintMemoryRegion("        SystemUnsafe",   KMemoryLayout::GetKernelSystemNonSecurePoolRegionPhysicalExtents());
-            PrintMemoryRegion("        Applet",         KMemoryLayout::GetKernelAppletPoolRegionPhysicalExtents());
-            PrintMemoryRegion("        Application",    KMemoryLayout::GetKernelApplicationPoolRegionPhysicalExtents());
+            PrintMemoryRegion("        System",         KMemoryLayout::GetKernelSystemPoolRegionPhysicalExtents());
+            if (KMemoryLayout::HasKernelSystemNonSecurePoolRegion()) {
+                PrintMemoryRegion("        SystemUnsafe",   KMemoryLayout::GetKernelSystemNonSecurePoolRegionPhysicalExtents());
+            }
+            if (KMemoryLayout::HasKernelAppletPoolRegion()) {
+                PrintMemoryRegion("        Applet",         KMemoryLayout::GetKernelAppletPoolRegionPhysicalExtents());
+            }
+            if (KMemoryLayout::HasKernelApplicationPoolRegion()) {
+                PrintMemoryRegion("        Application",    KMemoryLayout::GetKernelApplicationPoolRegionPhysicalExtents());
+            }
         } else {
-            PrintMemoryRegion("        Secure",     KMemoryLayout::GetKernelSystemPoolRegionPhysicalExtents());
             PrintMemoryRegion("        Management", KMemoryLayout::GetKernelPoolManagementRegionPhysicalExtents());
+            PrintMemoryRegion("        Secure",     KMemoryLayout::GetKernelSystemPoolRegionPhysicalExtents());
             PrintMemoryRegion("        Unsafe",     KMemoryLayout::GetKernelApplicationPoolRegionPhysicalExtents());
         }
         if constexpr (IsKTraceEnabled) {

libraries/libmesosphere/source/svc/kern_svc_cache.cpp

@@ -36,7 +36,7 @@ namespace ams::kern::svc {
             size_t remaining = size;
             while (remaining > 0) {
                 /* Get a contiguous range to operate on. */
-                KPageTableBase::MemoryRange contig_range = {};
+                KPageTableBase::MemoryRange contig_range = { .address = Null<KPhysicalAddress>, .size = 0 };
                 R_TRY(page_table.OpenMemoryRangeForProcessCacheOperation(std::addressof(contig_range), cur_address, aligned_end - cur_address));
 
                 /* Close the range when we're done operating on it. */

libraries/libmesosphere/source/svc/kern_svc_info.cpp

@@ -38,6 +38,80 @@ namespace ams::kern::svc {
             return ResultSuccess();
         }
 
+        Result GetInfoImpl(u64 *out, ams::svc::InfoType info_type, KProcess *process) {
+            switch (info_type) {
+                case ams::svc::InfoType_CoreMask:
+                    *out = process->GetCoreMask();
+                    break;
+                case ams::svc::InfoType_PriorityMask:
+                    *out = process->GetPriorityMask();
+                    break;
+                case ams::svc::InfoType_AliasRegionAddress:
+                    *out = GetInteger(process->GetPageTable().GetAliasRegionStart());
+                    break;
+                case ams::svc::InfoType_AliasRegionSize:
+                    *out = process->GetPageTable().GetAliasRegionSize();
+                    break;
+                case ams::svc::InfoType_HeapRegionAddress:
+                    *out = GetInteger(process->GetPageTable().GetHeapRegionStart());
+                    break;
+                case ams::svc::InfoType_HeapRegionSize:
+                    *out = process->GetPageTable().GetHeapRegionSize();
+                    break;
+                case ams::svc::InfoType_TotalMemorySize:
+                    *out = process->GetTotalUserPhysicalMemorySize();
+                    break;
+                case ams::svc::InfoType_UsedMemorySize:
+                    *out = process->GetUsedUserPhysicalMemorySize();
+                    break;
+                case ams::svc::InfoType_AslrRegionAddress:
+                    *out = GetInteger(process->GetPageTable().GetAliasCodeRegionStart());
+                    break;
+                case ams::svc::InfoType_AslrRegionSize:
+                    *out = process->GetPageTable().GetAliasCodeRegionSize();
+                    break;
+                case ams::svc::InfoType_StackRegionAddress:
+                    *out = GetInteger(process->GetPageTable().GetStackRegionStart());
+                    break;
+                case ams::svc::InfoType_StackRegionSize:
+                    *out = process->GetPageTable().GetStackRegionSize();
+                    break;
+                case ams::svc::InfoType_SystemResourceSizeTotal:
+                    *out = process->GetTotalSystemResourceSize();
+                    break;
+                case ams::svc::InfoType_SystemResourceSizeUsed:
+                    *out = process->GetUsedSystemResourceSize();
+                    break;
+                case ams::svc::InfoType_ProgramId:
+                    *out = process->GetProgramId();
+                    break;
+                case ams::svc::InfoType_UserExceptionContextAddress:
+                    *out = GetInteger(process->GetProcessLocalRegionAddress());
+                    break;
+                case ams::svc::InfoType_TotalNonSystemMemorySize:
+                    *out = process->GetTotalNonSystemUserPhysicalMemorySize();
+                    break;
+                case ams::svc::InfoType_UsedNonSystemMemorySize:
+                    *out = process->GetUsedNonSystemUserPhysicalMemorySize();
+                    break;
+                case ams::svc::InfoType_IsApplication:
+                    *out = process->IsApplication();
+                    break;
+                case ams::svc::InfoType_FreeThreadCount:
+                    if (KResourceLimit *resource_limit = process->GetResourceLimit(); resource_limit != nullptr) {
+                        const auto current_value = resource_limit->GetCurrentValue(ams::svc::LimitableResource_ThreadCountMax);
+                        const auto limit_value   = resource_limit->GetLimitValue(ams::svc::LimitableResource_ThreadCountMax);
+                        *out = limit_value - current_value;
+                    } else {
+                        *out = 0;
+                    }
+                    break;
+                MESOSPHERE_UNREACHABLE_DEFAULT_CASE();
+            }
+
+            return ResultSuccess();
+        }
+
         Result GetInfo(u64 *out, ams::svc::InfoType info_type, ams::svc::Handle handle, u64 info_subtype) {
             switch (info_type) {
                 case ams::svc::InfoType_CoreMask:
@@ -66,77 +140,34 @@ namespace ams::kern::svc {
 
                         /* Get the process from its handle. */
                         KScopedAutoObject process = GetCurrentProcess().GetHandleTable().GetObject<KProcess>(handle);
+
+                        #if defined(MESOSPHERE_ENABLE_GET_INFO_OF_DEBUG_PROCESS)
+                        /* If we the process is valid, use it. */
+                        if (process.IsNotNull()) {
+                            return GetInfoImpl(out, info_type, process.GetPointerUnsafe());
+                        }
+
+                        /* Otherwise, as a mesosphere extension check if we were passed a usable KDebug. */
+                        KScopedAutoObject debug = GetCurrentProcess().GetHandleTable().GetObject<KDebug>(handle);
+                        R_UNLESS(debug.IsNotNull(), svc::ResultInvalidHandle());
+
+                        /* Get the process from the debug object. */
+                        /* TODO: ResultInvalidHandle()? */
+                        R_UNLESS(debug->IsAttached(),  svc::ResultProcessTerminated());
+                        R_UNLESS(debug->OpenProcess(), svc::ResultProcessTerminated());
+
+                        /* Close the process when we're done. */
+                        ON_SCOPE_EXIT { debug->CloseProcess(); };
+
+                        /* Return the info. */
+                        return GetInfoImpl(out, info_type, debug->GetProcessUnsafe());
+                        #else
+                        /* Verify that the process is valid. */
                         R_UNLESS(process.IsNotNull(), svc::ResultInvalidHandle());
 
-                        switch (info_type) {
-                            case ams::svc::InfoType_CoreMask:
-                                *out = process->GetCoreMask();
-                                break;
-                            case ams::svc::InfoType_PriorityMask:
-                                *out = process->GetPriorityMask();
-                                break;
-                            case ams::svc::InfoType_AliasRegionAddress:
-                                *out = GetInteger(process->GetPageTable().GetAliasRegionStart());
-                                break;
-                            case ams::svc::InfoType_AliasRegionSize:
-                                *out = process->GetPageTable().GetAliasRegionSize();
-                                break;
-                            case ams::svc::InfoType_HeapRegionAddress:
-                                *out = GetInteger(process->GetPageTable().GetHeapRegionStart());
-                                break;
-                            case ams::svc::InfoType_HeapRegionSize:
-                                *out = process->GetPageTable().GetHeapRegionSize();
-                                break;
-                            case ams::svc::InfoType_TotalMemorySize:
-                                *out = process->GetTotalUserPhysicalMemorySize();
-                                break;
-                            case ams::svc::InfoType_UsedMemorySize:
-                                *out = process->GetUsedUserPhysicalMemorySize();
-                                break;
-                            case ams::svc::InfoType_AslrRegionAddress:
-                                *out = GetInteger(process->GetPageTable().GetAliasCodeRegionStart());
-                                break;
-                            case ams::svc::InfoType_AslrRegionSize:
-                                *out = process->GetPageTable().GetAliasCodeRegionSize();
-                                break;
-                            case ams::svc::InfoType_StackRegionAddress:
-                                *out = GetInteger(process->GetPageTable().GetStackRegionStart());
-                                break;
-                            case ams::svc::InfoType_StackRegionSize:
-                                *out = process->GetPageTable().GetStackRegionSize();
-                                break;
-                            case ams::svc::InfoType_SystemResourceSizeTotal:
-                                *out = process->GetTotalSystemResourceSize();
-                                break;
-                            case ams::svc::InfoType_SystemResourceSizeUsed:
-                                *out = process->GetUsedSystemResourceSize();
-                                break;
-                            case ams::svc::InfoType_ProgramId:
-                                *out = process->GetProgramId();
-                                break;
-                            case ams::svc::InfoType_UserExceptionContextAddress:
-                                *out = GetInteger(process->GetProcessLocalRegionAddress());
-                                break;
-                            case ams::svc::InfoType_TotalNonSystemMemorySize:
-                                *out = process->GetTotalNonSystemUserPhysicalMemorySize();
-                                break;
-                            case ams::svc::InfoType_UsedNonSystemMemorySize:
-                                *out = process->GetUsedNonSystemUserPhysicalMemorySize();
-                                break;
-                            case ams::svc::InfoType_IsApplication:
-                                *out = process->IsApplication();
-                                break;
-                            case ams::svc::InfoType_FreeThreadCount:
-                                if (KResourceLimit *resource_limit = process->GetResourceLimit(); resource_limit != nullptr) {
-                                    const auto current_value = resource_limit->GetCurrentValue(ams::svc::LimitableResource_ThreadCountMax);
-                                    const auto limit_value   = resource_limit->GetLimitValue(ams::svc::LimitableResource_ThreadCountMax);
-                                    *out = limit_value - current_value;
-                                } else {
-                                    *out = 0;
-                                }
-                                break;
-                            MESOSPHERE_UNREACHABLE_DEFAULT_CASE();
-                        }
+                        /* Return the relevant info. */
+                        return GetInfoImpl(out, info_type, process.GetPointerUnsafe());
+                        #endif
                     }
                     break;
                 case ams::svc::InfoType_DebuggerAttached:

libraries/libstratosphere/Makefile

@@ -123,6 +123,7 @@ $(OFILES)	:	$(GCH_FILES)
 $(OFILES_SRC)	: $(HFILES_BIN)
 
 ams_environment_weak.o: CXXFLAGS += -fno-lto
+hos_version_api_weak_for_unit_test.o: CXXFLAGS += -fno-lto
 pm_info_api_weak.o: CXXFLAGS += -fno-lto
 hos_stratosphere_api.o: CXXFLAGS += -fno-lto
 

libraries/libstratosphere/include/stratosphere/fssystem/fssystem_utility.hpp

@@ -118,12 +118,12 @@ namespace ams::fssystem {
 
     /* Copy API. */
     Result CopyFile(fs::fsa::IFileSystem *dst_fs, fs::fsa::IFileSystem *src_fs, const char *dst_parent_path, const char *src_path, const fs::DirectoryEntry *dir_ent, void *work_buf, size_t work_buf_size);
-    NX_INLINE Result CopyFile(fs::fsa::IFileSystem *fs, const char *dst_parent_path, const char *src_path, const fs::DirectoryEntry *dir_ent, void *work_buf, size_t work_buf_size) {
+    ALWAYS_INLINE Result CopyFile(fs::fsa::IFileSystem *fs, const char *dst_parent_path, const char *src_path, const fs::DirectoryEntry *dir_ent, void *work_buf, size_t work_buf_size) {
         return CopyFile(fs, fs, dst_parent_path, src_path, dir_ent, work_buf, work_buf_size);
     }
 
     Result CopyDirectoryRecursively(fs::fsa::IFileSystem *dst_fs, fs::fsa::IFileSystem *src_fs, const char *dst_path, const char *src_path, void *work_buf, size_t work_buf_size);
-    NX_INLINE Result CopyDirectoryRecursively(fs::fsa::IFileSystem *fs, const char *dst_path, const char *src_path, void *work_buf, size_t work_buf_size) {
+    ALWAYS_INLINE Result CopyDirectoryRecursively(fs::fsa::IFileSystem *fs, const char *dst_path, const char *src_path, void *work_buf, size_t work_buf_size) {
         return CopyDirectoryRecursively(fs, fs, dst_path, src_path, work_buf, work_buf_size);
     }
 
@@ -148,11 +148,11 @@ namespace ams::fssystem {
     Result EnsureDirectoryRecursively(fs::fsa::IFileSystem *fs, const char *path);
     Result EnsureParentDirectoryRecursively(fs::fsa::IFileSystem *fs, const char *path);
 
-    template<typename F>
-    NX_INLINE Result RetryFinitelyForTargetLocked(F f) {
+    template<s64 RetryMilliSeconds = 100>
+    ALWAYS_INLINE Result RetryFinitelyForTargetLocked(auto f) {
         /* Retry up to 10 times, 100ms between retries. */
         constexpr s32 MaxRetryCount = 10;
-        constexpr TimeSpan RetryWaitTime = TimeSpan::FromMilliSeconds(100);
+        constexpr TimeSpan RetryWaitTime = TimeSpan::FromMilliSeconds(RetryMilliSeconds);
 
         s32 remaining_retries = MaxRetryCount;
         while (true) {

libraries/libstratosphere/include/stratosphere/socket/socket_api.hpp

@@ -43,6 +43,7 @@ namespace ams::socket {
 
     s32 Shutdown(s32 desc, ShutdownMethod how);
 
+    s32 Socket(Family domain, Type type, Protocol protocol);
     s32 SocketExempt(Family domain, Type type, Protocol protocol);
 
     s32 Accept(s32 desc, SockAddr *out_address, SockLenT *out_addr_len);

libraries/libstratosphere/include/stratosphere/socket/socket_system_config.hpp

@@ -57,4 +57,42 @@ namespace ams::socket {
             }
     };
 
+    class SystemConfigLightDefault : public Config {
+        public:
+            static constexpr size_t DefaultTcpInitialSendBufferSize     = 16_KB;
+            static constexpr size_t DefaultTcpInitialReceiveBufferSize  = 16_KB;
+            static constexpr size_t DefaultTcpAutoSendBufferSizeMax     = 0_KB;
+            static constexpr size_t DefaultTcpAutoReceiveBufferSizeMax  = 0_KB;
+            static constexpr size_t DefaultUdpSendBufferSize            = 9_KB;
+            static constexpr size_t DefaultUdpReceiveBufferSize         = 42240;
+            static constexpr auto   DefaultSocketBufferEfficiency       = 2;
+            static constexpr auto   DefaultConcurrency                  = 2;
+            static constexpr size_t DefaultAllocatorPoolSize            = 64_KB;
+
+            static constexpr size_t PerTcpSocketWorstCaseMemoryPoolSize = [] {
+                constexpr size_t WorstCaseTcpSendBufferSize    = AlignMss(std::max(DefaultTcpInitialSendBufferSize, DefaultTcpAutoSendBufferSizeMax));
+                constexpr size_t WorstCaseTcpReceiveBufferSize = AlignMss(std::max(DefaultTcpInitialReceiveBufferSize, DefaultTcpAutoReceiveBufferSizeMax));
+
+                return util::AlignUp(WorstCaseTcpSendBufferSize * DefaultSocketBufferEfficiency + WorstCaseTcpReceiveBufferSize * DefaultSocketBufferEfficiency, os::MemoryPageSize);
+            }();
+
+            static constexpr size_t PerUdpSocketWorstCaseMemoryPoolSize = [] {
+                constexpr size_t WorstCaseUdpSendBufferSize    = AlignMss(DefaultUdpSendBufferSize);
+                constexpr size_t WorstCaseUdpReceiveBufferSize = AlignMss(DefaultUdpReceiveBufferSize);
+
+                return util::AlignUp(WorstCaseUdpSendBufferSize * DefaultSocketBufferEfficiency + WorstCaseUdpReceiveBufferSize * DefaultSocketBufferEfficiency, os::MemoryPageSize);
+            }();
+        public:
+            constexpr SystemConfigLightDefault(void *mp, size_t mp_sz, size_t ap, int c=DefaultConcurrency)
+                : Config(mp, mp_sz, ap,
+                         DefaultTcpInitialSendBufferSize, DefaultTcpInitialReceiveBufferSize,
+                         DefaultTcpAutoSendBufferSizeMax, DefaultTcpAutoReceiveBufferSizeMax,
+                         DefaultUdpSendBufferSize, DefaultUdpReceiveBufferSize,
+                         DefaultSocketBufferEfficiency, c)
+            {
+                /* Mark as system. */
+                m_system = true;
+            }
+    };
+
 }

libraries/libstratosphere/include/stratosphere/spl/impl/spl_api_impl.hpp

@@ -51,7 +51,7 @@ namespace ams::spl::impl {
 
     Result AllocateAesKeySlot(s32 *out_keyslot);
     Result DeallocateAesKeySlot(s32 keyslot);
-    Result TestAesKeySlot(s32 *out_index, s32 keyslot);
+    Result TestAesKeySlot(s32 *out_index, bool *out_virtual, s32 keyslot);
 
     os::SystemEvent *GetAesKeySlotAvailableEvent();
 

libraries/libstratosphere/include/stratosphere/svc/svc_stratosphere_shims.hpp

@@ -26,6 +26,11 @@
                     return ::svcSetHeapSize(reinterpret_cast<void **>(out_address), size);
                 }
 
+                ALWAYS_INLINE Result SetHeapSize(uintptr_t *out_address, ::ams::svc::Size size) {
+                    static_assert(sizeof(::ams::svc::Address) == sizeof(uintptr_t));
+                    return ::svcSetHeapSize(reinterpret_cast<void **>(out_address), size);
+                }
+
                 ALWAYS_INLINE Result SetMemoryPermission(::ams::svc::Address address, ::ams::svc::Size size, ::ams::svc::MemoryPermission perm) {
                     return ::svcSetMemoryPermission(reinterpret_cast<void *>(static_cast<uintptr_t>(address)), size, static_cast<u32>(perm));
                 }

libraries/libstratosphere/source/boot2/boot2_api.cpp

@@ -188,6 +188,12 @@ namespace ams::boot2 {
             return enable_htc != 0;
         }
 
+        bool IsStandaloneGdbstubEnabled() {
+            u8 enable_gdbstub = 0;
+            settings::fwdbg::GetSettingsItemValue(std::addressof(enable_gdbstub), sizeof(enable_gdbstub), "atmosphere", "enable_standalone_gdbstub");
+            return enable_gdbstub != 0;
+        }
+
         bool IsAtmosphereLogManagerEnabled() {
             /* If htc is enabled, ams log manager is enabled. */
             if (IsHtcEnabled()) {
@@ -403,6 +409,9 @@ namespace ams::boot2 {
             LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::Htc,      ncm::StorageId::None), 0);
             LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::Cs,       ncm::StorageId::None), 0);
             LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::DmntGen2, ncm::StorageId::None), 0);
+        } else if (IsStandaloneGdbstubEnabled()) {
+            LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::DmntGen2, ncm::StorageId::None), 0);
+            LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::Tma,  ncm::StorageId::BuiltInSystem), 0);
         } else {
             LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::Dmnt, ncm::StorageId::None), 0);
             LaunchProgram(nullptr, ncm::ProgramLocation::Make(ncm::SystemProgramId::Tma,  ncm::StorageId::BuiltInSystem), 0);

libraries/libstratosphere/source/fs/fs_save_data_management.cpp

@@ -89,9 +89,8 @@ namespace ams::fs {
     Result DeleteSystemSaveData(SaveDataSpaceId space_id, SystemSaveDataId id, UserId user_id) {
         const auto attribute = SaveDataAttribute::Make(ncm::InvalidProgramId, SaveDataType::System, user_id, id);
 
-        /* TODO: Libnx binding for DeleteSaveDataFileSystemBySaveDataAttribute */
-        AMS_UNUSED(space_id, attribute);
-        AMS_ABORT();
+        static_assert(sizeof(attribute) == sizeof(::FsSaveDataAttribute));
+        return fsDeleteSaveDataFileSystemBySaveDataAttribute(static_cast<::FsSaveDataSpaceId>(space_id), reinterpret_cast<const ::FsSaveDataAttribute *>(std::addressof(attribute)));
     }
 
     Result GetSaveDataFlags(u32 *out, SaveDataId id) {

libraries/libstratosphere/source/hos/hos_stratosphere_api.cpp

@@ -48,6 +48,7 @@ namespace ams::hos {
 
     }
 
+    bool IsUnitTestProgramForSetVersion();
     void InitializeVersionInternal(bool allow_approximate);
 
     void InitializeForStratosphere() {
@@ -58,7 +59,7 @@ namespace ams::hos {
         hos::InitializeVersionInternal(CanAllowTemporaryApproximateVersion());
 
         /* Check that we're running under mesosphere. */
-        AMS_ABORT_UNLESS(svc::IsKernelMesosphere());
+        AMS_ABORT_UNLESS(IsUnitTestProgramForSetVersion() || svc::IsKernelMesosphere());
     }
 
 }

libraries/libstratosphere/source/hos/hos_version_api.cpp

@@ -61,51 +61,61 @@ namespace ams::hos {
 
     }
 
+    bool IsUnitTestProgramForSetVersion();
+
     void InitializeVersionInternal(bool allow_approximate) {
-        /* Get the current (and previous approximation of) target firmware. */
-        hos::Version prev, current;
-        bool has_prev = false;
-        {
-            /* Acquire exclusive access to set hos version. */
-            std::scoped_lock lk(g_hos_init_lock);
+        hos::Version current = hos::Version_Current;
 
-            /* Save the previous value of g_hos_version. */
-            prev     = g_hos_version;
-            has_prev = g_set_hos_version;
-
-            /* Set hos version = exosphere api version target firmware. */
-            g_hos_version = static_cast<hos::Version>(GetExosphereApiInfo(allow_approximate).GetTargetFirmware());
-
-            /* Save the current value of g_hos_version. */
-            current = g_hos_version;
-
-            /* Note that we've set a previous hos version. */
+        /* If we're unit testing, just set the version and move on. */
+        if (IsUnitTestProgramForSetVersion()) {
+            g_hos_version     = hos::Version_Current;
             g_set_hos_version = true;
-        }
+        } else {
+            /* Get the current (and previous approximation of) target firmware. */
+            hos::Version prev;
+            bool has_prev = false;
+            {
+                /* Acquire exclusive access to set hos version. */
+                std::scoped_lock lk(g_hos_init_lock);
 
-        /* Ensure that this is a hos version we can sanely *try* to run. */
-        /* To be friendly, we will only require that we recognize the major and minor versions. */
-        /* We can consider only recognizing major in the future, but micro seems safe to ignore as */
-        /* there are no breaking IPC changes in minor updates. */
-        {
-            constexpr u32 MaxMajor = (static_cast<u32>(hos::Version_Max) >> 24) & 0xFF;
-            constexpr u32 MaxMinor = (static_cast<u32>(hos::Version_Max) >> 16) & 0xFF;
+                /* Save the previous value of g_hos_version. */
+                prev     = g_hos_version;
+                has_prev = g_set_hos_version;
 
-            const u32 major = (static_cast<u32>(current) >> 24) & 0xFF;
-            const u32 minor = (static_cast<u32>(current) >> 16) & 0xFF;
+                /* Set hos version = exosphere api version target firmware. */
+                g_hos_version = static_cast<hos::Version>(GetExosphereApiInfo(allow_approximate).GetTargetFirmware());
 
-            const bool is_safely_tryable_version = (current <= hos::Version_Max) || (major == MaxMajor && minor <= MaxMinor);
-            AMS_ABORT_UNLESS(is_safely_tryable_version);
-        }
+                /* Save the current value of g_hos_version. */
+                current = g_hos_version;
 
-        /* Ensure that this is a hos version compatible with previous approximations. */
-        if (has_prev) {
-            AMS_ABORT_UNLESS(current >= prev);
+                /* Note that we've set a previous hos version. */
+                g_set_hos_version = true;
+            }
 
-            const u32 current_major = (static_cast<u32>(current) >> 24) & 0xFF;
-            const u32 prev_major    = (static_cast<u32>(prev) >> 24) & 0xFF;
+            /* Ensure that this is a hos version we can sanely *try* to run. */
+            /* To be friendly, we will only require that we recognize the major and minor versions. */
+            /* We can consider only recognizing major in the future, but micro seems safe to ignore as */
+            /* there are no breaking IPC changes in minor updates. */
+            {
+                constexpr u32 MaxMajor = (static_cast<u32>(hos::Version_Max) >> 24) & 0xFF;
+                constexpr u32 MaxMinor = (static_cast<u32>(hos::Version_Max) >> 16) & 0xFF;
 
-            AMS_ABORT_UNLESS(current_major == prev_major);
+                const u32 major = (static_cast<u32>(current) >> 24) & 0xFF;
+                const u32 minor = (static_cast<u32>(current) >> 16) & 0xFF;
+
+                const bool is_safely_tryable_version = (current <= hos::Version_Max) || (major == MaxMajor && minor <= MaxMinor);
+                AMS_ABORT_UNLESS(is_safely_tryable_version);
+            }
+
+            /* Ensure that this is a hos version compatible with previous approximations. */
+            if (has_prev) {
+                AMS_ABORT_UNLESS(current >= prev);
+
+                const u32 current_major = (static_cast<u32>(current) >> 24) & 0xFF;
+                const u32 prev_major    = (static_cast<u32>(prev) >> 24) & 0xFF;
+
+                AMS_ABORT_UNLESS(current_major == prev_major);
+            }
         }
 
         /* Set the version for libnx. */

libraries/libstratosphere/source/hos/hos_version_api_weak_for_unit_test.cpp

@@ -0,0 +1,24 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <stratosphere.hpp>
+
+namespace ams::hos {
+
+    WEAK_SYMBOL bool IsUnitTestProgramForSetVersion() {
+        return false;
+    }
+
+}

libraries/libstratosphere/source/init/init_libnx_shim.os.horizon.cpp

@@ -20,6 +20,8 @@ extern "C" {
     constinit u32 __nx_fs_num_sessions = 1;
     constinit u32 __nx_applet_type = AppletType_None;
 
+    constinit bool __nx_fsdev_support_cwd = false;
+
     extern int __system_argc;
     extern char** __system_argv;
 
@@ -38,6 +40,8 @@ namespace ams {
 
     namespace init {
 
+        void InitializeSystemModuleBeforeConstructors();
+
         void InitializeSystemModule();
         void FinalizeSystemModule();
 
@@ -66,6 +70,9 @@ extern "C" void __libnx_initheap(void) {
 extern "C" void __appInit(void) {
     /* The very first thing all stratosphere code must do is initialize the os library. */
     ::ams::hos::InitializeForStratosphere();
+
+    /* Perform pre-C++ constructor init. */
+    ::ams::init::InitializeSystemModuleBeforeConstructors();
 }
 
 extern "C" void __appExit(void) {

libraries/libstratosphere/source/init/init_system_module.cpp

@@ -17,6 +17,10 @@
 
 namespace ams::init {
 
+    WEAK_SYMBOL void InitializeSystemModuleBeforeConstructors() {
+        /* This should only be used in exceptional circumstances. */
+    }
+
     WEAK_SYMBOL void InitializeSystemModule() {
         /* TODO: What should we do here, if anything? */
         /* Nintendo does nndiagStartup(); nn::diag::InitializeSystemProcessAbortObserver(); */

libraries/libstratosphere/source/os/impl/os_thread_manager_impl.os.horizon.cpp

@@ -52,6 +52,26 @@ namespace ams::os::impl {
         /* Get the thread impl object from libnx. */
         ThreadImpl *thread_impl = ::threadGetSelf();
 
+        /* Hack around libnx's main thread, to ensure stratosphere thread type consistency. */
+        {
+            auto *tlr = reinterpret_cast<uintptr_t *>(svc::GetThreadLocalRegion());
+            for (size_t i = sizeof(svc::ThreadLocalRegion) / sizeof(uintptr_t); i > 0; --i) {
+                if (auto *candidate = reinterpret_cast<ThreadImpl *>(tlr[i - 1]); candidate == thread_impl) {
+                    ThreadImpl *embedded_thread = std::addressof(main_thread->thread_impl_storage);
+
+                    *embedded_thread = *thread_impl;
+
+                    if (embedded_thread->next) {
+                        embedded_thread->next->prev_next = std::addressof(embedded_thread->next);
+                    }
+
+                    thread_impl = embedded_thread;
+                    tlr[i-1]    = reinterpret_cast<uintptr_t>(thread_impl);
+                    break;
+                }
+            }
+        }
+
         /* Get the thread priority. */
         s32 horizon_priority;
         R_ABORT_UNLESS(svc::GetThreadPriority(std::addressof(horizon_priority), thread_impl->handle));

libraries/libstratosphere/source/osdbg/impl/osdbg_thread_local_region.os.horizon.hpp

@@ -54,7 +54,7 @@ namespace ams::osdbg::impl {
     static_assert(AMS_OFFSETOF(ThreadLocalRegionIlp32, tls) == 0x1C0);
 
     struct LibnxThreadVars {
-        static constexpr u32 Magic = util::FourCC<'!','T','V','$'>::Code;
+        static constexpr u32 Magic = util::ReverseFourCC<'!','T','V','$'>::Code;
 
         u32 magic;
         ::Handle handle;
@@ -69,11 +69,11 @@ namespace ams::osdbg::impl {
         volatile u16 disable_counter;
         volatile u16 interrupt_flag;
         u32 reserved0;
-        u64 tls[(0x1E0 - 0x108) / sizeof(u64)];
+        u64 tls[(0x200 - sizeof(LibnxThreadVars) - 0x108) / sizeof(u64)];
         LibnxThreadVars thread_vars;
     };
     static_assert(sizeof(ThreadLocalRegionLibnx) == sizeof(svc::ThreadLocalRegion));
-    static_assert(AMS_OFFSETOF(ThreadLocalRegionLibnx, thread_vars) == 0x1E0);
+    static_assert(AMS_OFFSETOF(ThreadLocalRegionLibnx, thread_vars) == 0x200 - sizeof(LibnxThreadVars));
 
     struct LibnxThreadEntryArgs {
         u64 t;

libraries/libstratosphere/source/osdbg/osdbg_thread.cpp

@@ -50,9 +50,9 @@ namespace ams::osdbg {
         } else {
             /* Special-case libnx threads. */
             if (thread_info->_thread_type_type == ThreadTypeType_Libnx) {
-                util::TSNPrintf(dst, os::ThreadNameLengthMax, "libnx Thread_0x%p", reinterpret_cast<void *>(thread_info->_thread_type));
+                util::TSNPrintf(dst, os::ThreadNameLengthMax, "libnx Thread_0x%010lx", reinterpret_cast<uintptr_t>(thread_info->_thread_type));
             } else {
-                util::TSNPrintf(dst, os::ThreadNameLengthMax, "Thread_0x%p", reinterpret_cast<void *>(thread_info->_thread_type));
+                util::TSNPrintf(dst, os::ThreadNameLengthMax, "Thread_0x%010lx", reinterpret_cast<uintptr_t>(thread_info->_thread_type));
             }
 
             return ResultSuccess();

libraries/libstratosphere/source/socket/impl/socket_api.hpp

@@ -41,6 +41,7 @@ namespace ams::socket::impl {
 
     s32 Shutdown(s32 desc, ShutdownMethod how);
 
+    s32 Socket(Family domain, Type type, Protocol protocol);
     s32 SocketExempt(Family domain, Type type, Protocol protocol);
 
     s32 Accept(s32 desc, SockAddr *out_address, SockLenT *out_addr_len);

libraries/libstratosphere/source/socket/impl/socket_api.os.horizon.cpp

@@ -206,6 +206,8 @@ namespace ams::socket::impl {
 
             /* TODO: socket::resolver::EnableResolverCalls()? Not necessary in our case (htc), but consider calling it. */
 
+            g_initialized = true;
+
             return ResultSuccess();
         }
 
@@ -422,6 +424,22 @@ namespace ams::socket::impl {
         return result;
     }
 
+    s32 Socket(Family domain, Type type, Protocol protocol) {
+        /* Check pre-conditions. */
+        AMS_ABORT_UNLESS(IsInitialized());
+
+        /* Perform the call. */
+        Errno error = Errno::ESuccess;
+        int result  = ::bsdSocket(static_cast<int>(domain), static_cast<int>(type), static_cast<int>(protocol));
+        TranslateResultToBsdError(error, result);
+
+        if (result < 0) {
+            socket::impl::SetLastError(error);
+        }
+
+        return result;
+    }
+
     s32 SocketExempt(Family domain, Type type, Protocol protocol) {
         /* Check pre-conditions. */
         AMS_ABORT_UNLESS(IsInitialized());

libraries/libstratosphere/source/socket/socket_api.cpp

@@ -74,6 +74,10 @@ namespace ams::socket {
         return impl::Shutdown(desc, how);
     }
 
+    s32 Socket(Family domain, Type type, Protocol protocol) {
+        return impl::Socket(domain, type, protocol);
+    }
+
     s32 SocketExempt(Family domain, Type type, Protocol protocol) {
         return impl::SocketExempt(domain, type, protocol);
     }

libraries/libstratosphere/source/spl/impl/spl_api_impl.cpp

@@ -729,9 +729,10 @@ namespace ams::spl::impl {
         return ResultSuccess();
     }
 
-    Result TestAesKeySlot(s32 *out_index, s32 keyslot) {
+    Result TestAesKeySlot(s32 *out_index, bool *out_virtual, s32 keyslot) {
         if (g_is_physical_keyslot_allowed && IsPhysicalAesKeySlot(keyslot)) {
-            *out_index = keyslot;
+            *out_index   = keyslot;
+            *out_virtual = false;
             return ResultSuccess();
         }
 
@@ -740,7 +741,8 @@ namespace ams::spl::impl {
         const s32 index = GetVirtualAesKeySlotIndex(keyslot);
         R_UNLESS(g_is_aes_keyslot_allocated[index], spl::ResultInvalidKeySlot());
 
-        *out_index = index;
+        *out_index   = index;
+        *out_virtual = true;
         return ResultSuccess();
     }
 

libraries/libstratosphere/source/sprofile/srv/sprofile_srv_api.cpp

@@ -34,7 +34,7 @@ namespace ams::sprofile::srv {
         constexpr const sm::ServiceName ServiceNameForSystemProcess = sm::ServiceName::Encode("sprof:sp");
 
         constexpr inline size_t BgAgentSessionCountMax       = 2;
-        constexpr inline size_t SystemProcessSessionCountMax = 5;
+        constexpr inline size_t SystemProcessSessionCountMax = 10;
 
         constexpr inline size_t SessionCountMax = BgAgentSessionCountMax + SystemProcessSessionCountMax;
 
@@ -42,8 +42,8 @@ namespace ams::sprofile::srv {
 
         struct ServerManagerOptions {
             static constexpr size_t PointerBufferSize   = 0x0;
-            static constexpr size_t MaxDomains          = SessionCountMax; /* NOTE: Official is 3 */
-            static constexpr size_t MaxDomainObjects    = 16;              /* NOTE: Official is 8 */
+            static constexpr size_t MaxDomains          = SessionCountMax; /* NOTE: Official is 9 */
+            static constexpr size_t MaxDomainObjects    = 16;              /* NOTE: Official is 14 */
             static constexpr bool CanDeferInvokeRequest = false;
             static constexpr bool CanManageMitmServers  = false;
         };

libraries/libstratosphere/source/sprofile/srv/sprofile_srv_i_profile_importer.hpp

@@ -18,8 +18,8 @@
 #include "sprofile_srv_types.hpp"
 
 #define AMS_SPROFILE_I_PROFILE_IMPORTER_INTERFACE_INFO(C, H) \
-    AMS_SF_METHOD_INFO(C, H, 0, Result, ImportProfile,  (const sprofile::srv::ProfileDataForImportData &data),         (data)) \
-    AMS_SF_METHOD_INFO(C, H, 1, Result, Commit,         (),                                                            ())     \
-    AMS_SF_METHOD_INFO(C, H, 0, Result, ImportMetadata, (const sprofile::srv::ProfileMetadataForImportMetadata &data), (data)) \
+    AMS_SF_METHOD_INFO(C, H, 0, Result, ImportProfile,  (const sprofile::srv::ProfileDataForImportData &import),         (import)) \
+    AMS_SF_METHOD_INFO(C, H, 1, Result, Commit,         (),                                                              ())     \
+    AMS_SF_METHOD_INFO(C, H, 2, Result, ImportMetadata, (const sprofile::srv::ProfileMetadataForImportMetadata &import), (import)) \
 
 AMS_SF_DEFINE_INTERFACE(ams::sprofile, IProfileImporter, AMS_SPROFILE_I_PROFILE_IMPORTER_INTERFACE_INFO)