kern: simplify random bitmap selection to match latest Nintendo logic

This is an optimization that saves the most common type of virtual call in the kernel (DynamicCast)
by storing class token as a member, rather than getting it via virtual call every time.

This does not currently cost any memory space on 64-bit targets, due to pre-existing padding space.

This optimization can be turned off via a compile-time flag for accuracy.

docs/building.md

@@ -7,6 +7,7 @@ Building Atmosphère is a very straightforward process that relies almost exclus
 + [Python 2](https://www.python.org) (Python 3 may work as well, but this is not guaranteed)
 + [LZ4](https://pypi.org/project/lz4)
 + [PyCryptodome](https://pypi.org/project/pycryptodome) (optional)
++ [hactool](https://github.com/SciresM/hactool)
 
 ## Instructions
 1. Follow the guide located [here](https://devkitpro.org/wiki/Getting_Started) to install and configure all the tools necessary for the build process.
@@ -17,6 +18,7 @@ Building Atmosphère is a very straightforward process that relies almost exclus
     + `switch-libjpeg-turbo`
     + `devkitARM`
     + `devkitarm-rules`
+    + `hactool`
 
 3. Install the following library via python's package manager `pip`, required by [exosphere](components/exosphere.md):
     + `lz4`

docs/changelog.md

@@ -1,4 +1,38 @@
 # Changelog
+## 1.2.0
++ `boot` was updated to reflect the latest official behavior for display/battery management.
+  + This should fix any issues that might result from running older releases on the OLED model, if you're somehow in a position to do so.
++ The "target firmware" system was changed to allow the bootloader to specify an approximation, rather than the true target firmware.
+  + Previously we expected compliant bootloaders to inspect SYSTEM:/ to determine the specific target firmware.
+  + Now, we only require an approximate version, with major version == true major version and approximate version <= true version.
+  + This greatly simplifies bootloader requirements, and correspondingly all code for accessing SYSTEM has been removed from fusee.
+    + This should result in a substantial speedup when booting emummc with fusee, as SYSTEM accesses were the most expensive thing done previously.
+  + This should resolve any inconsistency in firmware detection when booting via fusee vs hekate.
+  + This should also improve our compatibility with micro firmware releases, making it more likely that atmosphere "just works" if nothing important has changed.
++ Dynamic resource limit determination logic was implemented in `pm` to match latest official behavior.
+  + This greatly simplifies/makes consistent the resource limits on older firmwares, as well.
++ An enormous amount of refactoring was performed under the hood, including:
+  + **Please Note**: If you are a developer who uses Atmosphere-libs, a number of changes here are breaking.
+    + Feel free to contact SciresM#0524 for help updating your program.
+  + The OS namespace had many primitives implemented/made more accurate.
+  + Since mesosphere is now always-on, os::LightEvent (which required newer SVCs) is now globally usable (and used by stratosphere where relevant).
+  + Assertions are now true no-ops when building for release.
+  + Stratosphere is now built with -Wextra/-Werror.
+  + Most "common" logic in system module main.cpp files was moved into libstratosphere.
+    + **Please Note**: main.cpp files for prior atmosphere-libs will no longer work, for a really large number of reasons.
+  + A number of longstanding code style issues were corrected.
+  + Mesosphere now uses util::BitFlagSet for SVC permissions.
+  + Mesosphere now puts its relocation table inside .bss, which allows that memory to be reclaimed after relocations are performed.
+    + These changes save ~16KB of memory in the kernel, all said and done.
+  + A number of locations in stratosphere where memory could be saved were spotted and taken advantage of, leading to ~150-200KB of saved memory.
+  + The `spl` and `loader` system module was refactored to better reflect official logic.
+  + `sf` ipc server code was updated to only emit mitm/defer logic when that logic is actually required somewhere in process.
+  + `tipc` ipc server code was updated to reflect changes to official logic made in 13.0.0.
+  + Many, many other minor changes, please talk to SciresM#0524 or read the relevant commits if you want to know more.
++ A number of minor issues were fixed, including:
+  + Mesosphere's handling of SVC permissions on thread pin/unpin was updated to reflect official kernel behavior.
+  + util::CountTrailingZeroes() was fixed to calculate the correct value when used at compile-time.
++ General system stability improvements to enhance the user's experience.
 ## 1.1.1
 + A bug was fixed which caused some memory to leak when launching a game with mods enabled, eventually causing a crash after enough game launches without rebooting.
 + General system stability improvements to enhance the user's experience.

exosphere/loader_stub/source/secmon_loader_uncompress.cpp

@@ -22,14 +22,14 @@ namespace ams::secmon::loader {
 
         class Lz4Uncompressor {
             private:
-                const u8 *src;
-                size_t src_size;
-                size_t src_offset;
-                u8 *dst;
-                size_t dst_size;
-                size_t dst_offset;
+                const u8 *m_src;
+                size_t m_src_size;
+                size_t m_src_offset;
+                u8 *m_dst;
+                size_t m_dst_size;
+                size_t m_dst_offset;
             public:
-                Lz4Uncompressor(void *dst, size_t dst_size, const void *src, size_t src_size) : src(static_cast<const u8 *>(src)), src_size(src_size), src_offset(0), dst(static_cast<u8 *>(dst)), dst_size(dst_size), dst_offset(0) {
+                Lz4Uncompressor(void *dst, size_t dst_size, const void *src, size_t src_size) : m_src(static_cast<const u8 *>(src)), m_src_size(src_size), m_src_offset(0), m_dst(static_cast<u8 *>(dst)), m_dst_size(dst_size), m_dst_offset(0) {
                     /* ... */
                 }
 
@@ -42,7 +42,7 @@ namespace ams::secmon::loader {
                         this->Copy(this->GetCopySize(control >> 4));
 
                         /* If we've exceeded size, we're done. */
-                        if (this->src_offset >= this->src_size) {
+                        if (m_src_offset >= m_src_size) {
                             break;
                         }
 
@@ -55,21 +55,21 @@ namespace ams::secmon::loader {
                         const size_t wide_copy_size = this->GetCopySize(control & 0xF);
 
                         /* Copy bytes. */
-                        const size_t end_offset = this->dst_offset + wide_copy_size + 4;
-                        for (size_t cur_offset = this->dst_offset; cur_offset < end_offset; this->dst_offset = (++cur_offset)) {
+                        const size_t end_offset = m_dst_offset + wide_copy_size + 4;
+                        for (size_t cur_offset = m_dst_offset; cur_offset < end_offset; m_dst_offset = (++cur_offset)) {
                             AMS_ABORT_UNLESS(wide_offset <= cur_offset);
 
-                            this->dst[cur_offset] = this->dst[cur_offset - wide_offset];
+                            m_dst[cur_offset] = m_dst[cur_offset - wide_offset];
                         }
                     }
                 }
             private:
                 u8 ReadByte() {
-                    return this->src[this->src_offset++];
+                    return m_src[m_src_offset++];
                 }
 
                 bool CanRead() const {
-                    return this->src_offset < this->src_size;
+                    return m_src_offset < m_src_size;
                 }
 
                 size_t GetCopySize(u8 control) {
@@ -87,9 +87,9 @@ namespace ams::secmon::loader {
                 }
 
                 void Copy(size_t size) {
-                    __builtin_memcpy(this->dst + this->dst_offset, this->src + this->src_offset, size);
-                    this->dst_offset += size;
-                    this->src_offset += size;
+                    __builtin_memcpy(m_dst + m_dst_offset, m_src + m_src_offset, size);
+                    m_dst_offset += size;
+                    m_src_offset += size;
                 }
         };
 

exosphere/mariko_fatal/source/fatal_device_page_table.cpp

@@ -76,10 +76,10 @@ namespace ams::secmon::fatal {
                     Bit_Readable  = 31,
                 };
             private:
-                u32 value;
+                u32 m_value;
             protected:
                 constexpr ALWAYS_INLINE u32 SelectBit(Bit n) const {
-                    return (this->value & (1u << n));
+                    return (m_value & (1u << n));
                 }
 
                 constexpr ALWAYS_INLINE bool GetBit(Bit n) const {
@@ -97,7 +97,7 @@ namespace ams::secmon::fatal {
                 ALWAYS_INLINE void SetValue(u32 v) {
                     /* Prevent re-ordering around entry modifications. */
                     __asm__ __volatile__("" ::: "memory");
-                    this->value = v;
+                    m_value = v;
                     __asm__ __volatile__("" ::: "memory");
                 }
             public:
@@ -112,7 +112,7 @@ namespace ams::secmon::fatal {
 
                 constexpr ALWAYS_INLINE u32 GetAttributes() const { return this->SelectBit(Bit_NonSecure) | this->SelectBit(Bit_Writeable) | this->SelectBit(Bit_Readable); }
 
-                constexpr ALWAYS_INLINE dd::PhysicalAddress GetPhysicalAddress() const { return (static_cast<u64>(this->value) << DevicePageBits) & PhysicalAddressMask; }
+                constexpr ALWAYS_INLINE dd::PhysicalAddress GetPhysicalAddress() const { return (static_cast<u64>(m_value) << DevicePageBits) & PhysicalAddressMask; }
 
                 ALWAYS_INLINE void Invalidate() { this->SetValue(0); }
         };

exosphere/mariko_fatal/source/fatal_display.cpp

@@ -221,7 +221,7 @@ namespace ams::secmon::fatal {
                         const int prefix_len = std::strlen(automatic_backups_prefix);
 
                         for (size_t i = 0; i + prefix_len < f_ctx->stack_dump_size; ++i) {
-                            if (std::memcmp(&f_ctx->stack_dump[i], automatic_backups_prefix, prefix_len) == 0) {
+                            if (std::memcmp(f_ctx->stack_dump + i, automatic_backups_prefix, prefix_len) == 0) {
                                 suggestion = "The atmosphere directory may improperly have archive bits set.\n"
                                              "Please try running an archive bit fixer tool (for example, the one in Hekate).\n";
                                 break;

exosphere/mariko_fatal/source/fs/fatal_fs_api.hpp

@@ -28,7 +28,7 @@ namespace ams::fs {
     };
 
     struct ReadOption {
-        u32 value;
+        u32 _value;
 
         static const ReadOption None;
     };
@@ -36,7 +36,7 @@ namespace ams::fs {
     inline constexpr const ReadOption ReadOption::None = {0};
 
     inline constexpr bool operator==(const ReadOption &lhs, const ReadOption &rhs) {
-        return lhs.value == rhs.value;
+        return lhs._value == rhs._value;
     }
 
     inline constexpr bool operator!=(const ReadOption &lhs, const ReadOption &rhs) {
@@ -46,10 +46,10 @@ namespace ams::fs {
     static_assert(util::is_pod<ReadOption>::value && sizeof(ReadOption) == sizeof(u32));
 
     struct WriteOption {
-        u32 value;
+        u32 _value;
 
         constexpr inline bool HasFlushFlag() const {
-            return this->value & 1;
+            return _value & 1;
         }
 
         static const WriteOption None;
@@ -60,7 +60,7 @@ namespace ams::fs {
     inline constexpr const WriteOption WriteOption::Flush = {1};
 
     inline constexpr bool operator==(const WriteOption &lhs, const WriteOption &rhs) {
-        return lhs.value == rhs.value;
+        return lhs._value == rhs._value;
     }
 
     inline constexpr bool operator!=(const WriteOption &lhs, const WriteOption &rhs) {

exosphere/program/source/secmon_exception_handler.cpp

@@ -22,7 +22,7 @@ namespace ams::secmon {
 
         constexpr inline uintptr_t PMC = MemoryRegionVirtualDevicePmc.GetAddress();
 
-        constinit std::atomic_bool g_is_locked = false;
+        constinit util::Atomic<bool> g_is_locked = false;
 
     }
 
@@ -72,7 +72,7 @@ namespace ams::secmon {
         }
 
         /* Acquire exclusive access to exception handling logic. */
-        if (!g_is_locked.exchange(true)) {
+        if (!g_is_locked.Exchange(true)) {
             /* Invoke the exception handler impl. */
             ExceptionHandlerImpl(lr, sp);
 

exosphere/program/source/secmon_page_mapper.cpp

@@ -23,16 +23,16 @@ namespace ams::secmon {
 
         void *PageMapperImpl::GetPointerTo(uintptr_t phys, size_t size) const {
             /* Ensure we stay within the page. */
-            if (util::AlignDown(phys, 4_KB) != this->physical_address) {
+            if (util::AlignDown(phys, 4_KB) != m_physical_address) {
                 return nullptr;
             }
             if (size != 0) {
-                if (util::AlignDown(phys + size - 1, 4_KB) != this->physical_address) {
+                if (util::AlignDown(phys + size - 1, 4_KB) != m_physical_address) {
                     return nullptr;
                 }
             }
 
-            return reinterpret_cast<void *>(phys + (this->virtual_address - this->physical_address));
+            return reinterpret_cast<void *>(phys + (m_virtual_address - m_physical_address));
         }
 
         bool PageMapperImpl::CopyToMapping(uintptr_t dst_phys, const void *src, size_t size) const {

exosphere/program/source/secmon_page_mapper.hpp

@@ -22,10 +22,10 @@ namespace ams::secmon {
 
         class PageMapperImpl {
             private:
-                uintptr_t physical_address;
-                uintptr_t virtual_address;
+                uintptr_t m_physical_address;
+                uintptr_t m_virtual_address;
             public:
-                constexpr PageMapperImpl(uintptr_t phys) : physical_address(util::AlignDown(phys, 4_KB)), virtual_address() { /* ... */ }
+                constexpr PageMapperImpl(uintptr_t phys) : m_physical_address(util::AlignDown(phys, 4_KB)), m_virtual_address() { /* ... */ }
 
                 void *GetPointerTo(uintptr_t phys, size_t size) const;
 
@@ -37,14 +37,14 @@ namespace ams::secmon {
 
                 template<auto F>
                 bool MapImpl() {
-                    this->virtual_address = F(this->physical_address);
-                    return this->virtual_address != 0;
+                    m_virtual_address = F(m_physical_address);
+                    return m_virtual_address != 0;
                 }
 
                 template<auto F>
                 void UnmapImpl() {
                     F();
-                    this->virtual_address = 0;
+                    m_virtual_address = 0;
                 }
         };
 

exosphere/program/source/smc/secmon_smc_aes.cpp

@@ -417,7 +417,7 @@ namespace ams::secmon::smc {
                 case CipherMode_CbcDecryption: se::DecryptAes128CbcAsync(output_address, slot, input_address, size, iv, sizeof(iv), SecurityEngineDoneHandler); break;
                 case CipherMode_Ctr:           se::ComputeAes128CtrAsync(output_address, slot, input_address, size, iv, sizeof(iv), SecurityEngineDoneHandler); break;
                 case CipherMode_Cmac:
-                    return SmcResult::NotImplemented;
+                    return SmcResult::NotSupported;
                 default:
                     return SmcResult::InvalidArgument;
             }
@@ -765,8 +765,8 @@ namespace ams::secmon::smc {
             const auto which = static_cast<SecureData>(args.r[1]);
 
             /* Validate arguments/conditions. */
-            SMC_R_UNLESS(fuse::GetPatchVersion() < fuse::PatchVersion_Odnx02A2, NotImplemented);
-            SMC_R_UNLESS(which < SecureData_Count,                              NotImplemented);
+            SMC_R_UNLESS(fuse::GetPatchVersion() < fuse::PatchVersion_Odnx02A2, NotSupported);
+            SMC_R_UNLESS(which < SecureData_Count,                              NotSupported);
 
             /* Use a temporary buffer. */
             u8 secure_data[AesKeySize];

exosphere/program/source/smc/secmon_smc_common.hpp

@@ -20,7 +20,7 @@ namespace ams::secmon::smc {
 
     enum class SmcResult : u32 {
         Success               = 0,
-        NotImplemented        = 1,
+        NotSupported          = 1,
         InvalidArgument       = 2,
         Busy                  = 3,
         NoAsyncOperation      = 4,

exosphere/program/source/smc/secmon_smc_info.cpp

@@ -164,6 +164,7 @@ namespace ams::secmon::smc {
         }
 
         constinit u64 g_payload_address = 0;
+        constinit bool g_set_true_target_firmware = false;
 
         SmcResult GetConfig(SmcArguments &args, bool kern) {
             switch (static_cast<ConfigItem>(args.r[1])) {
@@ -213,8 +214,8 @@ namespace ams::secmon::smc {
                 case ConfigItem::IsChargerHiZModeEnabled:
                     args.r[1] = IsChargerHiZModeEnabled();
                     break;
-                case ConfigItem::QuestState:
-                    args.r[1] = fuse::GetQuestState();
+                case ConfigItem::RetailInteractiveDisplayState:
+                    args.r[1] = fuse::GetRetailInteractiveDisplayState();
                     break;
                 case ConfigItem::RegulatorType:
                     args.r[1] = fuse::GetRegulator();
@@ -240,11 +241,15 @@ namespace ams::secmon::smc {
                     break;
                 case ConfigItem::ExosphereApiVersion:
                     /* Get information about the current exosphere version. */
-                    args.r[1] = (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MAJOR & 0xFF) << 56) |
-                                (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MINOR & 0xFF) << 48) |
-                                (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MICRO & 0xFF) << 40) |
-                                (static_cast<u64>(GetKeyGeneration())                      << 32) |
-                                (static_cast<u64>(GetTargetFirmware())                     <<  0);
+                    if (kern || g_set_true_target_firmware) {
+                        args.r[1] = (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MAJOR & 0xFF) << 56) |
+                                    (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MINOR & 0xFF) << 48) |
+                                    (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MICRO & 0xFF) << 40) |
+                                    (static_cast<u64>(GetKeyGeneration())                      << 32) |
+                                    (static_cast<u64>(GetTargetFirmware())                     <<  0);
+                    } else {
+                        return SmcResult::NotInitialized;
+                    }
                     break;
                 case ConfigItem::ExosphereNeedsReboot:
                     /* We are executing, so we aren't in the process of rebooting. */
@@ -297,6 +302,18 @@ namespace ams::secmon::smc {
                                 (static_cast<u64>(ATMOSPHERE_SUPPORTED_HOS_VERSION_MINOR & 0xFF) << 16) |
                                 (static_cast<u64>(ATMOSPHERE_SUPPORTED_HOS_VERSION_MICRO & 0xFF) <<  8);
                     break;
+                case ConfigItem::ExosphereApproximateApiVersion:
+                    /* Get information about the current exosphere version. */
+                    if (!g_set_true_target_firmware) {
+                        args.r[1] = (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MAJOR & 0xFF) << 56) |
+                                    (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MINOR & 0xFF) << 48) |
+                                    (static_cast<u64>(ATMOSPHERE_RELEASE_VERSION_MICRO & 0xFF) << 40) |
+                                    (static_cast<u64>(GetKeyGeneration())                      << 32) |
+                                    (static_cast<u64>(GetTargetFirmware())                     <<  0);
+                    } else {
+                        return SmcResult::Busy;
+                    }
+                    break;
                 default:
                     return SmcResult::InvalidArgument;
             }
@@ -312,6 +329,14 @@ namespace ams::secmon::smc {
                     /* Configure the HiZ mode. */
                     SetChargerHiZModeEnabled(static_cast<bool>(args.r[3]));
                     break;
+                case ConfigItem::ExosphereApiVersion:
+                    if (!g_set_true_target_firmware) {
+                        ::ams::secmon::impl::SetTargetFirmware(static_cast<ams::TargetFirmware>(args.r[3] & 0xFFFFFFFF));
+                        g_set_true_target_firmware = true;
+                    } else {
+                        return SmcResult::Busy;
+                    }
+                    break;
                 case ConfigItem::ExosphereNeedsReboot:
                     if (soc_type == fuse::SocType_Erista) {
                         switch (static_cast<UserRebootType>(args.r[3])) {
@@ -345,7 +370,7 @@ namespace ams::secmon::smc {
                             PerformUserShutDown();
                         }
                     } else /* if (soc_type == fuse::SocType_Mariko) */ {
-                        return SmcResult::NotImplemented;
+                        return SmcResult::NotSupported;
                     }
                     break;
                 case ConfigItem::ExospherePayloadAddress:
@@ -389,7 +414,7 @@ namespace ams::secmon::smc {
 
         /* Validate arguments. */
         /* NOTE: In the future, configuration for non-NAND storage may be implemented. */
-        SMC_R_UNLESS(mmc == EmummcMmc_Nand,                             NotImplemented);
+        SMC_R_UNLESS(mmc == EmummcMmc_Nand,                            NotSupported);
         SMC_R_UNLESS(user_offset + 2 * sizeof(EmummcFilePath) <= 4_KB, InvalidArgument);
 
         /* Get the emummc config. */

exosphere/program/source/smc/secmon_smc_info.hpp

@@ -34,24 +34,25 @@ namespace ams::secmon::smc {
         IsDevelopmentFunctionEnabled  = 11,
         KernelConfiguration           = 12,
         IsChargerHiZModeEnabled       = 13,
-        QuestState                    = 14,
+        RetailInteractiveDisplayState = 14,
         RegulatorType                 = 15,
         DeviceUniqueKeyGeneration     = 16,
         Package2Hash                  = 17,
 
         /* Extension config items for exosphere. */
-        ExosphereApiVersion          = 65000,
-        ExosphereNeedsReboot         = 65001,
-        ExosphereNeedsShutdown       = 65002,
-        ExosphereGitCommitHash       = 65003,
-        ExosphereHasRcmBugPatch      = 65004,
-        ExosphereBlankProdInfo       = 65005,
-        ExosphereAllowCalWrites      = 65006,
-        ExosphereEmummcType          = 65007,
-        ExospherePayloadAddress      = 65008,
-        ExosphereLogConfiguration    = 65009,
-        ExosphereForceEnableUsb30    = 65010,
-        ExosphereSupportedHosVersion = 65011,
+        ExosphereApiVersion            = 65000,
+        ExosphereNeedsReboot           = 65001,
+        ExosphereNeedsShutdown         = 65002,
+        ExosphereGitCommitHash         = 65003,
+        ExosphereHasRcmBugPatch        = 65004,
+        ExosphereBlankProdInfo         = 65005,
+        ExosphereAllowCalWrites        = 65006,
+        ExosphereEmummcType            = 65007,
+        ExospherePayloadAddress        = 65008,
+        ExosphereLogConfiguration      = 65009,
+        ExosphereForceEnableUsb30      = 65010,
+        ExosphereSupportedHosVersion   = 65011,
+        ExosphereApproximateApiVersion = 65012,
     };
 
     SmcResult SmcGetConfigUser(SmcArguments &args);

exosphere/program/source/smc/secmon_smc_memory_access.cpp

@@ -70,7 +70,7 @@ namespace ams::secmon::smc {
     SmcResult SmcWriteAddress(SmcArguments &args) {
         /* NOTE: This smc was deprecated in Atmosphère 0.13.0. */
         AMS_UNUSED(args);
-        return SmcResult::NotImplemented;
+        return SmcResult::NotSupported;
     }
 
 }

exosphere/program/source/smc/secmon_smc_rsa.cpp

@@ -55,31 +55,31 @@ namespace ams::secmon::smc {
 
         class PrepareEsDeviceUniqueKeyAsyncArguments {
             private:
-                int generation;
-                EsCommonKeyType type;
-                u8 label_digest[crypto::Sha256Generator::HashSize];
+                int m_generation;
+                EsCommonKeyType m_type;
+                u8 m_label_digest[crypto::Sha256Generator::HashSize];
             public:
                 void Set(int gen, EsCommonKeyType t, const u8 ld[crypto::Sha256Generator::HashSize]) {
-                    this->generation = gen;
-                    this->type       = t;
-                    std::memcpy(this->label_digest, ld, sizeof(this->label_digest));
+                    m_generation = gen;
+                    m_type       = t;
+                    std::memcpy(m_label_digest, ld, sizeof(m_label_digest));
                 }
 
-                int GetKeyGeneration() const { return this->generation; }
-                EsCommonKeyType GetCommonKeyType() const { return this->type; }
-                void GetLabelDigest(u8 dst[crypto::Sha256Generator::HashSize]) const { std::memcpy(dst, this->label_digest, sizeof(this->label_digest)); }
+                int GetKeyGeneration() const { return m_generation; }
+                EsCommonKeyType GetCommonKeyType() const { return m_type; }
+                void GetLabelDigest(u8 dst[crypto::Sha256Generator::HashSize]) const { std::memcpy(dst, m_label_digest, sizeof(m_label_digest)); }
         };
 
         class ModularExponentiateByStorageKeyAsyncArguments {
             private:
-                u8 msg[se::RsaSize];
+                u8 m_msg[se::RsaSize];
             public:
                 void Set(const void *m, size_t m_size) {
                     AMS_UNUSED(m_size);
-                    std::memcpy(this->msg, m, sizeof(this->msg));
+                    std::memcpy(m_msg, m, sizeof(m_msg));
                 }
 
-                const u8 *GetMessage() const { return this->msg; }
+                const u8 *GetMessage() const { return m_msg; }
         };
 
         constinit SmcResult    g_exp_mod_result    = SmcResult::Success;

exosphere/program/source/smc/secmon_smc_se_lock.cpp

@@ -21,35 +21,47 @@ namespace ams::secmon::smc {
 
     namespace {
 
-        constinit std::atomic_bool g_is_locked = false;
+        constinit util::Atomic<bool> g_is_locked = false;
+
+        ALWAYS_INLINE bool TryLockSecurityEngineImpl() {
+            bool value = false;
+            return g_is_locked.CompareExchangeStrong(value, true);
+        }
+
+        ALWAYS_INLINE void UnlockSecurityEngineImpl() {
+            g_is_locked = false;
+        }
+
+        ALWAYS_INLINE bool IsSecurityEngineLockedImpl() {
+            return g_is_locked.Load();
+        }
 
     }
 
     bool TryLockSecurityEngine() {
-        bool value = false;
-        return g_is_locked.compare_exchange_strong(value, true);
+        return TryLockSecurityEngineImpl();
     }
 
     void UnlockSecurityEngine() {
-        g_is_locked = false;
+        return UnlockSecurityEngineImpl();
     }
 
     bool IsSecurityEngineLocked() {
-        return g_is_locked;
+        return IsSecurityEngineLockedImpl();
     }
 
     SmcResult LockSecurityEngineAndInvoke(SmcArguments &args, SmcHandler impl) {
         /* Try to lock the security engine. */
-        SMC_R_UNLESS(TryLockSecurityEngine(), Busy);
-        ON_SCOPE_EXIT { UnlockSecurityEngine(); };
+        SMC_R_UNLESS(TryLockSecurityEngineImpl(), Busy);
+        ON_SCOPE_EXIT { UnlockSecurityEngineImpl(); };
 
         return impl(args);
     }
 
     SmcResult LockSecurityEngineAndInvokeAsync(SmcArguments &args, SmcHandler impl, GetResultHandler result_handler) {
         /* Try to lock the security engine. */
-        SMC_R_UNLESS(TryLockSecurityEngine(), Busy);
-        auto se_guard = SCOPE_GUARD { UnlockSecurityEngine(); };
+        SMC_R_UNLESS(TryLockSecurityEngineImpl(), Busy);
+        auto se_guard = SCOPE_GUARD { UnlockSecurityEngineImpl(); };
 
         /* Try to start an async operation. */
         const u64 async_key = BeginAsyncOperation(result_handler);

fusee/loader_stub/loader_stub.ld

@@ -73,6 +73,7 @@ SECTIONS
         fusee_loader_main.o(.text*)
         fusee_loader_uncompress.o(.text*)
         fusee_loader_error.o(.text*)
+        *(.text.memcpy)
         fusee_loader_main.o(.rodata*)
         fusee_loader_uncompress.o(.rodata*)
         fusee_loader_error.o(.rodata*)

fusee/loader_stub/source/fusee_loader_uncompress.cpp

@@ -22,14 +22,14 @@ namespace ams::nxboot::loader {
 
         class Lz4Uncompressor {
             private:
-                const u8 *src;
-                size_t src_size;
-                size_t src_offset;
-                u8 *dst;
-                size_t dst_size;
-                size_t dst_offset;
+                const u8 *m_src;
+                size_t m_src_size;
+                size_t m_src_offset;
+                u8 *m_dst;
+                size_t m_dst_size;
+                size_t m_dst_offset;
             public:
-                Lz4Uncompressor(void *dst, size_t dst_size, const void *src, size_t src_size) : src(static_cast<const u8 *>(src)), src_size(src_size), src_offset(0), dst(static_cast<u8 *>(dst)), dst_size(dst_size), dst_offset(0) {
+                Lz4Uncompressor(void *dst, size_t dst_size, const void *src, size_t src_size) : m_src(static_cast<const u8 *>(src)), m_src_size(src_size), m_src_offset(0), m_dst(static_cast<u8 *>(dst)), m_dst_size(dst_size), m_dst_offset(0) {
                     /* ... */
                 }
 
@@ -42,7 +42,7 @@ namespace ams::nxboot::loader {
                         this->Copy(this->GetCopySize(control >> 4));
 
                         /* If we've exceeded size, we're done. */
-                        if (this->src_offset >= this->src_size) {
+                        if (m_src_offset >= m_src_size) {
                             break;
                         }
 
@@ -55,21 +55,21 @@ namespace ams::nxboot::loader {
                         const size_t wide_copy_size = this->GetCopySize(control & 0xF);
 
                         /* Copy bytes. */
-                        const size_t end_offset = this->dst_offset + wide_copy_size + 4;
-                        for (size_t cur_offset = this->dst_offset; cur_offset < end_offset; this->dst_offset = (++cur_offset)) {
+                        const size_t end_offset = m_dst_offset + wide_copy_size + 4;
+                        for (size_t cur_offset = m_dst_offset; cur_offset < end_offset; m_dst_offset = (++cur_offset)) {
                             AMS_ABORT_UNLESS(wide_offset <= cur_offset);
 
-                            this->dst[cur_offset] = this->dst[cur_offset - wide_offset];
+                            m_dst[cur_offset] = m_dst[cur_offset - wide_offset];
                         }
                     }
                 }
             private:
                 u8 ReadByte() {
-                    return this->src[this->src_offset++];
+                    return m_src[m_src_offset++];
                 }
 
                 bool CanRead() const {
-                    return this->src_offset < this->src_size;
+                    return m_src_offset < m_src_size;
                 }
 
                 size_t GetCopySize(u8 control) {
@@ -87,11 +87,9 @@ namespace ams::nxboot::loader {
                 }
 
                 void Copy(size_t size) {
-                    for (size_t i = 0; i < size; ++i) {
-                        this->dst[this->dst_offset + i] = this->src[this->src_offset + i];
-                    }
-                    this->dst_offset += size;
-                    this->src_offset += size;
+                    __builtin_memcpy(m_dst + m_dst_offset, m_src + m_src_offset, size);
+                    m_dst_offset += size;
+                    m_src_offset += size;
                 }
         };
 

fusee/program/source/fatfs/fusee_diskio.cpp

@@ -27,7 +27,7 @@ bool diskio_write_sd_card(size_t sector_index, size_t sector_count, const void *
 }
 
 bool diskio_read_system(void *dst, size_t size, size_t sector_index, size_t sector_count) {
-    return R_SUCCEEDED(::ams::nxboot::ReadSystem(sector_index * 0x200, dst, size));
+    return false;
 }
 
 bool diskio_write_system(size_t sector_index, size_t sector_count, const void *src, size_t size) {

fusee/program/source/fs/fusee_fs_api.cpp

@@ -27,10 +27,8 @@ namespace ams::fs {
         constexpr size_t MaxDirectories = 2;
 
         constinit bool g_is_sd_mounted = false;
-        constinit bool g_is_sys_mounted = false;
 
         alignas(0x10) constinit FATFS g_sd_fs = {};
-        alignas(0x10) constinit FATFS g_sys_fs = {};
 
         alignas(0x10) constinit FIL g_files[MaxFiles] = {};
         alignas(0x10) constinit DIR g_dirs[MaxDirectories] = {};
@@ -131,18 +129,6 @@ namespace ams::fs {
         g_is_sd_mounted = false;
     }
 
-    bool MountSystem() {
-        AMS_ASSERT(!g_is_sys_mounted);
-        g_is_sys_mounted = f_mount(std::addressof(g_sys_fs), "sys:", 1) == FR_OK;
-        return g_is_sys_mounted;
-    }
-
-    void UnmountSystem() {
-        AMS_ASSERT(g_is_sys_mounted);
-        f_unmount("sys:");
-        g_is_sys_mounted = false;
-    }
-
     Result GetEntryType(DirectoryEntryType *out_entry_type, bool *out_archive, const char *path) {
         /* Get the file info. */
         FILINFO info;

fusee/program/source/fs/fusee_fs_api.hpp

@@ -28,7 +28,7 @@ namespace ams::fs {
     };
 
     struct ReadOption {
-        u32 value;
+        u32 _value;
 
         static const ReadOption None;
     };
@@ -36,7 +36,7 @@ namespace ams::fs {
     inline constexpr const ReadOption ReadOption::None = {0};
 
     inline constexpr bool operator==(const ReadOption &lhs, const ReadOption &rhs) {
-        return lhs.value == rhs.value;
+        return lhs._value == rhs._value;
     }
 
     inline constexpr bool operator!=(const ReadOption &lhs, const ReadOption &rhs) {
@@ -46,10 +46,10 @@ namespace ams::fs {
     static_assert(util::is_pod<ReadOption>::value && sizeof(ReadOption) == sizeof(u32));
 
     struct WriteOption {
-        u32 value;
+        u32 _value;
 
         constexpr inline bool HasFlushFlag() const {
-            return this->value & 1;
+            return _value & 1;
         }
 
         static const WriteOption None;
@@ -60,7 +60,7 @@ namespace ams::fs {
     inline constexpr const WriteOption WriteOption::Flush = {1};
 
     inline constexpr bool operator==(const WriteOption &lhs, const WriteOption &rhs) {
-        return lhs.value == rhs.value;
+        return lhs._value == rhs._value;
     }
 
     inline constexpr bool operator!=(const WriteOption &lhs, const WriteOption &rhs) {
@@ -98,9 +98,6 @@ namespace ams::fs {
     bool MountSdCard();
     void UnmountSdCard();
 
-    bool MountSystem();
-    void UnmountSystem();
-
     Result GetEntryType(DirectoryEntryType *out_entry_type, bool *out_archive, const char *path);
 
     Result CreateFile(const char *path, s64 size);

fusee/program/source/fusee_emummc.cpp

@@ -181,112 +181,6 @@ namespace ams::nxboot {
         constinit fs::IStorage *g_boot0_storage = nullptr;
         constinit fs::IStorage *g_user_storage  = nullptr;
 
-        class SystemPartitionStorage : public fs::IStorage {
-            private:
-                static constexpr size_t CacheEntries = BITSIZEOF(u32);
-                static constexpr size_t SectorSize   = 0x4000;
-            private:
-                fs::SubStorage m_storage;
-                u8 *m_sector_cache;
-                u32 *m_sector_ids;
-                u32 m_sector_flags;
-                u32 m_next_idx;
-            private:
-                Result LoadSector(u8 *sector, u32 sector_id) {
-                    /* Read the sector data. */
-                    R_TRY(m_storage.Read(static_cast<s64>(sector_id) * SectorSize, sector, SectorSize));
-
-                    /* Decrypt the sector. */
-                    se::DecryptAes128Xts(sector, SectorSize, pkg1::AesKeySlot_BootloaderSystem0, pkg1::AesKeySlot_BootloaderSystem1, sector, SectorSize, sector_id);
-
-                    /* Mark the sector as freshly loaded. */
-                    m_sector_flags &= ~(1u << (sector_id % CacheEntries));
-
-                    return ResultSuccess();
-                }
-
-                Result GetSector(u8 **out_sector, u32 sector_id) {
-                    /* Try to find in the cache. */
-                    for (size_t i = 0; i < CacheEntries; ++i) {
-                        if (m_sector_ids[i] == sector_id) {
-                            m_sector_flags &= ~(1u << i);
-                            *out_sector = m_sector_cache + SectorSize * i;
-                            return ResultSuccess();
-                        }
-                    }
-
-                    /* Find a sector to evict. */
-                    while ((m_sector_flags & (1u << m_next_idx)) == 0) {
-                        m_sector_flags |= (1u << m_next_idx);
-                        m_next_idx = (m_next_idx + 1) % CacheEntries;
-                    }
-
-                    /* Get the chosen sector. */
-                    *out_sector = m_sector_cache + SectorSize * m_next_idx;
-                    m_next_idx = (m_next_idx + 1) % CacheEntries;
-
-                    /* Load the sector. */
-                    return this->LoadSector(*out_sector, sector_id);
-                }
-            public:
-                SystemPartitionStorage(s64 ofs, s64 size) : m_storage(*g_user_storage, ofs, size) {
-                    /* Allocate sector cache. */
-                    m_sector_cache = static_cast<u8 *>(AllocateAligned(CacheEntries * SectorSize, SectorSize));
-
-                    /* Allocate sector ids. */
-                    m_sector_ids   = static_cast<u32 *>(AllocateAligned(CacheEntries * sizeof(u32), alignof(u32)));
-                    for (size_t i = 0; i < CacheEntries; ++i) {
-                        m_sector_ids[i] = std::numeric_limits<u32>::max();
-                    }
-
-                    /* All sectors are dirty. */
-                    m_sector_flags = ~0u;
-
-                    /* Next sector is 0. */
-                    m_next_idx = 0;
-                }
-
-                virtual Result Read(s64 offset, void *buffer, size_t size) override {
-                    u32 sector_id = offset / SectorSize;
-                    s64 subofs    = offset % SectorSize;
-
-                    u8 *cur_dst = static_cast<u8 *>(buffer);
-                    while (size > 0) {
-                        /* Get the current sector. */
-                        u8 *sector;
-                        R_TRY(this->GetSector(std::addressof(sector), sector_id++));
-
-                        /* Copy the data. */
-                        const size_t cur_size = std::min<size_t>(SectorSize - subofs, size);
-                        std::memcpy(cur_dst, sector + subofs, cur_size);
-
-                        /* Advance. */
-                        cur_dst += cur_size;
-                        size -= cur_size;
-                        subofs = 0;
-                    }
-
-                    return ResultSuccess();
-                }
-
-                virtual Result Flush() override {
-                    return m_storage.Flush();
-                }
-
-                virtual Result GetSize(s64 *out) override {
-                    return m_storage.GetSize(out);
-                }
-
-                virtual Result Write(s64 offset, const void *buffer, size_t size) override {
-                    return fs::ResultUnsupportedOperation();
-                }
-
-                virtual Result SetSize(s64 size) override {
-                    return fs::ResultUnsupportedOperation();
-                }
-        };
-
-        constinit SystemPartitionStorage *g_system_storage = nullptr;
         constinit fs::SubStorage *g_package2_storage = nullptr;
 
         struct Guid {
@@ -333,10 +227,6 @@ namespace ams::nxboot {
         };
         static_assert(sizeof(Gpt) == 16_KB + 0x200);
 
-        constexpr const u16 SystemPartitionName[] = {
-            'S', 'Y', 'S', 'T', 'E', 'M', 0
-        };
-
         constexpr const u16 Package2PartitionName[] = {
             'B', 'C', 'P', 'K', 'G', '2', '-', '1', '-', 'N', 'o', 'r', 'm', 'a', 'l', '-', 'M', 'a', 'i', 'n', 0
         };
@@ -447,27 +337,15 @@ namespace ams::nxboot {
             const s64 offset =  INT64_C(0x200) * gpt->entries[i].starting_lba;
             const u64 size   = UINT64_C(0x200) * (gpt->entries[i].ending_lba + 1 - gpt->entries[i].starting_lba);
 
-            if (std::memcmp(gpt->entries[i].partition_name, SystemPartitionName, sizeof(SystemPartitionName)) == 0) {
-                g_system_storage = AllocateObject<SystemPartitionStorage>(offset, size);
-            } else if (std::memcmp(gpt->entries[i].partition_name, Package2PartitionName, sizeof(Package2PartitionName)) == 0) {
+            if (std::memcmp(gpt->entries[i].partition_name, Package2PartitionName, sizeof(Package2PartitionName)) == 0) {
                 g_package2_storage = AllocateObject<fs::SubStorage>(*g_user_storage, offset, size);
             }
         }
 
-        /* Check that we created system storage. */
-        if (g_system_storage == nullptr) {
-            ShowFatalError("Failed to initialize SYSTEM\n");
-        }
-
         /* Check that we created package2 storage. */
         if (g_package2_storage == nullptr) {
             ShowFatalError("Failed to initialize Package2\n");
         }
-
-        /* Mount system. */
-        if (!fs::MountSystem()) {
-            ShowFatalError("Failed to mount SYSTEM\n");
-        }
     }
 
     Result ReadBoot0(s64 offset, void *dst, size_t size) {
@@ -478,8 +356,4 @@ namespace ams::nxboot {
         return g_package2_storage->Read(offset, dst, size);
     }
 
-    Result ReadSystem(s64 offset, void *dst, size_t size) {
-        return g_system_storage->Read(offset, dst, size);
-    }
-
-}
+}

fusee/program/source/fusee_overlay_manager.cpp

@@ -33,11 +33,11 @@ namespace ams::nxboot {
             u32 verif_hash;
             u32 store_hash;
             if (fuse::GetSocType() == fuse::SocType_Erista) {
-                result = fs::ReadFile(archive_file, __builtin_offsetof(ExternalPackage, ovl_mtc_erista), GetOverlayDestination(), sizeof(ExternalPackage{}.ovl_mtc_erista));
+                result = fs::ReadFile(archive_file, AMS_OFFSETOF(ExternalPackage, ovl_mtc_erista), GetOverlayDestination(), sizeof(ExternalPackage{}.ovl_mtc_erista));
                 verif_hash = reinterpret_cast<const u32 *>(GetOverlayDestination())[-2];
                 store_hash = reinterpret_cast<const u32 *>(GetOverlayDestination())[(sizeof(ExternalPackage{}.ovl_mtc_erista) / sizeof(u32)) - 1];
             } else /* if (fuse::GetSocType() == fuse::SocType_Mariko) */ {
-                result = fs::ReadFile(archive_file, __builtin_offsetof(ExternalPackage, ovl_mtc_mariko), GetOverlayDestination(), sizeof(ExternalPackage{}.ovl_mtc_mariko));
+                result = fs::ReadFile(archive_file, AMS_OFFSETOF(ExternalPackage, ovl_mtc_mariko), GetOverlayDestination(), sizeof(ExternalPackage{}.ovl_mtc_mariko));
                 verif_hash = reinterpret_cast<const u32 *>(GetOverlayDestination())[-1];
                 store_hash = reinterpret_cast<const u32 *>(GetOverlayDestination())[(sizeof(ExternalPackage{}.ovl_mtc_mariko) / sizeof(u32)) - 1];
             }

fusee/program/source/fusee_setup_horizon.cpp

@@ -118,26 +118,12 @@ namespace ams::nxboot {
             return R_SUCCEEDED(fs::GetEntryType(std::addressof(entry_type), std::addressof(archive), path)) && entry_type == fs::DirectoryEntryType_Directory;
         }
 
-        [[maybe_unused]] bool IsFileExist(const char *path) {
+        bool IsFileExist(const char *path) {
             fs::DirectoryEntryType entry_type;
             bool archive;
             return R_SUCCEEDED(fs::GetEntryType(std::addressof(entry_type), std::addressof(archive), path)) && entry_type == fs::DirectoryEntryType_File;
         }
 
-        bool IsConcatenationFileExist(const char *path) {
-            fs::DirectoryEntryType entry_type;
-            bool archive;
-            return R_SUCCEEDED(fs::GetEntryType(std::addressof(entry_type), std::addressof(archive), path)) && ((entry_type == fs::DirectoryEntryType_File) || (entry_type == fs::DirectoryEntryType_Directory && archive));
-        }
-
-        constinit char g_nca_path[0x40] = "sys:/contents/registered/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.nca";
-
-        bool IsNcaExist(const char *nca_name) {
-            std::memcpy(g_nca_path + 0x19, nca_name, 0x20);
-
-            return IsConcatenationFileExist(g_nca_path);
-        }
-
         bool ConfigureEmummc() {
             /* Set magic. */
             g_emummc_cfg.base_cfg.magic = secmon::EmummcBaseConfiguration::Magic;
@@ -160,7 +146,7 @@ namespace ams::nxboot {
                         /* Handle individual fields. */
                         for (const auto &entry : section.kv_list) {
                             if (std::strcmp(entry.key, "enabled") == 0) {
-                                enabled = entry.value[0] == '1';
+                                enabled = entry.value[0] != '0';
                             } else if (std::strcmp(entry.key, "id") == 0) {
                                 id = ParseHexInteger(entry.value);
                             } else if (std::strcmp(entry.key, "sector") == 0) {
@@ -220,143 +206,56 @@ namespace ams::nxboot {
             return package1;
         }
 
-        ams::TargetFirmware GetTargetFirmware(const u8 *package1) {
-            /* Get first an approximation of the target firmware. */
-            ams::TargetFirmware target_firmware = ams::TargetFirmware_Current;
+        ams::TargetFirmware GetApproximateTargetFirmware(const u8 *package1) {
+            /* Get an approximation of the target firmware. */
             switch (package1[0x1F]) {
                 case 0x01:
-                    target_firmware = ams::TargetFirmware_1_0_0;
-                    break;
+                    return ams::TargetFirmware_1_0_0;
                 case 0x02:
-                    target_firmware = ams::TargetFirmware_2_0_0;
-                    break;
+                    return ams::TargetFirmware_2_0_0;
                 case 0x04:
-                    target_firmware = ams::TargetFirmware_3_0_0;
-                    break;
+                    return ams::TargetFirmware_3_0_0;
                 case 0x07:
-                    target_firmware = ams::TargetFirmware_4_0_0;
-                    break;
+                    return ams::TargetFirmware_4_0_0;
                 case 0x0B:
-                    target_firmware = ams::TargetFirmware_5_0_0;
-                    break;
+                    return ams::TargetFirmware_5_0_0;
                 case 0x0E:
                     if (std::memcmp(package1 + 0x10, "20180802", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_6_0_0;
+                        return ams::TargetFirmware_6_0_0;
                     } else if (std::memcmp(package1 + 0x10, "20181107", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_6_2_0;
-                    } else {
-                        ShowFatalError("Unable to identify package1!\n");
+                        return ams::TargetFirmware_6_2_0;
                     }
                     break;
                 case 0x0F:
-                    target_firmware = ams::TargetFirmware_7_0_0;
-                    break;
+                    return ams::TargetFirmware_7_0_0;
                 case 0x10:
                     if (std::memcmp(package1 + 0x10, "20190314", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_8_0_0;
+                        return ams::TargetFirmware_8_0_0;
                     } else if (std::memcmp(package1 + 0x10, "20190531", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_8_1_0;
+                        return ams::TargetFirmware_8_1_0;
                     } else if (std::memcmp(package1 + 0x10, "20190809", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_9_0_0;
+                        return ams::TargetFirmware_9_0_0;
                     } else if (std::memcmp(package1 + 0x10, "20191021", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_9_1_0;
+                        return ams::TargetFirmware_9_1_0;
                     } else if (std::memcmp(package1 + 0x10, "20200303", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_10_0_0;
+                        return ams::TargetFirmware_10_0_0;
                     } else if (std::memcmp(package1 + 0x10, "20201030", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_11_0_0;
+                        return ams::TargetFirmware_11_0_0;
                     } else if (std::memcmp(package1 + 0x10, "20210129", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_12_0_0;
+                        return ams::TargetFirmware_12_0_0;
                     } else if (std::memcmp(package1 + 0x10, "20210422", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_12_0_2;
+                        return ams::TargetFirmware_12_0_2;
                     } else if (std::memcmp(package1 + 0x10, "20210607", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_12_1_0;
+                        return ams::TargetFirmware_12_1_0;
                     } else if (std::memcmp(package1 + 0x10, "20210805", 8) == 0) {
-                        target_firmware = ams::TargetFirmware_13_0_0;
-                    } else {
-                        ShowFatalError("Unable to identify package1!\n");
+                        return ams::TargetFirmware_13_0_0;
                     }
                     break;
                 default:
-                    ShowFatalError("Unable to identify package1!\n");
                     break;
             }
 
-            #define CHECK_NCA(NCA_ID, VERSION) do { if (IsNcaExist(NCA_ID)) { return ams::TargetFirmware_##VERSION; } } while(0)
-
-            if (target_firmware >= ams::TargetFirmware_13_0_0) {
-                CHECK_NCA("bf2337ee88bd9f963a33b3ecbbc3732a", 13_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_12_1_0) {
-                CHECK_NCA("9d9d83d68d9517f245f3e8cd7f93c416", 12_1_0);
-            } else if (target_firmware >= ams::TargetFirmware_12_0_2) {
-                CHECK_NCA("a1863a5c0e1cedd442f5e60b0422dc15", 12_0_3);
-                CHECK_NCA("63d928b5a3016fe8cc0e76d2f06f4e98", 12_0_2);
-            } else if (target_firmware >= ams::TargetFirmware_12_0_0) {
-                CHECK_NCA("e65114b456f9d0b566a80e53bade2d89", 12_0_1);
-                CHECK_NCA("bd4185843550fbba125b20787005d1d2", 12_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_11_0_0) {
-                CHECK_NCA("56211c7a5ed20a5332f5cdda67121e37", 11_0_1);
-                CHECK_NCA("594c90bcdbcccad6b062eadba0cd0e7e", 11_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_10_0_0) {
-                CHECK_NCA("26325de4db3909e0ef2379787c7e671d", 10_2_0);
-                CHECK_NCA("5077973537f6735b564dd7475b779f87", 10_1_1); /* Exclusive to China. */
-                CHECK_NCA("fd1faed0ca750700d254c0915b93d506", 10_1_0);
-                CHECK_NCA("34728c771299443420820d8ae490ea41", 10_0_4);
-                CHECK_NCA("5b1df84f88c3334335bbb45d8522cbb4", 10_0_3);
-                CHECK_NCA("e951bc9dedcd54f65ffd83d4d050f9e0", 10_0_2);
-                CHECK_NCA("36ab1acf0c10a2beb9f7d472685f9a89", 10_0_1);
-                CHECK_NCA("5625cdc21d5f1ca52f6c36ba261505b9", 10_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_9_1_0) {
-                CHECK_NCA("09ef4d92bb47b33861e695ba524a2c17", 9_2_0);
-                CHECK_NCA("c5fbb49f2e3648c8cfca758020c53ecb", 9_1_0);
-            } else if (target_firmware >= ams::TargetFirmware_9_0_0) {
-                CHECK_NCA("fd1ffb82dc1da76346343de22edbc97c", 9_0_1);
-                CHECK_NCA("a6af05b33f8f903aab90c8b0fcbcc6a4", 9_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_8_1_0) {
-                CHECK_NCA("724d9b432929ea43e787ad81bf09ae65", 8_1_1); /* 8.1.1-100 from Lite */
-                CHECK_NCA("e9bb0602e939270a9348bddd9b78827b", 8_1_1); /* 8.1.1-12  from chinese gamecard */
-                CHECK_NCA("7eedb7006ad855ec567114be601b2a9d", 8_1_0);
-            } else if (target_firmware >= ams::TargetFirmware_8_0_0) {
-                CHECK_NCA("6c5426d27c40288302ad616307867eba", 8_0_1);
-                CHECK_NCA("4fe7b4abcea4a0bcc50975c1a926efcb", 8_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_7_0_0) {
-                CHECK_NCA("e6b22c40bb4fa66a151f1dc8db5a7b5c", 7_0_1);
-                CHECK_NCA("c613bd9660478de69bc8d0e2e7ea9949", 7_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_6_2_0) {
-                CHECK_NCA("6dfaaf1a3cebda6307aa770d9303d9b6", 6_2_0);
-            } else if (target_firmware >= ams::TargetFirmware_6_0_0) {
-                CHECK_NCA("1d21680af5a034d626693674faf81b02", 6_1_0);
-                CHECK_NCA("663e74e45ffc86fbbaeb98045feea315", 6_0_1);
-                CHECK_NCA("258c1786b0f6844250f34d9c6f66095b", 6_0_0); /* Release     6.0.0-5.0 */
-                CHECK_NCA("286e30bafd7e4197df6551ad802dd815", 6_0_0); /* Pre-Release 6.0.0-4.0 */
-            } else if (target_firmware >= ams::TargetFirmware_5_0_0) {
-                CHECK_NCA("fce3b0ea366f9c95fe6498b69274b0e7", 5_1_0);
-                CHECK_NCA("c5758b0cb8c6512e8967e38842d35016", 5_0_2);
-                CHECK_NCA("53eb605d4620e8fd50064b24fd57783a", 5_0_1);
-                CHECK_NCA("09a2f9c16ce1c121ae6d231b35d17515", 5_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_4_0_0) {
-                CHECK_NCA("77e1ae7661ad8a718b9b13b70304aeea", 4_1_0);
-                CHECK_NCA("d0e5d20e3260f3083bcc067483b71274", 4_0_1);
-                CHECK_NCA("483a24ee3fd7149f9112d1931166a678", 4_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_3_0_0) {
-                CHECK_NCA("704129fc89e1fcb85c37b3112e51b0fc", 3_0_2);
-                CHECK_NCA("1fb00543307337d523ccefa9923e0c50", 3_0_1);
-                CHECK_NCA("6ebd3447473bade18badbeb5032af87d", 3_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_2_0_0) {
-                CHECK_NCA("d1c991c53a8a9038f8c3157a553d876d", 2_3_0);
-                CHECK_NCA("7f90353dff2d7ce69e19e07ebc0d5489", 2_2_0);
-                CHECK_NCA("e9b3e75fce00e52fe646156634d229b4", 2_1_0);
-                CHECK_NCA("7a1f79f8184d4b9bae1755090278f52c", 2_0_0);
-            } else if (target_firmware >= ams::TargetFirmware_1_0_0) {
-                CHECK_NCA("a1b287e07f8455e8192f13d0e45a2aaf", 1_0_0); /* 1.0.0 from Factory */
-                CHECK_NCA("117f7b9c7da3e8cef02340596af206b3", 1_0_0); /* 1.0.0 from Gamecard */
-            } else {
-                ShowFatalError("Unable to determine target firmware!\n");
-            }
-
-            #undef CHECK_NCA
-
-            /* If we didn't find a more specific firmware, return our package1 approximation. */
-            return target_firmware;
+            ShowFatalError("Unable to identify package1!\n");
         }
 
         u8 *LoadBootConfigAndPackage2() {
@@ -371,7 +270,7 @@ namespace ams::nxboot {
             u8 *package2;
             size_t package2_size;
             {
-                constexpr s64 Package2Offset = __builtin_offsetof(pkg2::StorageLayout, package2_header);
+                constexpr s64 Package2Offset = AMS_OFFSETOF(pkg2::StorageLayout, package2_header);
 
                 pkg2::Package2Header header;
                 if (R_FAILED((result = ReadPackage2(Package2Offset, std::addressof(header), sizeof(header))))) {
@@ -825,7 +724,7 @@ namespace ams::nxboot {
         const u8 * const package1 = LoadPackage1(soc_type);
 
         /* Get target firmware. */
-        const auto target_firmware = GetTargetFirmware(package1);
+        const auto target_firmware = GetApproximateTargetFirmware(package1);
 
         /* Read/decrypt package2. */
         u8 * const package2 = LoadBootConfigAndPackage2();

fusee/program/source/fusee_uncompress.cpp

@@ -22,14 +22,14 @@ namespace ams::nxboot {
 
         class Lz4Uncompressor {
             private:
-                const u8 *src;
-                size_t src_size;
-                size_t src_offset;
-                u8 *dst;
-                size_t dst_size;
-                size_t dst_offset;
+                const u8 *m_src;
+                size_t m_src_size;
+                size_t m_src_offset;
+                u8 *m_dst;
+                size_t m_dst_size;
+                size_t m_dst_offset;
             public:
-                Lz4Uncompressor(void *dst, size_t dst_size, const void *src, size_t src_size) : src(static_cast<const u8 *>(src)), src_size(src_size), src_offset(0), dst(static_cast<u8 *>(dst)), dst_size(dst_size), dst_offset(0) {
+                Lz4Uncompressor(void *dst, size_t dst_size, const void *src, size_t src_size) : m_src(static_cast<const u8 *>(src)), m_src_size(src_size), m_src_offset(0), m_dst(static_cast<u8 *>(dst)), m_dst_size(dst_size), m_dst_offset(0) {
                     /* ... */
                 }
 
@@ -42,7 +42,7 @@ namespace ams::nxboot {
                         this->Copy(this->GetCopySize(control >> 4));
 
                         /* If we've exceeded size, we're done. */
-                        if (this->src_offset >= this->src_size) {
+                        if (m_src_offset >= m_src_size) {
                             break;
                         }
 
@@ -55,21 +55,21 @@ namespace ams::nxboot {
                         const size_t wide_copy_size = this->GetCopySize(control & 0xF);
 
                         /* Copy bytes. */
-                        const size_t end_offset = this->dst_offset + wide_copy_size + 4;
-                        for (size_t cur_offset = this->dst_offset; cur_offset < end_offset; this->dst_offset = (++cur_offset)) {
+                        const size_t end_offset = m_dst_offset + wide_copy_size + 4;
+                        for (size_t cur_offset = m_dst_offset; cur_offset < end_offset; m_dst_offset = (++cur_offset)) {
                             AMS_ABORT_UNLESS(wide_offset <= cur_offset);
 
-                            this->dst[cur_offset] = this->dst[cur_offset - wide_offset];
+                            m_dst[cur_offset] = m_dst[cur_offset - wide_offset];
                         }
                     }
                 }
             private:
                 u8 ReadByte() {
-                    return this->src[this->src_offset++];
+                    return m_src[m_src_offset++];
                 }
 
                 bool CanRead() const {
-                    return this->src_offset < this->src_size;
+                    return m_src_offset < m_src_size;
                 }
 
                 size_t GetCopySize(u8 control) {
@@ -87,9 +87,9 @@ namespace ams::nxboot {
                 }
 
                 void Copy(size_t size) {
-                    __builtin_memcpy(this->dst + this->dst_offset, this->src + this->src_offset, size);
-                    this->dst_offset += size;
-                    this->src_offset += size;
+                    __builtin_memcpy(m_dst + m_dst_offset, m_src + m_src_offset, size);
+                    m_dst_offset += size;
+                    m_src_offset += size;
                 }
         };
 

libraries/.gitrepo

@@ -6,7 +6,7 @@
 [subrepo]
 	remote = https://github.com/Atmosphere-NX/Atmosphere-libs
 	branch = master
-	commit = 0a4c2759a1f73cb5581ed2e87c20e05440a2c037
-	parent = d14290e3572c52f4dd9ed2a16f698ba12e9eea67
+	commit = cf765c0946cc5c828364ae6bfccddc4041304f28
+	parent = 8634ea0f7c4f0e68adf2dfaaddc6ae1e225c4fc2
 	method = merge
 	cmdver = 0.4.1

libraries/config/board/qemu/virt/board.mk

@@ -0,0 +1,5 @@
+export ATMOSPHERE_DEFINES  += -DATMOSPHERE_BOARD_QEMU_VIRT -D__SWITCH__
+export ATMOSPHERE_SETTINGS +=
+export ATMOSPHERE_CFLAGS   +=
+export ATMOSPHERE_CXXFLAGS +=
+export ATMOSPHERE_ASFLAGS  +=

libraries/config/common.mk

@@ -23,7 +23,8 @@ export ATMOSPHERE_CFLAGS   := -Wall -ffunction-sections -fdata-sections -fno-str
                               -fno-asynchronous-unwind-tables -fno-unwind-tables -fno-stack-protector \
                               -Wno-format-truncation -Wno-format-zero-length -Wno-stringop-truncation
 
-export ATMOSPHERE_CXXFLAGS := -fno-rtti -fno-exceptions -std=gnu++20
+
+export ATMOSPHERE_CXXFLAGS := -fno-rtti -fno-exceptions -std=gnu++20 -Wno-invalid-offsetof
 export ATMOSPHERE_ASFLAGS  :=
 
 
@@ -51,6 +52,21 @@ export ATMOSPHERE_OS_NAME    := horizon
 export ATMOSPHERE_CPU_EXTENSIONS :=
 endif
 
+else ifeq ($(ATMOSPHERE_BOARD),qemu-virt)
+
+
+ifeq ($(ATMOSPHERE_CPU),arm-cortex-a57)
+export ATMOSPHERE_ARCH_DIR   := arm64
+export ATMOSPHERE_BOARD_DIR  := qemu/virt
+export ATMOSPHERE_OS_DIR     := horizon
+
+export ATMOSPHERE_ARCH_NAME  := arm64
+export ATMOSPHERE_BOARD_NAME := qemu_virt
+export ATMOSPHERE_OS_NAME    := horizon
+
+export ATMOSPHERE_CPU_EXTENSIONS := arm_crypto_extension aarch64_crypto_extension
+endif
+
 endif
 
 ifeq ($(ATMOSPHERE_CPU),arm-cortex-a57)

libraries/libexosphere/include/exosphere/fuse.hpp

@@ -81,9 +81,9 @@ namespace ams::fuse {
         DramId_Count,
     };
 
-    enum QuestState {
-        QuestState_Disabled = 0,
-        QuestState_Enabled  = 1,
+    enum RetailInteractiveDisplayState {
+        RetailInteractiveDisplayState_Disabled = 0,
+        RetailInteractiveDisplayState_Enabled  = 1,
     };
 
     void SetRegisterAddress(uintptr_t address);
@@ -102,19 +102,19 @@ namespace ams::fuse {
 
     bool GetSecureBootKey(void *dst);
 
-    void            GetEcid(br::BootEcid *out);
-    HardwareType    GetHardwareType();
-    HardwareState   GetHardwareState();
-    u64             GetDeviceId();
-    PatchVersion    GetPatchVersion();
-    QuestState      GetQuestState();
-    pmic::Regulator GetRegulator();
-    int             GetDeviceUniqueKeyGeneration();
+    void                          GetEcid(br::BootEcid *out);
+    HardwareType                  GetHardwareType();
+    HardwareState                 GetHardwareState();
+    u64                           GetDeviceId();
+    PatchVersion                  GetPatchVersion();
+    RetailInteractiveDisplayState GetRetailInteractiveDisplayState();
+    pmic::Regulator               GetRegulator();
+    int                           GetDeviceUniqueKeyGeneration();
 
-    SocType         GetSocType();
-    int             GetExpectedFuseVersion(TargetFirmware target_fw);
-    int             GetFuseVersion();
-    bool            HasRcmVulnerabilityPatch();
+    SocType                       GetSocType();
+    int                           GetExpectedFuseVersion(TargetFirmware target_fw);
+    int                           GetFuseVersion();
+    bool                          HasRcmVulnerabilityPatch();
 
     bool IsOdmProductionMode();
     void ConfigureFuseBypass();

libraries/libexosphere/include/exosphere/pkg1/pkg1_bootloader_parameters.hpp

@@ -55,9 +55,9 @@ namespace ams::pkg1 {
     static_assert(util::is_pod<SecureMonitorParameters>::value);
     static_assert(sizeof(SecureMonitorParameters) == 0x200);
 
-    static_assert(offsetof(SecureMonitorParameters, bct_params)       == 0x10);
-    static_assert(offsetof(SecureMonitorParameters, bootloader_state) == 0xF8);
-    static_assert(offsetof(SecureMonitorParameters, secmon_state) == 0xFC);
+    static_assert(AMS_OFFSETOF(SecureMonitorParameters, bct_params)       == 0x10);
+    static_assert(AMS_OFFSETOF(SecureMonitorParameters, bootloader_state) == 0xF8);
+    static_assert(AMS_OFFSETOF(SecureMonitorParameters, secmon_state) == 0xFC);
 
     enum BootloaderAttribute {
         BootloaderAttribute_None         = (0u << 0),

libraries/libexosphere/include/exosphere/secmon/secmon_configuration_context.arch.arm64.hpp

@@ -70,6 +70,10 @@ namespace ams::secmon {
             GetConfigurationContext().secmon_cfg.key_generation = generation;
         }
 
+        ALWAYS_INLINE void SetTargetFirmware(ams::TargetFirmware target_firmware) {
+            GetConfigurationContext().secmon_cfg.target_firmware = target_firmware;
+        }
+
         ALWAYS_INLINE pkg1::BootConfig *GetBootConfigStorage() {
             return std::addressof(GetConfigurationContext().boot_config);
         }

libraries/libexosphere/include/exosphere/secmon/secmon_memory_layout.hpp

@@ -22,52 +22,53 @@ namespace ams::secmon {
     using Address = u64;
 
     struct MemoryRegion {
-        Address start_address;
-        Address end_address;
-
-        constexpr MemoryRegion(Address address, size_t size) : start_address(address), end_address(address + size) {
-            if (end_address < start_address) {
-                __builtin_unreachable();
+        private:
+            Address m_start_address;
+            Address m_end_address;
+        public:
+            consteval MemoryRegion(Address address, size_t size) : m_start_address(address), m_end_address(address + size) {
+                if (m_end_address < m_start_address) {
+                    __builtin_unreachable();
+                }
             }
-        }
 
-        constexpr Address GetStartAddress() const {
-            return this->start_address;
-        }
+            constexpr Address GetStartAddress() const {
+                return m_start_address;
+            }
 
-        constexpr Address GetAddress() const {
-            return this->GetStartAddress();
-        }
+            constexpr Address GetAddress() const {
+                return this->GetStartAddress();
+            }
 
-        constexpr Address GetEndAddress() const {
-            return this->end_address;
-        }
+            constexpr Address GetEndAddress() const {
+                return m_end_address;
+            }
 
-        constexpr Address GetLastAddress() const {
-            return this->end_address - 1;
-        }
+            constexpr Address GetLastAddress() const {
+                return m_end_address - 1;
+            }
 
-        constexpr size_t GetSize() const {
-            return this->end_address - this->start_address;
-        }
+            constexpr size_t GetSize() const {
+                return m_end_address - m_start_address;
+            }
 
-        constexpr bool Contains(Address address, size_t size) const {
-            return this->start_address <= address && (address + size - 1) <= this->GetLastAddress();
-        }
+            constexpr bool Contains(Address address, size_t size) const {
+                return m_start_address <= address && (address + size - 1) <= this->GetLastAddress();
+            }
 
-        constexpr bool Contains(const MemoryRegion &rhs) const {
-            return this->Contains(rhs.GetStartAddress(), rhs.GetSize());
-        }
+            constexpr bool Contains(const MemoryRegion &rhs) const {
+                return this->Contains(rhs.GetStartAddress(), rhs.GetSize());
+            }
 
-        template<typename T = void> requires (std::is_same<T, void>::value || util::is_pod<T>::value)
-        ALWAYS_INLINE T *GetPointer() const {
-            return reinterpret_cast<T *>(this->GetAddress());
-        }
+            template<typename T = void> requires (std::is_same<T, void>::value || util::is_pod<T>::value)
+            ALWAYS_INLINE T *GetPointer() const {
+                return reinterpret_cast<T *>(this->GetAddress());
+            }
 
-        template<typename T = void> requires (std::is_same<T, void>::value || util::is_pod<T>::value)
-        ALWAYS_INLINE T *GetEndPointer() const {
-            return reinterpret_cast<T *>(this->GetEndAddress());
-        }
+            template<typename T = void> requires (std::is_same<T, void>::value || util::is_pod<T>::value)
+            ALWAYS_INLINE T *GetEndPointer() const {
+                return reinterpret_cast<T *>(this->GetEndAddress());
+            }
     };
 
     constexpr inline const MemoryRegion MemoryRegionVirtual  = MemoryRegion(UINT64_C(0x1F0000000), 2_MB);

libraries/libexosphere/include/exosphere/secmon/secmon_volatile_context.hpp

@@ -128,10 +128,10 @@ namespace ams::secmon {
 
     }
 
-    constexpr inline const Address WarmbootStackAddress   = MemoryRegionVirtualTzramVolatileData.GetAddress() + offsetof(VolatileData, warmboot_stack)    + sizeof(VolatileData::warmboot_stack);
-    constexpr inline const Address Core012SmcStackAddress = MemoryRegionVirtualTzramVolatileData.GetAddress() + offsetof(VolatileData, core012_smc_stack) + sizeof(VolatileData::core012_smc_stack);
+    constexpr inline const Address WarmbootStackAddress   = MemoryRegionVirtualTzramVolatileData.GetAddress() + AMS_OFFSETOF(VolatileData, warmboot_stack)    + sizeof(VolatileData::warmboot_stack);
+    constexpr inline const Address Core012SmcStackAddress = MemoryRegionVirtualTzramVolatileData.GetAddress() + AMS_OFFSETOF(VolatileData, core012_smc_stack) + sizeof(VolatileData::core012_smc_stack);
 
-    constexpr inline const Address Core0ExceptionStackAddress = MemoryRegionVirtualTzramVolatileData.GetAddress() + offsetof(VolatileData, core_exception_stacks) + CoreExceptionStackSize;
+    constexpr inline const Address Core0ExceptionStackAddress = MemoryRegionVirtualTzramVolatileData.GetAddress() + AMS_OFFSETOF(VolatileData, core_exception_stacks) + CoreExceptionStackSize;
     constexpr inline const Address Core1ExceptionStackAddress = Core0ExceptionStackAddress + CoreExceptionStackSize;
     constexpr inline const Address Core2ExceptionStackAddress = Core1ExceptionStackAddress + CoreExceptionStackSize;
 

libraries/libexosphere/source/crypto/crypto_aes_impl_security_engine.cpp

@@ -37,7 +37,7 @@ namespace ams::crypto::impl {
         AMS_UNUSED(key_size, is_encrypt);
 
         /* Set the security engine keyslot. */
-        this->slot = *static_cast<const int *>(key);
+        m_slot = *static_cast<const int *>(key);
     }
 
     template<size_t KeySize>
@@ -48,14 +48,14 @@ namespace ams::crypto::impl {
 
         if constexpr (KeySize == 16) {
             /* Aes 128. */
-            se::EncryptAes128(dst, dst_size, this->slot, src, src_size);
+            se::EncryptAes128(dst, dst_size, m_slot, src, src_size);
         } else if constexpr (KeySize == 24) {
             /* Aes 192. */
-            /* TODO: se::EncryptAes192(dst, dst_size, this->slot, src, src_size); */
+            /* TODO: se::EncryptAes192(dst, dst_size, m_slot, src, src_size); */
             AMS_UNUSED(dst, dst_size, src, src_size);
         } else if constexpr (KeySize == 32) {
             /* Aes 256. */
-            /* TODO: se::EncryptAes256(dst, dst_size, this->slot, src, src_size); */
+            /* TODO: se::EncryptAes256(dst, dst_size, m_slot, src, src_size); */
             AMS_UNUSED(dst, dst_size, src, src_size);
         } else {
             /* Invalid key size. */
@@ -71,14 +71,14 @@ namespace ams::crypto::impl {
 
         if constexpr (KeySize == 16) {
             /* Aes 128. */
-            se::DecryptAes128(dst, dst_size, this->slot, src, src_size);
+            se::DecryptAes128(dst, dst_size, m_slot, src, src_size);
         } else if constexpr (KeySize == 24) {
             /* Aes 192. */
-            /* TODO: se::DecryptAes192(dst, dst_size, this->slot, src, src_size); */
+            /* TODO: se::DecryptAes192(dst, dst_size, m_slot, src, src_size); */
             AMS_UNUSED(dst, dst_size, src, src_size);
         } else if constexpr (KeySize == 32) {
             /* Aes 256. */
-            /* TODO: se::DecryptAes256(dst, dst_size, this->slot, src, src_size); */
+            /* TODO: se::DecryptAes256(dst, dst_size, m_slot, src, src_size); */
             AMS_UNUSED(dst, dst_size, src, src_size);
         } else {
             /* Invalid key size. */

libraries/libexosphere/source/fuse/fuse_api.cpp

@@ -37,15 +37,15 @@ namespace ams::fuse {
         };
 
         struct OdmWord4 {
-            using HardwareState1 = util::BitPack32::Field<0,                    2, int>;
-            using HardwareType1  = util::BitPack32::Field<HardwareState1::Next, 1, int>;
-            using DramId         = util::BitPack32::Field<HardwareType1::Next,  5, int>;
-            using HardwareType2  = util::BitPack32::Field<DramId::Next,         1, int>;
-            using HardwareState2 = util::BitPack32::Field<HardwareType2::Next,  1, int>;
-            using QuestState     = util::BitPack32::Field<HardwareState2::Next, 1, int>;
-            using FormatVersion  = util::BitPack32::Field<QuestState::Next,     1, int>;
-            using Reserved       = util::BitPack32::Field<FormatVersion::Next,  4, int>;
-            using HardwareType3  = util::BitPack32::Field<Reserved::Next,       4, int>;
+            using HardwareState1                = util::BitPack32::Field<0,                                   2, int>;
+            using HardwareType1                 = util::BitPack32::Field<HardwareState1::Next,                1, int>;
+            using DramId                        = util::BitPack32::Field<HardwareType1::Next,                 5, int>;
+            using HardwareType2                 = util::BitPack32::Field<DramId::Next,                        1, int>;
+            using HardwareState2                = util::BitPack32::Field<HardwareType2::Next,                 1, int>;
+            using RetailInteractiveDisplayState = util::BitPack32::Field<HardwareState2::Next,                1, int>;
+            using FormatVersion                 = util::BitPack32::Field<RetailInteractiveDisplayState::Next, 1, int>;
+            using Reserved                      = util::BitPack32::Field<FormatVersion::Next,                 4, int>;
+            using HardwareType3                 = util::BitPack32::Field<Reserved::Next,                      4, int>;
         };
 
         struct OdmWord28 {
@@ -343,8 +343,8 @@ namespace ams::fuse {
         return static_cast<PatchVersion>(static_cast<int>(GetSocType() << 12) | patch_version);
     }
 
-    QuestState GetQuestState() {
-        return static_cast<QuestState>(util::BitPack32{GetCommonOdmWord(4)}.Get<OdmWord4::QuestState>());
+    RetailInteractiveDisplayState GetRetailInteractiveDisplayState() {
+        return static_cast<RetailInteractiveDisplayState>(util::BitPack32{GetCommonOdmWord(4)}.Get<OdmWord4::RetailInteractiveDisplayState>());
     }
 
     pmic::Regulator GetRegulator() {

libraries/libexosphere/source/gic/gic_api.cpp

@@ -191,27 +191,27 @@ namespace ams::gic {
     }
 
     void SetPriority(int interrupt_id, int priority) {
-        ReadWrite(g_distributor_address + offsetof(GicDistributor, ipriorityr), BITSIZEOF(u8), interrupt_id, priority);
+        ReadWrite(g_distributor_address + AMS_OFFSETOF(GicDistributor, ipriorityr), BITSIZEOF(u8), interrupt_id, priority);
     }
 
     void SetInterruptGroup(int interrupt_id, int group) {
-        ReadWrite(g_distributor_address + offsetof(GicDistributor, igroupr), 1, interrupt_id, group);
+        ReadWrite(g_distributor_address + AMS_OFFSETOF(GicDistributor, igroupr), 1, interrupt_id, group);
     }
 
     void SetEnable(int interrupt_id, bool enable) {
-        Write(g_distributor_address + offsetof(GicDistributor, isenabler), 1, interrupt_id, enable);
+        Write(g_distributor_address + AMS_OFFSETOF(GicDistributor, isenabler), 1, interrupt_id, enable);
     }
 
     void SetSpiTargetCpu(int interrupt_id, u32 cpu_mask) {
-        ReadWrite(g_distributor_address + offsetof(GicDistributor, itargetsr), BITSIZEOF(u8), interrupt_id, cpu_mask);
+        ReadWrite(g_distributor_address + AMS_OFFSETOF(GicDistributor, itargetsr), BITSIZEOF(u8), interrupt_id, cpu_mask);
     }
 
     void SetSpiMode(int interrupt_id, InterruptMode mode) {
-        ReadWrite(g_distributor_address + offsetof(GicDistributor, icfgr), 2, interrupt_id, static_cast<u32>(mode) << 1);
+        ReadWrite(g_distributor_address + AMS_OFFSETOF(GicDistributor, icfgr), 2, interrupt_id, static_cast<u32>(mode) << 1);
     }
 
     void SetPending(int interrupt_id) {
-        Write(g_distributor_address + offsetof(GicDistributor, ispendr), 1, interrupt_id, 1);
+        Write(g_distributor_address + AMS_OFFSETOF(GicDistributor, ispendr), 1, interrupt_id, 1);
     }
 
     int GetInterruptRequestId() {

libraries/libmesosphere/Makefile

@@ -104,6 +104,18 @@ $(eval $(call ATMOSPHERE_ADD_TARGET, audit, $(TARGET)_audit.a, \
 	ATMOSPHERE_BUILD_SETTINGS="-DMESOSPHERE_BUILD_FOR_AUDITING" \
 ))
 
+$(eval $(call ATMOSPHERE_ADD_TARGET, qemu_virt_release, $(TARGET)_qemu_virt.a, \
+	ATMOSPHERE_BUILD_SETTINGS="" \
+))
+
+$(eval $(call ATMOSPHERE_ADD_TARGET, qemu_virt_debug, $(TARGET)_qemu_virt_debug.a, \
+	ATMOSPHERE_BUILD_SETTINGS="-DMESOSPHERE_BUILD_FOR_DEBUGGING" \
+))
+
+$(eval $(call ATMOSPHERE_ADD_TARGET, qemu_virt_audit, $(TARGET)_qemu_virt_audit.a, \
+	ATMOSPHERE_BUILD_SETTINGS="-DMESOSPHERE_BUILD_FOR_AUDITING" \
+))
+
 #---------------------------------------------------------------------------------
 
 -include $(ATMOSPHERE_BOARD_NAME)_$(ATMOSPHERE_ARCH_NAME).mk
@@ -142,8 +154,6 @@ $(OFILES_SRC)	: $(HFILES_BIN)
 
 kern_libc_generic.o: CFLAGS += -fno-builtin
 
-kern_k_auto_object.o kern_k_debug_base_process_holder.o: CXXFLAGS += -fno-lto
-
 #---------------------------------------------------------------------------------
 %_bin.h %.bin.o	:	%.bin
 #---------------------------------------------------------------------------------

libraries/libmesosphere/include/mesosphere.hpp

@@ -90,3 +90,6 @@
 
 /* Main functionality. */
 #include <mesosphere/kern_main.hpp>
+
+/* Deferred includes. */
+#include <mesosphere/kern_k_auto_object_impls.hpp>

libraries/libmesosphere/include/mesosphere/arch/arm/kern_generic_interrupt_controller.hpp

@@ -248,8 +248,9 @@ namespace ams::kern::arch::arm {
                 return id;
             }
         private:
-            static constexpr size_t PriorityShift = BITSIZEOF(u8) - __builtin_ctz(NumPriorityLevels);
+            static constexpr size_t PriorityShift = BITSIZEOF(u8) - util::CountTrailingZeros(NumPriorityLevels);
             static_assert(PriorityShift < BITSIZEOF(u8));
+            static_assert(util::IsPowerOfTwo(NumPriorityLevels));
 
             static constexpr ALWAYS_INLINE u8 ToGicPriorityValue(s32 level) {
                 return (level << PriorityShift) | ((1 << PriorityShift) - 1);

libraries/libmesosphere/include/mesosphere/arch/arm64/init/kern_k_init_arguments.hpp

@@ -35,17 +35,17 @@ namespace ams::kern::init {
     static_assert(alignof(KInitArguments) == util::CeilingPowerOfTwo(INIT_ARGUMENTS_SIZE));
     static_assert(sizeof(KInitArguments) == std::max(INIT_ARGUMENTS_SIZE, util::CeilingPowerOfTwo(INIT_ARGUMENTS_SIZE)));
 
-    static_assert(__builtin_offsetof(KInitArguments, ttbr0)           == INIT_ARGUMENTS_TTBR0);
-    static_assert(__builtin_offsetof(KInitArguments, ttbr1)           == INIT_ARGUMENTS_TTBR1);
-    static_assert(__builtin_offsetof(KInitArguments, tcr)             == INIT_ARGUMENTS_TCR);
-    static_assert(__builtin_offsetof(KInitArguments, mair)            == INIT_ARGUMENTS_MAIR);
-    static_assert(__builtin_offsetof(KInitArguments, cpuactlr)        == INIT_ARGUMENTS_CPUACTLR);
-    static_assert(__builtin_offsetof(KInitArguments, cpuectlr)        == INIT_ARGUMENTS_CPUECTLR);
-    static_assert(__builtin_offsetof(KInitArguments, sctlr)           == INIT_ARGUMENTS_SCTLR);
-    static_assert(__builtin_offsetof(KInitArguments, sp)              == INIT_ARGUMENTS_SP);
-    static_assert(__builtin_offsetof(KInitArguments, entrypoint)      == INIT_ARGUMENTS_ENTRYPOINT);
-    static_assert(__builtin_offsetof(KInitArguments, argument)        == INIT_ARGUMENTS_ARGUMENT);
-    static_assert(__builtin_offsetof(KInitArguments, setup_function)  == INIT_ARGUMENTS_SETUP_FUNCTION);
-    static_assert(__builtin_offsetof(KInitArguments, exception_stack) == INIT_ARGUMENTS_EXCEPTION_STACK);
+    static_assert(AMS_OFFSETOF(KInitArguments, ttbr0)           == INIT_ARGUMENTS_TTBR0);
+    static_assert(AMS_OFFSETOF(KInitArguments, ttbr1)           == INIT_ARGUMENTS_TTBR1);
+    static_assert(AMS_OFFSETOF(KInitArguments, tcr)             == INIT_ARGUMENTS_TCR);
+    static_assert(AMS_OFFSETOF(KInitArguments, mair)            == INIT_ARGUMENTS_MAIR);
+    static_assert(AMS_OFFSETOF(KInitArguments, cpuactlr)        == INIT_ARGUMENTS_CPUACTLR);
+    static_assert(AMS_OFFSETOF(KInitArguments, cpuectlr)        == INIT_ARGUMENTS_CPUECTLR);
+    static_assert(AMS_OFFSETOF(KInitArguments, sctlr)           == INIT_ARGUMENTS_SCTLR);
+    static_assert(AMS_OFFSETOF(KInitArguments, sp)              == INIT_ARGUMENTS_SP);
+    static_assert(AMS_OFFSETOF(KInitArguments, entrypoint)      == INIT_ARGUMENTS_ENTRYPOINT);
+    static_assert(AMS_OFFSETOF(KInitArguments, argument)        == INIT_ARGUMENTS_ARGUMENT);
+    static_assert(AMS_OFFSETOF(KInitArguments, setup_function)  == INIT_ARGUMENTS_SETUP_FUNCTION);
+    static_assert(AMS_OFFSETOF(KInitArguments, exception_stack) == INIT_ARGUMENTS_EXCEPTION_STACK);
 
 }

libraries/libmesosphere/include/mesosphere/arch/arm64/init/kern_k_init_page_table.hpp

@@ -34,19 +34,22 @@ namespace ams::kern::arch::arm64::init {
         }
     }
 
-    class KInitialPageTable {
+    /* NOTE: Nintendo uses virtual functions, rather than a concept + template. */
+    template<typename T>
+    concept IsInitialPageAllocator = requires (T &t, KPhysicalAddress phys_addr, size_t size) {
+        { t.Allocate(size) }        -> std::same_as<KPhysicalAddress>;
+        { t.Free(phys_addr, size) } -> std::same_as<void>;
+    };
+
+    template<IsInitialPageAllocator _PageAllocator>
+    class KInitialPageTableTemplate {
         public:
-            class IPageAllocator {
-                public:
-                    virtual KPhysicalAddress Allocate(size_t size) = 0;
-                    virtual void Free(KPhysicalAddress phys_addr, size_t size) = 0;
-            };
+            using PageAllocator = _PageAllocator;
         private:
             KPhysicalAddress m_l1_tables[2];
             u32 m_num_entries[2];
-
         public:
-            KInitialPageTable(KVirtualAddress start_address, KVirtualAddress end_address, IPageAllocator &allocator) {
+            KInitialPageTableTemplate(KVirtualAddress start_address, KVirtualAddress end_address, PageAllocator &allocator) {
                 /* Set tables. */
                 m_l1_tables[0] = AllocateNewPageTable(allocator);
                 m_l1_tables[1] = AllocateNewPageTable(allocator);
@@ -56,7 +59,7 @@ namespace ams::kern::arch::arm64::init {
                 m_num_entries[1] = ((end_address / L1BlockSize) & (MaxPageTableEntries - 1)) - ((start_address / L1BlockSize) & (MaxPageTableEntries - 1)) + 1;
             }
 
-            KInitialPageTable() {
+            KInitialPageTableTemplate() {
                 /* Set tables. */
                 m_l1_tables[0] = util::AlignDown(cpu::GetTtbr0El1(), PageSize);
                 m_l1_tables[1] = util::AlignDown(cpu::GetTtbr1El1(), PageSize);
@@ -95,7 +98,7 @@ namespace ams::kern::arch::arm64::init {
                 return l3_table + ((GetInteger(address) / L3BlockSize) & (MaxPageTableEntries - 1));
             }
 
-            static ALWAYS_INLINE KPhysicalAddress AllocateNewPageTable(IPageAllocator &allocator) {
+            static ALWAYS_INLINE KPhysicalAddress AllocateNewPageTable(PageAllocator &allocator) {
                 auto address = allocator.Allocate(PageSize);
                 ClearNewPageTable(address);
                 return address;
@@ -323,7 +326,7 @@ namespace ams::kern::arch::arm64::init {
                 }
             }
         public:
-            void NOINLINE Map(KVirtualAddress virt_addr, size_t size, KPhysicalAddress phys_addr, const PageTableEntry &attr, IPageAllocator &allocator) {
+            void NOINLINE Map(KVirtualAddress virt_addr, size_t size, KPhysicalAddress phys_addr, const PageTableEntry &attr, PageAllocator &allocator) {
                 /* Ensure that addresses and sizes are page aligned. */
                 MESOSPHERE_INIT_ABORT_UNLESS(util::IsAligned(GetInteger(virt_addr),    PageSize));
                 MESOSPHERE_INIT_ABORT_UNLESS(util::IsAligned(GetInteger(phys_addr),    PageSize));
@@ -700,10 +703,9 @@ namespace ams::kern::arch::arm64::init {
                 this->PhysicallyRandomize(virt_addr, size, L3BlockSize, do_copy);
                 cpu::StoreEntireCacheForInit();
             }
-
     };
 
-    class KInitialPageAllocator final : public KInitialPageTable::IPageAllocator {
+    class KInitialPageAllocator final {
         private:
             static constexpr inline size_t FreeUnitSize = BITSIZEOF(u64) * PageSize;
             struct FreeListEntry {
@@ -807,11 +809,11 @@ namespace ams::kern::arch::arm64::init {
                 }
             }
 
-            virtual KPhysicalAddress Allocate(size_t size) override {
+            KPhysicalAddress Allocate(size_t size) {
                 return this->Allocate(size, size);
             }
 
-            virtual void Free(KPhysicalAddress phys_addr, size_t size) override {
+            void Free(KPhysicalAddress phys_addr, size_t size) {
                 auto **prev_next = std::addressof(m_state.free_head);
                 auto *new_chunk  = reinterpret_cast<FreeListEntry *>(GetInteger(phys_addr));
                 if (auto *cur = m_state.free_head; cur != nullptr) {
@@ -863,5 +865,8 @@ namespace ams::kern::arch::arm64::init {
                 *prev_next = new_chunk;
             }
     };
+    static_assert(IsInitialPageAllocator<KInitialPageAllocator>);
+
+    using KInitialPageTable = KInitialPageTableTemplate<KInitialPageAllocator>;
 
 }

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_cpu.hpp

@@ -30,6 +30,8 @@ namespace ams::kern::arch::arm64::cpu {
 
 #if defined(ATMOSPHERE_BOARD_NINTENDO_NX)
     constexpr inline size_t NumCores = 4;
+#elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+    constexpr inline size_t NumCores = 4;
 #else
     #error "Unknown Board for cpu::NumCores"
 #endif

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_debug.hpp

@@ -34,8 +34,9 @@ namespace ams::kern::arch::arm64 {
 
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
         public:
-            virtual Result GetThreadContextImpl(ams::svc::ThreadContext *out, KThread *thread, u32 context_flags) override;
-            virtual Result SetThreadContextImpl(const ams::svc::ThreadContext &ctx, KThread *thread, u32 context_flags) override;
+            /* NOTE: These are virtual in Nintendo's kernel. */
+            Result GetThreadContextImpl(ams::svc::ThreadContext *out, KThread *thread, u32 context_flags);
+            Result SetThreadContextImpl(const ams::svc::ThreadContext &ctx, KThread *thread, u32 context_flags);
         private:
             Result GetFpuContext(ams::svc::ThreadContext *out, KThread *thread, u32 context_flags);
             Result SetFpuContext(const ams::svc::ThreadContext &ctx, KThread *thread, u32 context_flags);

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_exception_context.hpp

@@ -45,40 +45,40 @@ namespace ams::kern::arch::arm64 {
     };
     static_assert(sizeof(KExceptionContext) == EXCEPTION_CONTEXT_SIZE);
 
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 0]) == EXCEPTION_CONTEXT_X0);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 1]) == EXCEPTION_CONTEXT_X1);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 2]) == EXCEPTION_CONTEXT_X2);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 3]) == EXCEPTION_CONTEXT_X3);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 4]) == EXCEPTION_CONTEXT_X4);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 5]) == EXCEPTION_CONTEXT_X5);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 6]) == EXCEPTION_CONTEXT_X6);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 7]) == EXCEPTION_CONTEXT_X7);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 8]) == EXCEPTION_CONTEXT_X8);
-    static_assert(__builtin_offsetof(KExceptionContext, x[ 9]) == EXCEPTION_CONTEXT_X9);
-    static_assert(__builtin_offsetof(KExceptionContext, x[10]) == EXCEPTION_CONTEXT_X10);
-    static_assert(__builtin_offsetof(KExceptionContext, x[11]) == EXCEPTION_CONTEXT_X11);
-    static_assert(__builtin_offsetof(KExceptionContext, x[12]) == EXCEPTION_CONTEXT_X12);
-    static_assert(__builtin_offsetof(KExceptionContext, x[13]) == EXCEPTION_CONTEXT_X13);
-    static_assert(__builtin_offsetof(KExceptionContext, x[14]) == EXCEPTION_CONTEXT_X14);
-    static_assert(__builtin_offsetof(KExceptionContext, x[15]) == EXCEPTION_CONTEXT_X15);
-    static_assert(__builtin_offsetof(KExceptionContext, x[16]) == EXCEPTION_CONTEXT_X16);
-    static_assert(__builtin_offsetof(KExceptionContext, x[17]) == EXCEPTION_CONTEXT_X17);
-    static_assert(__builtin_offsetof(KExceptionContext, x[18]) == EXCEPTION_CONTEXT_X18);
-    static_assert(__builtin_offsetof(KExceptionContext, x[19]) == EXCEPTION_CONTEXT_X19);
-    static_assert(__builtin_offsetof(KExceptionContext, x[20]) == EXCEPTION_CONTEXT_X20);
-    static_assert(__builtin_offsetof(KExceptionContext, x[21]) == EXCEPTION_CONTEXT_X21);
-    static_assert(__builtin_offsetof(KExceptionContext, x[22]) == EXCEPTION_CONTEXT_X22);
-    static_assert(__builtin_offsetof(KExceptionContext, x[23]) == EXCEPTION_CONTEXT_X23);
-    static_assert(__builtin_offsetof(KExceptionContext, x[24]) == EXCEPTION_CONTEXT_X24);
-    static_assert(__builtin_offsetof(KExceptionContext, x[25]) == EXCEPTION_CONTEXT_X25);
-    static_assert(__builtin_offsetof(KExceptionContext, x[26]) == EXCEPTION_CONTEXT_X26);
-    static_assert(__builtin_offsetof(KExceptionContext, x[27]) == EXCEPTION_CONTEXT_X27);
-    static_assert(__builtin_offsetof(KExceptionContext, x[28]) == EXCEPTION_CONTEXT_X28);
-    static_assert(__builtin_offsetof(KExceptionContext, x[29]) == EXCEPTION_CONTEXT_X29);
-    static_assert(__builtin_offsetof(KExceptionContext, x[30]) == EXCEPTION_CONTEXT_X30);
-    static_assert(__builtin_offsetof(KExceptionContext, sp)    == EXCEPTION_CONTEXT_SP);
-    static_assert(__builtin_offsetof(KExceptionContext, pc)    == EXCEPTION_CONTEXT_PC);
-    static_assert(__builtin_offsetof(KExceptionContext, psr)   == EXCEPTION_CONTEXT_PSR);
-    static_assert(__builtin_offsetof(KExceptionContext, tpidr) == EXCEPTION_CONTEXT_TPIDR);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 0]) == EXCEPTION_CONTEXT_X0);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 1]) == EXCEPTION_CONTEXT_X1);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 2]) == EXCEPTION_CONTEXT_X2);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 3]) == EXCEPTION_CONTEXT_X3);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 4]) == EXCEPTION_CONTEXT_X4);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 5]) == EXCEPTION_CONTEXT_X5);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 6]) == EXCEPTION_CONTEXT_X6);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 7]) == EXCEPTION_CONTEXT_X7);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 8]) == EXCEPTION_CONTEXT_X8);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[ 9]) == EXCEPTION_CONTEXT_X9);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[10]) == EXCEPTION_CONTEXT_X10);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[11]) == EXCEPTION_CONTEXT_X11);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[12]) == EXCEPTION_CONTEXT_X12);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[13]) == EXCEPTION_CONTEXT_X13);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[14]) == EXCEPTION_CONTEXT_X14);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[15]) == EXCEPTION_CONTEXT_X15);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[16]) == EXCEPTION_CONTEXT_X16);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[17]) == EXCEPTION_CONTEXT_X17);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[18]) == EXCEPTION_CONTEXT_X18);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[19]) == EXCEPTION_CONTEXT_X19);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[20]) == EXCEPTION_CONTEXT_X20);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[21]) == EXCEPTION_CONTEXT_X21);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[22]) == EXCEPTION_CONTEXT_X22);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[23]) == EXCEPTION_CONTEXT_X23);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[24]) == EXCEPTION_CONTEXT_X24);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[25]) == EXCEPTION_CONTEXT_X25);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[26]) == EXCEPTION_CONTEXT_X26);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[27]) == EXCEPTION_CONTEXT_X27);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[28]) == EXCEPTION_CONTEXT_X28);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[29]) == EXCEPTION_CONTEXT_X29);
+    static_assert(AMS_OFFSETOF(KExceptionContext, x[30]) == EXCEPTION_CONTEXT_X30);
+    static_assert(AMS_OFFSETOF(KExceptionContext, sp)    == EXCEPTION_CONTEXT_SP);
+    static_assert(AMS_OFFSETOF(KExceptionContext, pc)    == EXCEPTION_CONTEXT_PC);
+    static_assert(AMS_OFFSETOF(KExceptionContext, psr)   == EXCEPTION_CONTEXT_PSR);
+    static_assert(AMS_OFFSETOF(KExceptionContext, tpidr) == EXCEPTION_CONTEXT_TPIDR);
 
 }

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_interrupt_name.hpp

@@ -36,6 +36,10 @@ namespace ams::kern::arch::arm64 {
             KInterruptName_SecurePhysicalTimer    = 29,
             KInterruptName_NonSecurePhysicalTimer = 30,
             KInterruptName_LegacyNIrq             = 31,
+    #elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+            KInterruptName_VirtualTimer           = 27,
+            KInterruptName_SecurePhysicalTimer    = 29,
+            KInterruptName_NonSecurePhysicalTimer = 30,
     #endif
 
     #if defined(ATMOSPHERE_BOARD_NINTENDO_NX)

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_page_table.hpp

@@ -168,7 +168,8 @@ namespace ams::kern::arch::arm64 {
                 return entry;
             }
         public:
-            constexpr KPageTable() : KPageTableBase(), m_manager(), m_ttbr(), m_asid() { /* ... */ }
+            constexpr explicit KPageTable(util::ConstantInitializeTag) : KPageTableBase(util::ConstantInitialize), m_manager(), m_ttbr(), m_asid() { /* ... */ }
+            explicit KPageTable() { /* ... */ }
 
             static NOINLINE void Initialize(s32 core_id);
 

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_page_table_impl.hpp

@@ -42,7 +42,7 @@ namespace ams::kern::arch::arm64 {
                 const L3PageTableEntry *l3_entry;
             };
         private:
-            static constexpr size_t PageBits  = __builtin_ctzll(PageSize);
+            static constexpr size_t PageBits  = util::CountTrailingZeros(PageSize);
             static constexpr size_t NumLevels = 3;
             static constexpr size_t LevelBits = 9;
             static_assert(NumLevels > 0);
@@ -105,7 +105,9 @@ namespace ams::kern::arch::arm64 {
                 return GetL3EntryFromTable(KMemoryLayout::GetLinearVirtualAddress(entry->GetTable()), address);
             }
         public:
-            constexpr KPageTableImpl() : m_table(), m_is_kernel(), m_num_entries() { /* ... */ }
+            constexpr explicit KPageTableImpl(util::ConstantInitializeTag) : m_table(), m_is_kernel(), m_num_entries() { /* ... */ }
+
+            explicit KPageTableImpl() { /* ... */ }
 
             NOINLINE void InitializeForKernel(void *tb, KVirtualAddress start, KVirtualAddress end);
             NOINLINE void InitializeForProcess(void *tb, KVirtualAddress start, KVirtualAddress end);

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_process_page_table.hpp

@@ -23,8 +23,6 @@ namespace ams::kern::arch::arm64 {
         private:
             KPageTable m_page_table;
         public:
-            constexpr KProcessPageTable() : m_page_table() { /* ... */ }
-
             void Activate(u64 id) {
                 /* Activate the page table with the specified contextidr. */
                 m_page_table.Activate(id);

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_supervisor_page_table.hpp

@@ -25,7 +25,7 @@ namespace ams::kern::arch::arm64 {
             KPageTable m_page_table;
             u64 m_ttbr0_identity[cpu::NumCores];
         public:
-            constexpr KSupervisorPageTable() : m_page_table(), m_ttbr0_identity() { /* ... */ }
+            constexpr KSupervisorPageTable() : m_page_table(util::ConstantInitialize), m_ttbr0_identity() { /* ... */ }
 
             NOINLINE void Initialize(s32 core_id);
 

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_k_thread_context.hpp

@@ -57,7 +57,8 @@ namespace ams::kern::arch::arm64 {
             static void RestoreFpuRegisters64(const KThreadContext &);
             static void RestoreFpuRegisters32(const KThreadContext &);
         public:
-            constexpr explicit KThreadContext() : m_callee_saved(), m_lr(), m_sp(), m_cpacr(), m_fpcr(), m_fpsr(), m_fpu_registers(), m_locked() { /* ... */ }
+            constexpr explicit KThreadContext(util::ConstantInitializeTag) : m_callee_saved(), m_lr(), m_sp(), m_cpacr(), m_fpcr(), m_fpsr(), m_fpu_registers(), m_locked() { /* ... */ }
+            explicit KThreadContext() { /* ... */ }
 
             Result Initialize(KVirtualAddress u_pc, KVirtualAddress k_sp, KVirtualAddress u_sp, uintptr_t arg, bool is_user, bool is_64_bit, bool is_main);
             Result Finalize();
@@ -86,25 +87,25 @@ namespace ams::kern::arch::arm64 {
     consteval bool KThreadContext::ValidateOffsets() {
         static_assert(sizeof(KThreadContext) == THREAD_CONTEXT_SIZE);
 
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.registers) == THREAD_CONTEXT_CPU_REGISTERS);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x19)       == THREAD_CONTEXT_X19);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x20)       == THREAD_CONTEXT_X20);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x21)       == THREAD_CONTEXT_X21);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x22)       == THREAD_CONTEXT_X22);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x23)       == THREAD_CONTEXT_X23);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x24)       == THREAD_CONTEXT_X24);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x25)       == THREAD_CONTEXT_X25);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x26)       == THREAD_CONTEXT_X26);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x27)       == THREAD_CONTEXT_X27);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x28)       == THREAD_CONTEXT_X28);
-        static_assert(__builtin_offsetof(KThreadContext, m_callee_saved.x29)       == THREAD_CONTEXT_X29);
-        static_assert(__builtin_offsetof(KThreadContext, m_lr)                     == THREAD_CONTEXT_LR);
-        static_assert(__builtin_offsetof(KThreadContext, m_sp)                     == THREAD_CONTEXT_SP);
-        static_assert(__builtin_offsetof(KThreadContext, m_cpacr)                  == THREAD_CONTEXT_CPACR);
-        static_assert(__builtin_offsetof(KThreadContext, m_fpcr)                   == THREAD_CONTEXT_FPCR);
-        static_assert(__builtin_offsetof(KThreadContext, m_fpsr)                   == THREAD_CONTEXT_FPSR);
-        static_assert(__builtin_offsetof(KThreadContext, m_fpu_registers)          == THREAD_CONTEXT_FPU_REGISTERS);
-        static_assert(__builtin_offsetof(KThreadContext, m_locked)                 == THREAD_CONTEXT_LOCKED);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.registers) == THREAD_CONTEXT_CPU_REGISTERS);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x19)       == THREAD_CONTEXT_X19);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x20)       == THREAD_CONTEXT_X20);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x21)       == THREAD_CONTEXT_X21);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x22)       == THREAD_CONTEXT_X22);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x23)       == THREAD_CONTEXT_X23);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x24)       == THREAD_CONTEXT_X24);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x25)       == THREAD_CONTEXT_X25);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x26)       == THREAD_CONTEXT_X26);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x27)       == THREAD_CONTEXT_X27);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x28)       == THREAD_CONTEXT_X28);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_callee_saved.x29)       == THREAD_CONTEXT_X29);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_lr)                     == THREAD_CONTEXT_LR);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_sp)                     == THREAD_CONTEXT_SP);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_cpacr)                  == THREAD_CONTEXT_CPACR);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_fpcr)                   == THREAD_CONTEXT_FPCR);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_fpsr)                   == THREAD_CONTEXT_FPSR);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_fpu_registers)          == THREAD_CONTEXT_FPU_REGISTERS);
+        static_assert(AMS_OFFSETOF(KThreadContext, m_locked)                 == THREAD_CONTEXT_LOCKED);
 
         return true;
     }

libraries/libmesosphere/include/mesosphere/arch/arm64/kern_secure_monitor_base.hpp

@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_select_cpu.hpp>
+#include <mesosphere/kern_select_interrupt_manager.hpp>
+
+namespace ams::kern::arch::arm64::smc {
+
+    template<int SmcId, bool DisableInterrupt>
+    void SecureMonitorCall(u64 *buf) {
+        /* Load arguments into registers. */
+        register u64 x0 asm("x0") = buf[0];
+        register u64 x1 asm("x1") = buf[1];
+        register u64 x2 asm("x2") = buf[2];
+        register u64 x3 asm("x3") = buf[3];
+        register u64 x4 asm("x4") = buf[4];
+        register u64 x5 asm("x5") = buf[5];
+        register u64 x6 asm("x6") = buf[6];
+        register u64 x7 asm("x7") = buf[7];
+
+        /* Perform the call. */
+        if constexpr (DisableInterrupt) {
+            KScopedInterruptDisable di;
+
+            /* Backup the current thread pointer. */
+            const uintptr_t current_thread_pointer_value = cpu::GetCurrentThreadPointerValue();
+
+            __asm__ __volatile__("smc %c[smc_id]"
+                                : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
+                                : [smc_id]"i"(SmcId)
+                                : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
+                                );
+
+            /* Restore the current thread pointer into X18. */
+            cpu::SetCurrentThreadPointerValue(current_thread_pointer_value);
+        } else {
+            /* Backup the current thread pointer. */
+            const uintptr_t current_thread_pointer_value = cpu::GetCurrentThreadPointerValue();
+
+            __asm__ __volatile__("smc %c[smc_id]"
+                                : "+r"(x0), "+r"(x1), "+r"(x2), "+r"(x3), "+r"(x4), "+r"(x5), "+r"(x6), "+r"(x7)
+                                : [smc_id]"i"(SmcId)
+                                : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "x18", "cc", "memory"
+                                );
+
+            /* Restore the current thread pointer into X18. */
+            cpu::SetCurrentThreadPointerValue(current_thread_pointer_value);
+        }
+
+        /* Store arguments to output. */
+        buf[0] = x0;
+        buf[1] = x1;
+        buf[2] = x2;
+        buf[3] = x3;
+        buf[4] = x4;
+        buf[5] = x5;
+        buf[6] = x6;
+        buf[7] = x7;
+    }
+
+    enum PsciFunction {
+        PsciFunction_CpuSuspend = 0xC4000001,
+        PsciFunction_CpuOff     = 0x84000002,
+        PsciFunction_CpuOn      = 0xC4000003,
+    };
+
+    template<int SmcId, bool DisableInterrupt>
+    u64 PsciCall(PsciFunction function, u64 x1 = 0, u64 x2 = 0, u64 x3 = 0, u64 x4 = 0, u64 x5 = 0, u64 x6 = 0, u64 x7 = 0) {
+        ams::svc::lp64::SecureMonitorArguments args = { { function, x1, x2, x3, x4, x5, x6, x7 } };
+
+        SecureMonitorCall<SmcId, DisableInterrupt>(args.r);
+
+        return args.r[0];
+    }
+
+    template<int SmcId, bool DisableInterrupt>
+    u64 CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg) {
+        return PsciCall<SmcId, DisableInterrupt>(PsciFunction_CpuOn, core_id, entrypoint, arg);
+    }
+
+}

libraries/libmesosphere/include/mesosphere/board/generic/kern_k_device_page_table.hpp

@@ -44,13 +44,13 @@ namespace ams::kern::board::generic {
                 return ams::kern::svc::ResultNotImplemented();
             }
 
-            Result ALWAYS_INLINE Map(size_t *out_mapped_size, const KPageGroup &pg, KDeviceVirtualAddress device_address, ams::svc::MemoryPermission device_perm, bool refresh_mappings) {
-                MESOSPHERE_UNUSED(out_mapped_size, pg, device_address, device_perm, refresh_mappings);
+            Result ALWAYS_INLINE Map(KProcessPageTable *page_table, KProcessAddress process_address, size_t size, KDeviceVirtualAddress device_address, ams::svc::MemoryPermission device_perm, bool is_aligned) {
+                MESOSPHERE_UNUSED(page_table, process_address, size, device_address, device_perm, is_aligned);
                 return ams::kern::svc::ResultNotImplemented();
             }
 
-            Result ALWAYS_INLINE Unmap(const KPageGroup &pg, KDeviceVirtualAddress device_address) {
-                MESOSPHERE_UNUSED(pg, device_address);
+            Result ALWAYS_INLINE Unmap(KProcessPageTable *page_table, KProcessAddress process_address, size_t size, KDeviceVirtualAddress device_address) {
+                MESOSPHERE_UNUSED(page_table, process_address, size, device_address);
                 return ams::kern::svc::ResultNotImplemented();
             }
 

libraries/libmesosphere/include/mesosphere/board/nintendo/nx/kern_k_device_page_table.hpp

@@ -61,7 +61,12 @@ namespace ams::kern::board::nintendo::nx {
                 return KPageTable::GetPageTablePhysicalAddress(addr);
             }
         public:
-            constexpr KDevicePageTable() : m_tables(), m_table_asids(), m_attached_device(), m_attached_value(), m_detached_value(), m_hs_attached_value(), m_hs_detached_value() { /* ... */ }
+            constexpr KDevicePageTable()
+                : m_tables{Null<KVirtualAddress>, Null<KVirtualAddress>, Null<KVirtualAddress>, Null<KVirtualAddress>},
+                  m_table_asids(), m_attached_device(), m_attached_value(), m_detached_value(), m_hs_attached_value(), m_hs_detached_value()
+            {
+                /* ... */
+            }
 
             Result Initialize(u64 space_address, u64 space_size);
             void Finalize();

libraries/libmesosphere/include/mesosphere/board/nintendo/nx/kern_k_memory_layout.hpp

@@ -15,9 +15,12 @@
  */
 #pragma once
 #include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_typed_address.hpp>
 
 namespace ams::kern {
 
+    constexpr inline KPhysicalAddress MainMemoryAddress = 0x80000000;
+
     constexpr inline size_t MainMemorySize    = 4_GB;
     constexpr inline size_t MainMemorySizeMax = 8_GB;
 

libraries/libmesosphere/include/mesosphere/board/nintendo/nx/kern_k_system_control.hpp

@@ -15,23 +15,17 @@
  */
 #pragma once
 #include <mesosphere/kern_common.hpp>
-
-namespace ams::kern {
-
-    struct InitialProcessBinaryLayout;
-
-}
+#include <mesosphere/kern_k_system_control_base.hpp>
 
 namespace ams::kern::board::nintendo::nx {
 
-    class KSystemControl {
+    class KSystemControl : public KSystemControlBase {
         public:
-            class Init {
+            class Init : public KSystemControlBase::Init {
                 public:
                     /* Initialization. */
+                    static size_t GetRealMemorySize();
                     static size_t GetIntendedMemorySize();
-                    static KPhysicalAddress GetKernelPhysicalBaseAddress(uintptr_t base_address);
-                    static void GetInitialProcessBinaryLayout(InitialProcessBinaryLayout *out);
                     static bool ShouldIncreaseThreadResourceLimit();
                     static void CpuOn(u64 core_id, uintptr_t entrypoint, uintptr_t arg);
                     static size_t GetApplicationPoolSize();
@@ -40,7 +34,7 @@ namespace ams::kern::board::nintendo::nx {
                     static u8 GetDebugLogUartPort();
 
                     /* Randomness. */
-                    static void GenerateRandomBytes(void *dst, size_t size);
+                    static void GenerateRandom(u64 *dst, size_t count);
                     static u64  GenerateRandomRange(u64 min, u64 max);
             };
         public:
@@ -50,7 +44,7 @@ namespace ams::kern::board::nintendo::nx {
             static NOINLINE u32 GetCreateProcessMemoryPool();
 
             /* Randomness. */
-            static void GenerateRandomBytes(void *dst, size_t size);
+            static void GenerateRandom(u64 *dst, size_t count);
             static u64  GenerateRandomRange(u64 min, u64 max);
             static u64  GenerateRandomU64();
 
@@ -58,23 +52,12 @@ namespace ams::kern::board::nintendo::nx {
             static void ReadWriteRegisterPrivileged(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value);
             static Result ReadWriteRegister(u32 *out, ams::svc::PhysicalAddress address, u32 mask, u32 value);
 
-            static ALWAYS_INLINE u32 ReadRegisterPrivileged(ams::svc::PhysicalAddress address) {
-                u32 v;
-                ReadWriteRegisterPrivileged(std::addressof(v), address, 0x00000000u, 0);
-                return v;
-            }
-
-            static ALWAYS_INLINE void WriteRegisterPrivileged(ams::svc::PhysicalAddress address, u32 value) {
-                u32 v;
-                ReadWriteRegisterPrivileged(std::addressof(v), address, 0xFFFFFFFFu, value);
-            }
-
             /* Power management. */
             static void SleepSystem();
             static NORETURN void StopSystem(void *arg = nullptr);
 
             /* User access. */
-            static void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
+            static void CallSecureMonitorFromUserImpl(ams::svc::lp64::SecureMonitorArguments *args);
 
             /* Secure Memory. */
             static size_t CalculateRequiredSecureMemorySize(size_t size, u32 pool);

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_cpu_map.hpp

@@ -0,0 +1,33 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+
+namespace ams::kern::board::qemu::virt::impl::cpu {
+
+    /* Virtual to Physical core map. */
+    constexpr inline const s32 VirtualToPhysicalCoreMap[BITSIZEOF(u64)] = {
+        0, 1, 2, 3, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 0,
+        0, 0, 0, 0, 0, 0, 0, 3,
+    };
+
+}

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_k_memory_layout.hpp

@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_typed_address.hpp>
+
+namespace ams::kern {
+
+    constexpr inline KPhysicalAddress MainMemoryAddress = 0x40000000;
+
+    constexpr inline size_t MainMemorySize    = 4_GB;
+    constexpr inline size_t MainMemorySizeMax = 8_GB;
+
+}

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_k_memory_region_device_types.inc

@@ -0,0 +1,20 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* All architectures must define NumBoardDeviceRegions. */
+constexpr inline const auto NumBoardDeviceRegions = 0;
+                                                                                                /* UNUSED: .Derive(NumBoardDeviceRegions, 0); */
+

libraries/libmesosphere/include/mesosphere/board/qemu/virt/kern_k_system_control.hpp

@@ -0,0 +1,28 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_system_control_base.hpp>
+
+namespace ams::kern::board::qemu::virt {
+
+    class KSystemControl : public KSystemControlBase {
+        public:
+            /* User access. */
+            static void CallSecureMonitorFromUser(ams::svc::lp64::SecureMonitorArguments *args);
+    };
+
+}

libraries/libmesosphere/include/mesosphere/init/kern_init_elf64.hpp

@@ -54,63 +54,63 @@ namespace ams::kern::init::Elf::Elf64 {
 
     class Dyn {
         private:
-            SXword tag;
+            SXword m_tag;
             union {
-                Xword value;
-                Addr ptr;
+                Xword m_value;
+                Addr m_ptr;
             };
         public:
             constexpr ALWAYS_INLINE SXword GetTag() const {
-                return this->tag;
+                return m_tag;
             }
 
             constexpr ALWAYS_INLINE Xword GetValue() const {
-                return this->value;
+                return m_value;
             }
 
             constexpr ALWAYS_INLINE Addr GetPtr() const {
-                return this->ptr;
+                return m_ptr;
             }
     };
 
     class Rel {
         private:
-            Addr offset;
-            Xword info;
+            Addr m_offset;
+            Xword m_info;
         public:
             constexpr ALWAYS_INLINE Addr GetOffset() const {
-                return this->offset;
+                return m_offset;
             }
 
             constexpr ALWAYS_INLINE Xword GetSym() const {
-                return this->info >> 32;
+                return m_info >> 32;
             }
 
             constexpr ALWAYS_INLINE Xword GetType() const {
-                return this->info & 0xFFFFFFFF;
+                return m_info & 0xFFFFFFFF;
             }
     };
 
     class Rela {
         private:
-            Addr offset;
-            Xword info;
-            SXword addend;
+            Addr m_offset;
+            Xword m_info;
+            SXword m_addend;
         public:
             constexpr ALWAYS_INLINE Addr GetOffset() const {
-                return this->offset;
+                return m_offset;
             }
 
             constexpr ALWAYS_INLINE Xword GetSym() const {
-                return this->info >> 32;
+                return m_info >> 32;
             }
 
             constexpr ALWAYS_INLINE Xword GetType() const {
-                return this->info & 0xFFFFFFFF;
+                return m_info & 0xFFFFFFFF;
             }
 
             constexpr ALWAYS_INLINE SXword GetAddend() const {
-                return this->addend;
+                return m_addend;
             }
     };
 

libraries/libmesosphere/include/mesosphere/kern_build_config.hpp

@@ -32,3 +32,19 @@
 //#define MESOSPHERE_BUILD_FOR_TRACING
 #define MESOSPHERE_ENABLE_PANIC_REGISTER_DUMP
 #define MESOSPHERE_ENABLE_HARDWARE_SINGLE_STEP
+
+/* NOTE: This enables fast class token storage using a class member. */
+/* This saves a virtual call when doing KAutoObject->DynCast<>(), */
+/* at the cost of storing class tokens inside the class object. */
+/* However, as of (10/16/2021) KAutoObject has an unused class member */
+/* of the right side, and so this does not actually cost any space. */
+#define MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST
+
+/* NOTE: This uses currently-reserved bits inside the MapRange capability */
+/* in order to support large physical addresses (40-bit instead of 36). */
+/* This is toggleable in order to disable it if N ever uses those bits. */
+#if defined(ATMOSPHERE_BOARD_NINTENDO_NX)
+//#define MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES
+#else
+#define MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES
+#endif

libraries/libmesosphere/include/mesosphere/kern_debug_log.hpp

@@ -43,6 +43,8 @@ namespace ams::kern {
 
     #ifdef ATMOSPHERE_BOARD_NINTENDO_NX
         #define MESOSPHERE_DEBUG_LOG_USE_UART
+    #elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+        #define MESOSPHERE_DEBUG_LOG_USE_SEMIHOSTING
     #else
         #error "Unknown board for Default Debug Log Source"
     #endif

libraries/libmesosphere/include/mesosphere/kern_k_address_arbiter.hpp

@@ -25,7 +25,7 @@ namespace ams::kern {
         private:
             ThreadTree m_tree;
         public:
-            constexpr KAddressArbiter() : m_tree() { /* ... */ }
+            constexpr KAddressArbiter() = default;
 
             Result SignalToAddress(uintptr_t addr, ams::svc::SignalType type, s32 value, s32 count) {
                 switch (type) {

libraries/libmesosphere/include/mesosphere/kern_k_auto_object.hpp

@@ -22,27 +22,73 @@ namespace ams::kern {
 
     class KProcess;
 
+    #if defined(MESOSPHERE_BUILD_FOR_DEBUGGING) || defined(MESOSPHERE_BUILD_FOR_AUDITING)
+    #define MESOSPHERE_AUTO_OBJECT_TYPENAME_IMPL(CLASS) #CLASS
+    #else
+    #define MESOSPHERE_AUTO_OBJECT_TYPENAME_IMPL(CLASS) ""
+    #endif
+
     #define MESOSPHERE_AUTOOBJECT_TRAITS(CLASS, BASE_CLASS)                                                     \
         NON_COPYABLE(CLASS);                                                                                    \
         NON_MOVEABLE(CLASS);                                                                                    \
         private:                                                                                                \
             friend class ::ams::kern::KClassTokenGenerator;                                                     \
             static constexpr inline auto ObjectType = ::ams::kern::KClassTokenGenerator::ObjectType::CLASS;     \
-            static constexpr inline const char * const TypeName = #CLASS;                                       \
+            static constexpr inline const char * const TypeName = MESOSPHERE_AUTO_OBJECT_TYPENAME_IMPL(CLASS);  \
             static constexpr inline ClassTokenType ClassToken() { return ::ams::kern::ClassToken<CLASS>; }      \
         public:                                                                                                 \
             using BaseClass = BASE_CLASS;                                                                       \
-            static constexpr ALWAYS_INLINE TypeObj GetStaticTypeObj() {                                         \
+            static consteval ALWAYS_INLINE TypeObj GetStaticTypeObj() {                                         \
                 constexpr ClassTokenType Token = ClassToken();                                                  \
                 return TypeObj(TypeName, Token);                                                                \
             }                                                                                                   \
-            static constexpr ALWAYS_INLINE const char *GetStaticTypeName() { return TypeName; }                 \
+            static consteval ALWAYS_INLINE const char *GetStaticTypeName() { return TypeName; }                 \
             virtual TypeObj GetTypeObj() const { return GetStaticTypeObj(); }                                   \
             virtual const char *GetTypeName() { return GetStaticTypeName(); }                                   \
         private:
 
-
     class KAutoObject {
+        public:
+            class ReferenceCount {
+                NON_COPYABLE(ReferenceCount);
+                NON_MOVEABLE(ReferenceCount);
+                private:
+                    using Storage = u32;
+                private:
+                    util::Atomic<Storage> m_value;
+                public:
+                    ALWAYS_INLINE explicit ReferenceCount() { /* ... */ }
+                    constexpr ALWAYS_INLINE explicit ReferenceCount(Storage v) : m_value(v) { /* ... */ }
+
+                    ALWAYS_INLINE void operator=(Storage v) { m_value = v; }
+
+                    ALWAYS_INLINE Storage GetValue() const { return m_value.Load(); }
+
+                    ALWAYS_INLINE bool Open() {
+                        /* Atomically increment the reference count, only if it's positive. */
+                        u32 cur = m_value.Load<std::memory_order_relaxed>();
+                        do {
+                            if (AMS_UNLIKELY(cur == 0)) {
+                                MESOSPHERE_AUDIT(cur != 0);
+                                return false;
+                            }
+                            MESOSPHERE_ABORT_UNLESS(cur < cur + 1);
+                        } while (AMS_UNLIKELY(!m_value.CompareExchangeWeak<std::memory_order_relaxed>(cur, cur + 1)));
+
+                        return true;
+                    }
+
+                    ALWAYS_INLINE bool Close() {
+                        /* Atomically decrement the reference count, not allowing it to become negative. */
+                        u32 cur = m_value.Load<std::memory_order_relaxed>();
+                        do {
+                            MESOSPHERE_ABORT_UNLESS(cur > 0);
+                        } while (AMS_UNLIKELY(!m_value.CompareExchangeWeak<std::memory_order_relaxed>(cur, cur - 1)));
+
+                        /* Return whether the object was closed. */
+                        return cur - 1 == 0;
+                    }
+            };
         protected:
             class TypeObj {
                 private:
@@ -63,37 +109,61 @@ namespace ams::kern {
                     }
 
                     constexpr ALWAYS_INLINE bool IsDerivedFrom(const TypeObj &rhs) {
-                        return (this->GetClassToken() | rhs.GetClassToken()) == this->GetClassToken();
+                        return IsClassTokenDerivedFrom(this->GetClassToken(), rhs.GetClassToken());
+                    }
+
+                    static constexpr ALWAYS_INLINE bool IsClassTokenDerivedFrom(ClassTokenType derived, ClassTokenType base) {
+                        return (derived | base) == derived;
                     }
             };
         private:
             MESOSPHERE_AUTOOBJECT_TRAITS(KAutoObject, KAutoObject);
         private:
             KAutoObject *m_next_closed_object;
-            std::atomic<u32> m_ref_count;
+            ReferenceCount m_ref_count;
+            #if defined(MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST)
+            ClassTokenType m_class_token;
+            #endif
         public:
-            static KAutoObject *Create(KAutoObject *ptr);
-        public:
-            constexpr ALWAYS_INLINE explicit KAutoObject() : m_next_closed_object(nullptr), m_ref_count(0) { MESOSPHERE_ASSERT_THIS(); }
+            constexpr ALWAYS_INLINE explicit KAutoObject(util::ConstantInitializeTag) : m_next_closed_object(nullptr), m_ref_count(0)
+            #if defined(MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST)
+                , m_class_token(0)
+            #endif
+            {
+                MESOSPHERE_ASSERT_THIS();
+            }
+
+            ALWAYS_INLINE explicit KAutoObject() : m_ref_count(0) { MESOSPHERE_ASSERT_THIS(); }
 
             /* Destroy is responsible for destroying the auto object's resources when ref_count hits zero. */
             virtual void Destroy() { MESOSPHERE_ASSERT_THIS(); }
 
             /* Finalize is responsible for cleaning up resource, but does not destroy the object. */
-            virtual void Finalize() { MESOSPHERE_ASSERT_THIS(); }
+            /* NOTE: This is a virtual function in official kernel, but because everything which uses it */
+            /* is already using CRTP for slab heap, we have devirtualized it for performance gain. */
+            /* virtual void Finalize() { MESOSPHERE_ASSERT_THIS(); } */
 
-            virtual KProcess *GetOwner() const { return nullptr; }
+            /* NOTE: This is a virtual function which is unused-except-for-debug in Nintendo's kernel. */
+            /* virtual KProcess *GetOwner() const { return nullptr; } */
 
             u32 GetReferenceCount() const {
-                return m_ref_count.load();
+                return m_ref_count.GetValue();
             }
 
             ALWAYS_INLINE bool IsDerivedFrom(const TypeObj &rhs) const {
-                return this->GetTypeObj().IsDerivedFrom(rhs);
+                #if defined(MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST)
+                    return TypeObj::IsClassTokenDerivedFrom(m_class_token, rhs.GetClassToken());
+                #else
+                    return this->GetTypeObj().IsDerivedFrom(rhs);
+                #endif
             }
 
             ALWAYS_INLINE bool IsDerivedFrom(const KAutoObject &rhs) const {
-                return this->IsDerivedFrom(rhs.GetTypeObj());
+                #if defined(MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST)
+                    return TypeObj::IsClassTokenDerivedFrom(m_class_token, rhs.m_class_token);
+                #else
+                    return this->IsDerivedFrom(rhs.GetTypeObj());
+                #endif
             }
 
             template<typename Derived>
@@ -120,8 +190,19 @@ namespace ams::kern {
                 }
             }
 
-            bool Open();
-            void Close();
+            NOINLINE bool Open() {
+                MESOSPHERE_ASSERT_THIS();
+
+                return m_ref_count.Open();
+            }
+
+            NOINLINE void Close() {
+                MESOSPHERE_ASSERT_THIS();
+
+                if (m_ref_count.Close()) {
+                    this->ScheduleDestruction();
+                }
+            }
         private:
             /* NOTE: This has to be defined *after* KThread is defined. */
             /* Nintendo seems to handle this by defining Open/Close() in a cpp, but we'd like them to remain in headers. */
@@ -130,34 +211,46 @@ namespace ams::kern {
         public:
             /* Getter, for KThread. */
             ALWAYS_INLINE KAutoObject *GetNextClosedObject() { return m_next_closed_object; }
+        public:
+            template<typename Derived> requires (std::derived_from<Derived, KAutoObject>)
+            static ALWAYS_INLINE void Create(typename std::type_identity<Derived>::type *obj) {
+                /* Get auto object pointer. */
+                KAutoObject &auto_object = *static_cast<KAutoObject *>(obj);
+
+                /* If we should, set our class token. */
+                #if defined(MESOSPHERE_ENABLE_DEVIRTUALIZED_DYNAMIC_CAST)
+                {
+                    constexpr auto Token = Derived::GetStaticTypeObj().GetClassToken();
+                    auto_object.m_class_token = Token;
+                }
+                #endif
+
+                /* Initialize reference count to 1. */
+                auto_object.m_ref_count = 1;
+            }
     };
 
-    class KAutoObjectWithListContainer;
-
-    class KAutoObjectWithList : public KAutoObject {
+    class KAutoObjectWithListBase : public KAutoObject {
         private:
+            void *m_alignment_forcer_unused[0];
+        public:
+            constexpr ALWAYS_INLINE explicit KAutoObjectWithListBase(util::ConstantInitializeTag) : KAutoObject(util::ConstantInitialize), m_alignment_forcer_unused{} { /* ... */ }
+
+            ALWAYS_INLINE explicit KAutoObjectWithListBase() { /* ... */ }
+    };
+
+    class KAutoObjectWithList : public KAutoObjectWithListBase {
+        private:
+            template<typename>
             friend class KAutoObjectWithListContainer;
         private:
             util::IntrusiveRedBlackTreeNode m_list_node;
         public:
-            constexpr ALWAYS_INLINE KAutoObjectWithList() : m_list_node() { /* ... */ }
-
-            static ALWAYS_INLINE int Compare(const KAutoObjectWithList &lhs, const KAutoObjectWithList &rhs) {
-                const u64 lid = lhs.GetId();
-                const u64 rid = rhs.GetId();
-
-                if (lid < rid) {
-                    return -1;
-                } else if (lid > rid) {
-                    return 1;
-                } else {
-                    return 0;
-                }
-            }
+            constexpr ALWAYS_INLINE KAutoObjectWithList(util::ConstantInitializeTag) : KAutoObjectWithListBase(util::ConstantInitialize), m_list_node(util::ConstantInitialize) { /* ... */ }
+            ALWAYS_INLINE explicit KAutoObjectWithList() { /* ... */ }
         public:
-            virtual u64 GetId() const {
-                return reinterpret_cast<u64>(this);
-            }
+            /* NOTE: This is virtual in Nintendo's kernel. */
+            u64 GetId() const;
     };
 
     template<typename T> requires std::derived_from<T, KAutoObject>
@@ -173,7 +266,6 @@ namespace ams::kern {
                 std::swap(m_obj, rhs.m_obj);
             }
         public:
-            constexpr ALWAYS_INLINE KScopedAutoObject() : m_obj(nullptr) { /* ... */ }
             constexpr ALWAYS_INLINE KScopedAutoObject(T *o) : m_obj(o) {
                 if (m_obj != nullptr) {
                     m_obj->Open();
@@ -228,5 +320,52 @@ namespace ams::kern {
             constexpr ALWAYS_INLINE bool IsNotNull() const { return m_obj != nullptr; }
     };
 
+    template<typename T> requires std::derived_from<T, KAutoObject>
+    class KSharedAutoObject {
+        private:
+            T *m_object;
+            KAutoObject::ReferenceCount m_ref_count;
+        public:
+            explicit KSharedAutoObject() : m_object(nullptr) { /* ... */ }
+
+            void Attach(T *obj) {
+                MESOSPHERE_ASSERT(m_object == nullptr);
+
+                /* Set our object. */
+                m_object = obj;
+
+                /* Open reference to our object. */
+                m_object->Open();
+
+                /* Set our reference count. */
+                m_ref_count = 1;
+            }
+
+            bool Open() {
+                return m_ref_count.Open();
+            }
+
+            void Close() {
+                if (m_ref_count.Close()) {
+                    this->Detach();
+                }
+            }
+
+            ALWAYS_INLINE T *Get() const {
+                return m_object;
+            }
+        private:
+            void Detach() {
+                /* Close our object, if we have one. */
+                if (T * const object = m_object; AMS_LIKELY(object != nullptr)) {
+                    /* Set our object to a debug sentinel value, which will cause crash if accessed. */
+                    m_object = reinterpret_cast<T *>(1);
+
+                    /* Close reference to our object. */
+                    object->Close();
+                }
+            }
+    };
+
 
 }

libraries/libmesosphere/include/mesosphere/kern_k_auto_object_container.hpp

@@ -20,45 +20,131 @@
 
 namespace ams::kern {
 
-    class KAutoObjectWithListContainer {
-        NON_COPYABLE(KAutoObjectWithListContainer);
-        NON_MOVEABLE(KAutoObjectWithListContainer);
-        public:
-            using ListType = util::IntrusiveRedBlackTreeMemberTraits<&KAutoObjectWithList::m_list_node>::TreeType<KAutoObjectWithList>;
-        public:
-            class ListAccessor : public KScopedLightLock {
-                private:
-                    ListType &m_list;
-                public:
-                    explicit ListAccessor(KAutoObjectWithListContainer *container) : KScopedLightLock(container->m_lock), m_list(container->m_object_list) { /* ... */ }
-                    explicit ListAccessor(KAutoObjectWithListContainer &container) : KScopedLightLock(container.m_lock), m_list(container.m_object_list) { /* ... */ }
+    namespace impl {
 
-                    typename ListType::iterator begin() const {
-                        return m_list.begin();
+        template<typename T>
+        struct GetAutoObjectWithListComparator;
+
+        class KAutoObjectWithListContainerBase {
+            NON_COPYABLE(KAutoObjectWithListContainerBase);
+            NON_MOVEABLE(KAutoObjectWithListContainerBase);
+            protected:
+                template<typename ListType>
+                class ListAccessorImpl {
+                    NON_COPYABLE(ListAccessorImpl);
+                    NON_MOVEABLE(ListAccessorImpl);
+                    private:
+                        KScopedLightLock m_lk;
+                        ListType &m_list;
+                    public:
+                        explicit ALWAYS_INLINE ListAccessorImpl(KAutoObjectWithListContainerBase *container, ListType &list) : m_lk(container->m_lock), m_list(list) { /* ... */ }
+                        explicit ALWAYS_INLINE ListAccessorImpl(KAutoObjectWithListContainerBase &container, ListType &list) : m_lk(container.m_lock), m_list(list) { /* ... */ }
+
+                        ALWAYS_INLINE ~ListAccessorImpl() { /* ... */ }
+
+                        ALWAYS_INLINE typename ListType::iterator begin() const {
+                            return m_list.begin();
+                        }
+
+                        ALWAYS_INLINE typename ListType::iterator end() const {
+                            return m_list.end();
+                        }
+
+                        ALWAYS_INLINE typename ListType::iterator find(typename ListType::const_reference ref) const {
+                            return m_list.find(ref);
+                        }
+
+                        ALWAYS_INLINE typename ListType::iterator find_key(typename ListType::const_key_reference ref) const {
+                            return m_list.find_key(ref);
+                        }
+                };
+
+                template<typename ListType>
+                friend class ListAccessorImpl;
+            private:
+                KLightLock m_lock;
+            protected:
+                constexpr KAutoObjectWithListContainerBase() : m_lock() { /* ... */ }
+
+                ALWAYS_INLINE void InitializeImpl() { MESOSPHERE_ASSERT_THIS(); }
+                ALWAYS_INLINE void FinalizeImpl() { MESOSPHERE_ASSERT_THIS(); }
+
+                template<typename ListType>
+                void RegisterImpl(KAutoObjectWithList *obj, ListType &list) {
+                    MESOSPHERE_ASSERT_THIS();
+
+                    KScopedLightLock lk(m_lock);
+
+                    list.insert(*obj);
+                }
+
+                template<typename ListType>
+                void UnregisterImpl(KAutoObjectWithList *obj, ListType &list) {
+                    MESOSPHERE_ASSERT_THIS();
+
+                    KScopedLightLock lk(m_lock);
+
+                    list.erase(list.iterator_to(*obj));
+                }
+
+                template<typename U, typename ListType>
+                size_t GetOwnedCountImpl(const KProcess *owner, ListType &list) {
+                    MESOSPHERE_ASSERT_THIS();
+
+                    KScopedLightLock lk(m_lock);
+
+                    size_t count = 0;
+
+                    for (const auto &obj : list) {
+                        MESOSPHERE_AUDIT(obj.DynamicCast<const U *>() != nullptr);
+                        if (static_cast<const U &>(obj).GetOwner() == owner) {
+                            ++count;
+                        }
                     }
 
-                    typename ListType::iterator end() const {
-                        return m_list.end();
-                    }
+                    return count;
+                }
+        };
 
-                    typename ListType::iterator find(typename ListType::const_reference ref) const {
-                        return m_list.find(ref);
-                    }
-            };
+        struct DummyKAutoObjectWithListComparator {
+            static NOINLINE int Compare(KAutoObjectWithList &lhs, KAutoObjectWithList &rhs) {
+                MESOSPHERE_UNUSED(lhs, rhs);
+                MESOSPHERE_PANIC("DummyKAutoObjectWithListComparator invoked");
+            }
+        };
 
-            friend class ListAccessor;
+    }
+
+    template<typename T>
+    class KAutoObjectWithListContainer : public impl::KAutoObjectWithListContainerBase {
         private:
-            KLightLock m_lock;
-            ListType m_object_list;
+            using Base = impl::KAutoObjectWithListContainerBase;
         public:
-            constexpr KAutoObjectWithListContainer() : m_lock(), m_object_list() { MESOSPHERE_ASSERT_THIS(); }
+            class ListAccessor;
+            friend class ListAccessor;
 
-            void Initialize() { MESOSPHERE_ASSERT_THIS(); }
-            void Finalize() { MESOSPHERE_ASSERT_THIS(); }
+            template<typename Comparator>
+            using ListType = util::IntrusiveRedBlackTreeMemberTraits<&KAutoObjectWithList::m_list_node>::TreeType<Comparator>;
 
-            void Register(KAutoObjectWithList *obj);
-            void Unregister(KAutoObjectWithList *obj);
-            size_t GetOwnedCount(KProcess *owner);
+            using DummyListType = ListType<impl::DummyKAutoObjectWithListComparator>;
+        private:
+            DummyListType m_dummy_object_list;
+        public:
+            constexpr ALWAYS_INLINE KAutoObjectWithListContainer() : Base(), m_dummy_object_list() { static_assert(std::derived_from<T, KAutoObjectWithList>); }
+
+            ALWAYS_INLINE void Initialize() { return this->InitializeImpl(); }
+            ALWAYS_INLINE void Finalize() { return this->FinalizeImpl(); }
+
+            void Register(T *obj);
+            void Unregister(T *obj);
+
+        private:
+            size_t GetOwnedCountChecked(const KProcess *owner);
+        public:
+            template<typename U> requires (std::same_as<U, T> && requires (const U &u) { { u.GetOwner() } -> std::convertible_to<const KProcess *>; })
+            ALWAYS_INLINE size_t GetOwnedCount(const KProcess *owner) {
+                return this->GetOwnedCountChecked(owner);
+            }
     };
 
 

libraries/libmesosphere/include/mesosphere/kern_k_auto_object_impls.hpp

@@ -0,0 +1,163 @@
+/*
+ * Copyright (c) Atmosphère-NX
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#pragma once
+#include <mesosphere/kern_common.hpp>
+#include <mesosphere/kern_k_class_token.hpp>
+
+namespace ams::kern {
+
+    /* NOTE: This header is included after all other KAutoObjects. */
+    namespace impl {
+
+        template<typename T> requires std::derived_from<T, KAutoObject>
+        consteval bool IsAutoObjectInheritanceValidImpl() {
+            #define CLASS_TOKEN_HANDLER(CLASSNAME)                                                                               \
+                if constexpr (std::same_as<T, CLASSNAME>) {                                                                      \
+                    if (T::GetStaticTypeObj().GetClassToken() != ::ams::kern::ClassToken<CLASSNAME>) {                           \
+                        return false;                                                                                            \
+                    }                                                                                                            \
+                } else {                                                                                                         \
+                    if (T::GetStaticTypeObj().IsDerivedFrom(CLASSNAME::GetStaticTypeObj()) != std::derived_from<T, CLASSNAME>) { \
+                        return false;                                                                                            \
+                    }                                                                                                            \
+                }
+
+            FOR_EACH_K_CLASS_TOKEN_OBJECT_TYPE(CLASS_TOKEN_HANDLER)
+            #undef CLASS_TOKEN_HANDLER
+
+            return true;
+        }
+
+        consteval bool IsEveryAutoObjectInheritanceValid() {
+            #define CLASS_TOKEN_HANDLER(CLASSNAME) if (!IsAutoObjectInheritanceValidImpl<CLASSNAME>()) { return false; }
+            FOR_EACH_K_CLASS_TOKEN_OBJECT_TYPE(CLASS_TOKEN_HANDLER)
+            #undef CLASS_TOKEN_HANDLER
+
+            return true;
+        }
+
+        static_assert(IsEveryAutoObjectInheritanceValid());
+
+        template<typename T>
+        concept IsAutoObjectWithSpecializedGetId = std::derived_from<T, KAutoObjectWithList> && requires (const T &t, const KAutoObjectWithList &l) {
+            { t.GetIdImpl() } -> std::same_as<decltype(l.GetId())>;
+        };
+
+        template<typename T>
+        struct AutoObjectWithListComparatorImpl {
+            using RedBlackKeyType = u64;
+
+            static ALWAYS_INLINE RedBlackKeyType GetRedBlackKey(const RedBlackKeyType &v) { return v; }
+
+            static ALWAYS_INLINE RedBlackKeyType GetRedBlackKey(const KAutoObjectWithList &v) {
+                if constexpr (IsAutoObjectWithSpecializedGetId<T>) {
+                    return static_cast<const T &>(v).GetIdImpl();
+                } else {
+                    return reinterpret_cast<u64>(std::addressof(v));
+                }
+            }
+
+            template<typename U> requires (std::same_as<U, KAutoObjectWithList> || std::same_as<U, RedBlackKeyType>)
+            static ALWAYS_INLINE int Compare(const U &lhs, const KAutoObjectWithList &rhs) {
+                const u64 lid = GetRedBlackKey(lhs);
+                const u64 rid = GetRedBlackKey(rhs);
+
+                if (lid < rid) {
+                    return -1;
+                } else if (lid > rid) {
+                    return 1;
+                } else {
+                    return 0;
+                }
+            }
+        };
+
+        template<typename T>
+        using AutoObjectWithListComparator = AutoObjectWithListComparatorImpl<typename std::conditional<IsAutoObjectWithSpecializedGetId<T>, T, KAutoObjectWithList>::type>;
+
+        template<typename T>
+        using TrueObjectContainerListType = typename KAutoObjectWithListContainer<T>::ListType<AutoObjectWithListComparator<T>>;
+
+        template<typename T>
+        ALWAYS_INLINE TrueObjectContainerListType<T> &GetTrueObjectContainerList(typename KAutoObjectWithListContainer<T>::DummyListType &l) {
+            static_assert(alignof(l) == alignof(impl::TrueObjectContainerListType<T>));
+            static_assert(sizeof(l)  == sizeof(impl::TrueObjectContainerListType<T>));
+            return *reinterpret_cast<TrueObjectContainerListType<T> *>(std::addressof(l));
+        }
+
+    }
+
+    ALWAYS_INLINE void KAutoObject::ScheduleDestruction() {
+        MESOSPHERE_ASSERT_THIS();
+
+        /* Set our object to destroy. */
+        m_next_closed_object = GetCurrentThread().GetClosedObject();
+
+        /* Set ourselves as the thread's next object to destroy. */
+        GetCurrentThread().SetClosedObject(this);
+    }
+
+    template<typename T>
+    class KAutoObjectWithListContainer<T>::ListAccessor : public impl::KAutoObjectWithListContainerBase::ListAccessorImpl<impl::TrueObjectContainerListType<T>> {
+        NON_COPYABLE(ListAccessor);
+        NON_MOVEABLE(ListAccessor);
+        private:
+            using BaseListAccessor = impl::KAutoObjectWithListContainerBase::ListAccessorImpl<impl::TrueObjectContainerListType<T>>;
+        public:
+            explicit ALWAYS_INLINE ListAccessor(KAutoObjectWithListContainer *container) : BaseListAccessor(container, impl::GetTrueObjectContainerList<T>(container->m_dummy_object_list)) { /* ... */ }
+            explicit ALWAYS_INLINE ListAccessor(KAutoObjectWithListContainer &container) : BaseListAccessor(container, impl::GetTrueObjectContainerList<T>(container.m_dummy_object_list)) { /* ... */ }
+
+            ALWAYS_INLINE ~ListAccessor() { /* ... */ }
+    };
+
+
+    template<typename T>
+    ALWAYS_INLINE void KAutoObjectWithListContainer<T>::Register(T *obj) {
+        return this->RegisterImpl(obj, impl::GetTrueObjectContainerList<T>(m_dummy_object_list));
+    }
+
+    template<typename T>
+    ALWAYS_INLINE void KAutoObjectWithListContainer<T>::Unregister(T *obj) {
+        return this->UnregisterImpl(obj, impl::GetTrueObjectContainerList<T>(m_dummy_object_list));
+    }
+
+    template<typename T>
+    ALWAYS_INLINE size_t KAutoObjectWithListContainer<T>::GetOwnedCountChecked(const KProcess *owner) {
+        return this->GetOwnedCountImpl<T>(owner, impl::GetTrueObjectContainerList<T>(m_dummy_object_list));
+    }
+
+    inline u64 KAutoObjectWithList::GetId() const {
+        #define CLASS_TOKEN_HANDLER(CLASSNAME)                                                                      \
+            if constexpr (impl::IsAutoObjectWithSpecializedGetId<CLASSNAME>) {                                      \
+                if (const CLASSNAME * const derived = this->DynamicCast<const CLASSNAME *>(); derived != nullptr) { \
+                    return []<typename T>(const T * const t_derived) ALWAYS_INLINE_LAMBDA -> u64 {                  \
+                        static_assert(std::same_as<T, CLASSNAME>);                                                  \
+                        if constexpr (impl::IsAutoObjectWithSpecializedGetId<CLASSNAME>) {                          \
+                            return impl::AutoObjectWithListComparator<CLASSNAME>::GetRedBlackKey(*t_derived);       \
+                        } else {                                                                                    \
+                            AMS_ASSUME(false);                                                                      \
+                        }                                                                                           \
+                    }(derived);                                                                                     \
+                }                                                                                                   \
+            }
+
+        FOR_EACH_K_CLASS_TOKEN_OBJECT_TYPE(CLASS_TOKEN_HANDLER)
+        #undef CLASS_TOKEN_HANDLER
+
+        return impl::AutoObjectWithListComparator<KAutoObjectWithList>::GetRedBlackKey(*this);
+    }
+
+}

libraries/libmesosphere/include/mesosphere/kern_k_capabilities.hpp

@@ -55,48 +55,16 @@ namespace ams::kern {
                 return static_cast<u32>(type) + 1;
             }
 
-            static constexpr u32 CountTrailingZero(u32 flag) {
-                for (u32 i = 0; i < BITSIZEOF(u32); i++) {
-                    if (flag & (1u << i)) {
-                        return i;
-                    }
-                }
-                return BITSIZEOF(u32);
-            }
-
-            static constexpr u32 GetCapabilityId(CapabilityType type) {
-                const u32 flag = GetCapabilityFlag(type);
-                if (std::is_constant_evaluated()) {
-                    return CountTrailingZero(flag);
-                } else {
-                    return static_cast<u32>(__builtin_ctz(flag));
-                }
-            }
-
             template<size_t Index, size_t Count, typename T = u32>
             using Field = util::BitPack32::Field<Index, Count, T>;
 
             #define DEFINE_FIELD(name, prev, ...) using name = Field<prev::Next, __VA_ARGS__>
 
             template<CapabilityType Type>
-            static constexpr inline u32 CapabilityFlag = []() -> u32 {
-                return static_cast<u32>(Type) + 1;
-            }();
+            static constexpr inline u32 CapabilityFlag = static_cast<u32>(Type) + 1;
 
             template<CapabilityType Type>
-            static constexpr inline u32 CapabilityId = []() -> u32 {
-                const u32 flag = static_cast<u32>(Type) + 1;
-                if (std::is_constant_evaluated()) {
-                    for (u32 i = 0; i < BITSIZEOF(u32); i++) {
-                        if (flag & (1u << i)) {
-                            return i;
-                        }
-                    }
-                    return BITSIZEOF(u32);
-                } else {
-                    return __builtin_ctz(flag);
-                }
-            }();
+            static constexpr inline u32 CapabilityId = util::CountTrailingZeros<u32>(CapabilityFlag<Type>);
 
             struct CorePriority {
                 using IdBits = Field<0, CapabilityId<CapabilityType::CorePriority> + 1>;
@@ -114,7 +82,11 @@ namespace ams::kern {
                 DEFINE_FIELD(Index, Mask,    3);
             };
 
+            #if defined(MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES)
+            static constexpr u64 PhysicalMapAllowedMask = (1ul << 40) - 1;
+            #else
             static constexpr u64 PhysicalMapAllowedMask = (1ul << 36) - 1;
+            #endif
 
             struct MapRange {
                 using IdBits = Field<0, CapabilityId<CapabilityType::MapRange> + 1>;
@@ -126,9 +98,15 @@ namespace ams::kern {
             struct MapRangeSize {
                 using IdBits = Field<0, CapabilityId<CapabilityType::MapRange> + 1>;
 
-                DEFINE_FIELD(Pages,    IdBits,   20);
+                DEFINE_FIELD(Pages, IdBits, 20);
+
+                #if defined(MESOSPHERE_ENABLE_LARGE_PHYSICAL_ADDRESS_CAPABILITIES)
+                DEFINE_FIELD(AddressHigh, Pages,        4);
+                DEFINE_FIELD(Normal,      AddressHigh,  1, bool);
+                #else
                 DEFINE_FIELD(Reserved, Pages,     4);
                 DEFINE_FIELD(Normal,   Reserved,  1, bool);
+                #endif
             };
 
             struct MapIoPage {
@@ -203,14 +181,14 @@ namespace ams::kern {
                                                        CapabilityFlag<CapabilityType::HandleTable>   |
                                                        CapabilityFlag<CapabilityType::DebugFlags>;
         private:
-            svc::SvcAccessFlagSet m_svc_access_flags{};
-            InterruptFlagSet m_irq_access_flags{};
-            u64 m_core_mask{};
-            u64 m_priority_mask{};
-            util::BitPack32 m_debug_capabilities{0};
-            s32 m_handle_table_size{};
-            util::BitPack32 m_intended_kernel_version{0};
-            u32 m_program_type{};
+            svc::SvcAccessFlagSet m_svc_access_flags;
+            InterruptFlagSet m_irq_access_flags;
+            u64 m_core_mask;
+            u64 m_priority_mask;
+            util::BitPack32 m_debug_capabilities;
+            s32 m_handle_table_size;
+            util::BitPack32 m_intended_kernel_version;
+            u32 m_program_type;
         private:
             constexpr bool SetSvcAllowed(u32 id) {
                 if (AMS_LIKELY(id < m_svc_access_flags.GetCount())) {
@@ -245,7 +223,8 @@ namespace ams::kern {
             Result SetCapabilities(const u32 *caps, s32 num_caps, KProcessPageTable *page_table);
             Result SetCapabilities(svc::KUserPointer<const u32 *> user_caps, s32 num_caps, KProcessPageTable *page_table);
         public:
-            constexpr KCapabilities() = default;
+            constexpr explicit KCapabilities(util::ConstantInitializeTag) : m_svc_access_flags{}, m_irq_access_flags{}, m_core_mask{}, m_priority_mask{}, m_debug_capabilities{0}, m_handle_table_size{}, m_intended_kernel_version{}, m_program_type{} { /* ... */ }
+            KCapabilities() { /* ... */ }
 
             Result Initialize(const u32 *caps, s32 num_caps, KProcessPageTable *page_table);
             Result Initialize(svc::KUserPointer<const u32 *> user_caps, s32 num_caps, KProcessPageTable *page_table);

libraries/libmesosphere/include/mesosphere/kern_k_class_token.hpp

@@ -21,6 +21,35 @@ namespace ams::kern {
 
     class KAutoObject;
 
+    #define FOR_EACH_K_CLASS_TOKEN_OBJECT_TYPE(HANDLER) \
+                HANDLER(KAutoObject)                    \
+                                                        \
+                HANDLER(KSynchronizationObject)         \
+                HANDLER(KReadableEvent)                 \
+                                                        \
+                HANDLER(KInterruptEvent)                \
+                HANDLER(KDebug)                         \
+                HANDLER(KThread)                        \
+                HANDLER(KServerPort)                    \
+                HANDLER(KServerSession)                 \
+                HANDLER(KClientPort)                    \
+                HANDLER(KClientSession)                 \
+                HANDLER(KProcess)                       \
+                HANDLER(KResourceLimit)                 \
+                HANDLER(KLightSession)                  \
+                HANDLER(KPort)                          \
+                HANDLER(KSession)                       \
+                HANDLER(KSharedMemory)                  \
+                HANDLER(KEvent)                         \
+                HANDLER(KLightClientSession)            \
+                HANDLER(KLightServerSession)            \
+                HANDLER(KTransferMemory)                \
+                HANDLER(KDeviceAddressSpace)            \
+                HANDLER(KSessionRequest)                \
+                HANDLER(KCodeMemory)                    \
+                HANDLER(KIoPool)                        \
+                HANDLER(KIoRegion)
+
     class KClassTokenGenerator {
         public:
             using TokenBaseType = u16;
@@ -113,8 +142,11 @@ namespace ams::kern {
                 KIoPool,
                 KIoRegion,
 
+                FinalClassesLast,
+
                 FinalClassesEnd = FinalClassesStart + NumFinalClasses,
             };
+            static_assert(ObjectType::FinalClassesLast <= ObjectType::FinalClassesEnd);
 
             template<typename T>
             static constexpr inline TokenBaseType ClassToken = GetClassToken<T>();
@@ -125,4 +157,37 @@ namespace ams::kern {
     template<typename T>
     static constexpr inline ClassTokenType ClassToken = KClassTokenGenerator::ClassToken<T>;
 
+    namespace impl {
+
+        consteval bool IsKClassTokenGeneratorForEachMacroValid() {
+            auto IsObjectTypeIncludedByMacro = [](KClassTokenGenerator::ObjectType object_type) -> bool {
+                #define CLASS_TOKEN_HANDLER(CLASSNAME) if (object_type == KClassTokenGenerator::ObjectType::CLASSNAME) { return true; }
+                FOR_EACH_K_CLASS_TOKEN_OBJECT_TYPE(CLASS_TOKEN_HANDLER)
+                #undef CLASS_TOKEN_HANDLER
+                return false;
+            };
+
+            if (!IsObjectTypeIncludedByMacro(KClassTokenGenerator::ObjectType::KAutoObject)) {
+                return false;
+            }
+
+            for (auto base = util::ToUnderlying(KClassTokenGenerator::ObjectType::BaseClassesStart); base < util::ToUnderlying(KClassTokenGenerator::ObjectType::BaseClassesEnd); ++base) {
+                if (!IsObjectTypeIncludedByMacro(static_cast<KClassTokenGenerator::ObjectType>(base))) {
+                    return false;
+                }
+            }
+
+            for (auto fin = util::ToUnderlying(KClassTokenGenerator::ObjectType::FinalClassesStart); fin < util::ToUnderlying(KClassTokenGenerator::ObjectType::FinalClassesLast); ++fin) {
+                if (!IsObjectTypeIncludedByMacro(static_cast<KClassTokenGenerator::ObjectType>(fin))) {
+                    return false;
+                }
+            }
+
+            return true;
+        }
+
+        static_assert(IsKClassTokenGeneratorForEachMacroValid());
+
+    }
+
 }

libraries/libmesosphere/include/mesosphere/kern_k_client_port.hpp

@@ -28,12 +28,14 @@ namespace ams::kern {
     class KClientPort final : public KSynchronizationObject {
         MESOSPHERE_AUTOOBJECT_TRAITS(KClientPort, KSynchronizationObject);
         private:
-            std::atomic<s32> m_num_sessions;
-            std::atomic<s32> m_peak_sessions;
+            util::Atomic<s32> m_num_sessions;
+            util::Atomic<s32> m_peak_sessions;
             s32 m_max_sessions;
             KPort *m_parent;
         public:
-            constexpr KClientPort() : m_num_sessions(), m_peak_sessions(), m_max_sessions(), m_parent() { /* ... */ }
+            constexpr explicit KClientPort(util::ConstantInitializeTag) : KSynchronizationObject(util::ConstantInitialize), m_num_sessions(0), m_peak_sessions(0), m_max_sessions(), m_parent() { /* ... */ }
+
+            explicit KClientPort() { /* ... */ }
 
             void Initialize(KPort *parent, s32 max_sessions);
             void OnSessionFinalized();
@@ -41,8 +43,8 @@ namespace ams::kern {
 
             constexpr const KPort *GetParent() const { return m_parent; }
 
-            ALWAYS_INLINE s32 GetNumSessions()  const { return m_num_sessions; }
-            ALWAYS_INLINE s32 GetPeakSessions() const { return m_peak_sessions; }
+            ALWAYS_INLINE s32 GetNumSessions()  const { return m_num_sessions.Load(); }
+            ALWAYS_INLINE s32 GetPeakSessions() const { return m_peak_sessions.Load(); }
             ALWAYS_INLINE s32 GetMaxSessions()  const { return m_max_sessions; }
 
             bool IsLight() const;

libraries/libmesosphere/include/mesosphere/kern_k_client_session.hpp

@@ -27,7 +27,8 @@ namespace ams::kern {
         private:
             KSession *m_parent;
         public:
-            constexpr KClientSession() : m_parent() { /* ... */ }
+            constexpr explicit KClientSession(util::ConstantInitializeTag) : KAutoObject(util::ConstantInitialize), m_parent() { /* ... */ }
+            explicit KClientSession() { /* ... */ }
 
             void Initialize(KSession *parent) {
                 /* Set member variables. */
@@ -35,7 +36,6 @@ namespace ams::kern {
             }
 
             virtual void Destroy() override;
-            static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             constexpr KSession *GetParent() const { return m_parent; }
 

libraries/libmesosphere/include/mesosphere/kern_k_code_memory.hpp

@@ -36,14 +36,14 @@ namespace ams::kern {
             }
 
             Result Initialize(KProcessAddress address, size_t size);
-            virtual void Finalize() override;
+            void Finalize();
 
             Result Map(KProcessAddress address, size_t size);
             Result Unmap(KProcessAddress address, size_t size);
             Result MapToOwner(KProcessAddress address, size_t size, ams::svc::MemoryPermission perm);
             Result UnmapFromOwner(KProcessAddress address, size_t size);
 
-            virtual bool IsInitialized() const override { return m_is_initialized; }
+            bool IsInitialized() const { return m_is_initialized; }
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             KProcess *GetOwner() const { return m_owner; }

libraries/libmesosphere/include/mesosphere/kern_k_condition_variable.hpp

@@ -26,7 +26,7 @@ namespace ams::kern {
         private:
             ThreadTree m_tree;
         public:
-            constexpr KConditionVariable() : m_tree() { /* ... */ }
+            constexpr KConditionVariable() = default;
 
             /* Arbitration. */
             static Result SignalToAddress(KProcessAddress addr);

libraries/libmesosphere/include/mesosphere/kern_k_debug_base.hpp

@@ -25,35 +25,20 @@ namespace ams::kern {
     class KDebugBase : public KSynchronizationObject {
         protected:
             using DebugEventList = util::IntrusiveListBaseTraits<KEventInfo>::ListType;
-        private:
-            class ProcessHolder {
-                private:
-                    friend class KDebugBase;
-                private:
-                    KProcess *m_process;
-                    std::atomic<u32> m_ref_count;
-                private:
-                    explicit ProcessHolder() : m_process(nullptr) { /* ... */ }
-
-                    void Attach(KProcess *process);
-                    void Detach();
-
-                    bool Open();
-                    void Close();
-            };
         private:
             DebugEventList m_event_info_list;
             u32 m_continue_flags;
-            ProcessHolder m_process_holder;
+            KSharedAutoObject<KProcess> m_process_holder;
             KLightLock m_lock;
             KProcess::State m_old_process_state;
             bool m_is_attached;
         public:
-            explicit KDebugBase() : m_event_info_list(), m_process_holder(), m_lock() { /* ... */ }
+            explicit KDebugBase() { /* ... */ }
         protected:
             bool Is64Bit() const;
         public:
             void Initialize();
+            void Finalize();
 
             Result Attach(KProcess *process);
             Result BreakProcess();
@@ -68,9 +53,6 @@ namespace ams::kern {
             Result GetThreadContext(ams::svc::ThreadContext *out, u64 thread_id, u32 context_flags);
             Result SetThreadContext(const ams::svc::ThreadContext &ctx, u64 thread_id, u32 context_flags);
 
-            virtual Result GetThreadContextImpl(ams::svc::ThreadContext *out, KThread *thread, u32 context_flags) = 0;
-            virtual Result SetThreadContextImpl(const ams::svc::ThreadContext &ctx, KThread *thread, u32 context_flags) = 0;
-
             Result GetRunningThreadInfo(ams::svc::LastThreadContext *out_context, u64 *out_thread_id);
 
             Result GetDebugEventInfo(ams::svc::lp64::DebugEventInfo *out);
@@ -89,7 +71,7 @@ namespace ams::kern {
             }
 
             ALWAYS_INLINE KProcess *GetProcessUnsafe() const {
-                return m_process_holder.m_process;
+                return m_process_holder.Get();
             }
         private:
             void PushDebugEvent(ams::svc::DebugEvent event, uintptr_t param0 = 0, uintptr_t param1 = 0, uintptr_t param2 = 0, uintptr_t param3 = 0, uintptr_t param4 = 0);
@@ -98,8 +80,10 @@ namespace ams::kern {
             template<typename T> requires (std::same_as<T, ams::svc::lp64::DebugEventInfo> || std::same_as<T, ams::svc::ilp32::DebugEventInfo>)
             Result GetDebugEventInfoImpl(T *out);
         public:
-            virtual void OnFinalizeSynchronizationObject() override;
             virtual bool IsSignaled() const override;
+        private:
+            /* NOTE: This is public/virtual override in Nintendo's kernel. */
+            void OnFinalizeSynchronizationObject();
         private:
             static Result ProcessDebugEvent(ams::svc::DebugEvent event, uintptr_t param0, uintptr_t param1, uintptr_t param2, uintptr_t param3, uintptr_t param4);
         public:

libraries/libmesosphere/include/mesosphere/kern_k_device_address_space.hpp

@@ -30,12 +30,12 @@ namespace ams::kern {
             u64 m_space_size;
             bool m_is_initialized;
         public:
-            constexpr KDeviceAddressSpace() : m_lock(), m_table(), m_space_address(), m_space_size(), m_is_initialized() { /* ... */ }
+            explicit KDeviceAddressSpace() : m_is_initialized(false) { /* ... */ }
 
             Result Initialize(u64 address, u64 size);
-            virtual void Finalize() override;
+            void Finalize();
 
-            virtual bool IsInitialized() const override { return m_is_initialized; }
+            bool IsInitialized() const { return m_is_initialized; }
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             Result Attach(ams::svc::DeviceName device_name);

libraries/libmesosphere/include/mesosphere/kern_k_dynamic_page_manager.hpp

@@ -40,7 +40,7 @@ namespace ams::kern {
             KVirtualAddress m_address;
             size_t m_size;
         public:
-            KDynamicPageManager() : m_lock(), m_page_bitmap(), m_used(), m_peak(), m_count(), m_address(), m_size() { /* ... */ }
+            KDynamicPageManager() : m_lock(), m_page_bitmap(), m_used(), m_peak(), m_count(), m_address(Null<KVirtualAddress>), m_size() { /* ... */ }
 
             Result Initialize(KVirtualAddress memory, size_t sz) {
                 /* We need to have positive size. */

libraries/libmesosphere/include/mesosphere/kern_k_dynamic_slab_heap.hpp

@@ -29,19 +29,19 @@ namespace ams::kern {
         private:
             using PageBuffer = KDynamicPageManager::PageBuffer;
         private:
-            std::atomic<size_t> m_used{};
-            std::atomic<size_t> m_peak{};
-            std::atomic<size_t> m_count{};
-            KVirtualAddress m_address{};
+            util::Atomic<size_t> m_used{0};
+            util::Atomic<size_t> m_peak{0};
+            util::Atomic<size_t> m_count{0};
+            KVirtualAddress m_address{Null<KVirtualAddress>};
             size_t m_size{};
         public:
             constexpr KDynamicSlabHeap() = default;
 
             constexpr ALWAYS_INLINE KVirtualAddress GetAddress() const { return m_address; }
             constexpr ALWAYS_INLINE size_t GetSize() const { return m_size; }
-            constexpr ALWAYS_INLINE size_t GetUsed() const { return m_used.load(); }
-            constexpr ALWAYS_INLINE size_t GetPeak() const { return m_peak.load(); }
-            constexpr ALWAYS_INLINE size_t GetCount() const { return m_count.load(); }
+            constexpr ALWAYS_INLINE size_t GetUsed() const { return m_used.Load(); }
+            constexpr ALWAYS_INLINE size_t GetPeak() const { return m_peak.Load(); }
+            constexpr ALWAYS_INLINE size_t GetCount() const { return m_count.Load(); }
 
             constexpr ALWAYS_INLINE bool IsInRange(KVirtualAddress addr) const {
                 return this->GetAddress() <= addr && addr <= this->GetAddress() + this->GetSize() - 1;
@@ -58,7 +58,7 @@ namespace ams::kern {
                 KSlabHeapImpl::Initialize();
 
                 /* Allocate until we have the correct number of objects. */
-                while (m_count.load() < num_objects) {
+                while (m_count.Load() < num_objects) {
                     auto *allocated = reinterpret_cast<T *>(page_allocator->Allocate());
                     MESOSPHERE_ABORT_UNLESS(allocated != nullptr);
 
@@ -66,7 +66,7 @@ namespace ams::kern {
                         KSlabHeapImpl::Free(allocated + i);
                     }
 
-                    m_count.fetch_add(sizeof(PageBuffer) / sizeof(T));
+                    m_count += sizeof(PageBuffer) / sizeof(T);
                 }
             }
 
@@ -89,7 +89,7 @@ namespace ams::kern {
                             for (size_t i = 1; i < sizeof(PageBuffer) / sizeof(T); i++) {
                                 KSlabHeapImpl::Free(allocated + i);
                             }
-                            m_count.fetch_add(sizeof(PageBuffer) / sizeof(T));
+                            m_count += sizeof(PageBuffer) / sizeof(T);
                         }
                     }
                 }
@@ -99,10 +99,10 @@ namespace ams::kern {
                     std::construct_at(allocated);
 
                     /* Update our tracking. */
-                    size_t used = m_used.fetch_add(1) + 1;
-                    size_t peak = m_peak.load();
+                    const size_t used = ++m_used;
+                    size_t peak = m_peak.Load();
                     while (peak < used) {
-                        if (m_peak.compare_exchange_weak(peak, used, std::memory_order_relaxed)) {
+                        if (m_peak.CompareExchangeWeak<std::memory_order_relaxed>(peak, used)) {
                             break;
                         }
                     }
@@ -113,7 +113,7 @@ namespace ams::kern {
 
             ALWAYS_INLINE void Free(T *t) {
                 KSlabHeapImpl::Free(t);
-                m_used.fetch_sub(1);
+                --m_used;
             }
     };
 

libraries/libmesosphere/include/mesosphere/kern_k_event.hpp

@@ -29,21 +29,24 @@ namespace ams::kern {
             bool m_initialized;
             bool m_readable_event_destroyed;
         public:
-            constexpr KEvent()
-                : m_readable_event(), m_owner(), m_initialized(), m_readable_event_destroyed()
+            constexpr explicit KEvent(util::ConstantInitializeTag)
+                : KAutoObjectWithSlabHeapAndContainer<KEvent, KAutoObjectWithList, true>(util::ConstantInitialize),
+                  m_readable_event(util::ConstantInitialize), m_owner(), m_initialized(), m_readable_event_destroyed()
             {
                 /* ... */
             }
 
-            void Initialize();
-            virtual void Finalize() override;
+            explicit KEvent() : m_readable_event(), m_owner(), m_initialized(), m_readable_event_destroyed() { /* ... */ }
 
-            virtual bool IsInitialized() const override { return m_initialized; }
-            virtual uintptr_t GetPostDestroyArgument() const override { return reinterpret_cast<uintptr_t>(m_owner); }
+            void Initialize();
+            void Finalize();
+
+            bool IsInitialized() const { return m_initialized; }
+            uintptr_t GetPostDestroyArgument() const { return reinterpret_cast<uintptr_t>(m_owner); }
 
             static void PostDestroy(uintptr_t arg);
 
-            virtual KProcess *GetOwner() const override { return m_owner; }
+            KProcess *GetOwner() const { return m_owner; }
 
             KReadableEvent &GetReadableEvent() { return m_readable_event; }
 

libraries/libmesosphere/include/mesosphere/kern_k_handle_table.hpp

@@ -63,16 +63,16 @@ namespace ams::kern {
         private:
             EntryInfo m_entry_infos[MaxTableSize];
             KAutoObject *m_objects[MaxTableSize];
+            mutable KSpinLock m_lock;
             s32 m_free_head_index;
             u16 m_table_size;
             u16 m_max_count;
             u16 m_next_linear_id;
             u16 m_count;
-            mutable KSpinLock m_lock;
         public:
-            constexpr KHandleTable() :
-                m_entry_infos(), m_objects(), m_free_head_index(-1), m_table_size(0), m_max_count(0), m_next_linear_id(MinLinearId), m_count(0), m_lock()
-            { MESOSPHERE_ASSERT_THIS(); }
+            constexpr explicit KHandleTable(util::ConstantInitializeTag) : m_entry_infos(), m_objects(), m_lock(), m_free_head_index(-1), m_table_size(), m_max_count(), m_next_linear_id(MinLinearId), m_count() { /* ... */ }
+
+            explicit KHandleTable() : m_lock(), m_free_head_index(-1), m_count() { MESOSPHERE_ASSERT_THIS(); }
 
             constexpr NOINLINE Result Initialize(s32 size) {
                 MESOSPHERE_ASSERT_THIS();

libraries/libmesosphere/include/mesosphere/kern_k_interrupt_event.hpp

@@ -31,14 +31,25 @@ namespace ams::kern {
             s32 m_core_id;
             bool m_is_initialized;
         public:
-            constexpr KInterruptEvent() : m_interrupt_id(-1), m_core_id(-1), m_is_initialized(false) { /* ... */ }
+            constexpr explicit KInterruptEvent(util::ConstantInitializeTag) : KAutoObjectWithSlabHeapAndContainer<KInterruptEvent, KReadableEvent>(util::ConstantInitialize), m_interrupt_id(-1), m_core_id(-1), m_is_initialized(false) { /* ... */ }
+
+            explicit KInterruptEvent() : m_interrupt_id(-1), m_is_initialized(false) { /* ... */ }
 
             Result Initialize(int32_t interrupt_name, ams::svc::InterruptType type);
-            virtual void Finalize() override;
+            void Finalize();
 
-            virtual Result Reset() override;
+            Result Reset();
 
-            virtual bool IsInitialized() const override { return m_is_initialized; }
+            Result Clear() {
+                MESOSPHERE_ASSERT_THIS();
+
+                /* Try to perform a reset, succeeding unconditionally. */
+                this->Reset();
+
+                return ResultSuccess();
+            }
+
+            bool IsInitialized() const { return m_is_initialized; }
 
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 

libraries/libmesosphere/include/mesosphere/kern_k_io_pool.hpp

@@ -33,14 +33,14 @@ namespace ams::kern {
         public:
             static bool IsValidIoPoolType(ams::svc::IoPoolType pool_type);
         public:
-            explicit KIoPool() : m_lock(), m_io_region_list(), m_is_initialized(false) {
+            explicit KIoPool() : m_is_initialized(false) {
                 /* ... */
             }
 
             Result Initialize(ams::svc::IoPoolType pool_type);
-            virtual void Finalize() override;
+            void Finalize();
 
-            virtual bool IsInitialized() const override { return m_is_initialized; }
+            bool IsInitialized() const { return m_is_initialized; }
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             Result AddIoRegion(KIoRegion *region);

libraries/libmesosphere/include/mesosphere/kern_k_io_region.hpp

@@ -40,16 +40,12 @@ namespace ams::kern {
             util::IntrusiveListNode m_process_list_node;
             util::IntrusiveListNode m_pool_list_node;
         public:
-            explicit KIoRegion()
-                : m_lock(), m_pool(nullptr), m_is_initialized(false), m_process_list_node(), m_pool_list_node()
-            {
-                /* ... */
-            }
+            explicit KIoRegion() : m_pool(nullptr), m_is_initialized(false) { /* ... */ }
 
             Result Initialize(KIoPool *pool, KPhysicalAddress phys_addr, size_t size, ams::svc::MemoryMapping mapping, ams::svc::MemoryPermission perm);
-            virtual void Finalize() override;
+            void Finalize();
 
-            virtual bool IsInitialized() const override { return m_is_initialized; }
+            bool IsInitialized() const { return m_is_initialized; }
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             Result Map(KProcessAddress address, size_t size, ams::svc::MemoryPermission map_perm);

libraries/libmesosphere/include/mesosphere/kern_k_light_client_session.hpp

@@ -26,7 +26,7 @@ namespace ams::kern {
         private:
             KLightSession *m_parent;
         public:
-            constexpr KLightClientSession() : m_parent() { /* ... */ }
+            explicit KLightClientSession() { /* ... */ }
 
             void Initialize(KLightSession *parent) {
                 /* Set member variables. */
@@ -34,7 +34,6 @@ namespace ams::kern {
             }
 
             virtual void Destroy() override;
-            static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             constexpr const KLightSession *GetParent() const { return m_parent; }
 

libraries/libmesosphere/include/mesosphere/kern_k_light_condition_variable.hpp

@@ -26,7 +26,9 @@ namespace ams::kern {
         private:
             KThread::WaiterList m_wait_list;
         public:
-            constexpr ALWAYS_INLINE KLightConditionVariable() : m_wait_list() { /* ... */ }
+            constexpr explicit ALWAYS_INLINE KLightConditionVariable(util::ConstantInitializeTag) : m_wait_list() { /* ... */ }
+
+            explicit ALWAYS_INLINE KLightConditionVariable() { /* ... */ }
         public:
             void Wait(KLightLock *lock, s64 timeout = -1ll, bool allow_terminating_thread = true);
             void Broadcast();

libraries/libmesosphere/include/mesosphere/kern_k_light_lock.hpp

@@ -23,30 +23,25 @@ namespace ams::kern {
 
     class KLightLock {
         private:
-            std::atomic<uintptr_t> m_tag;
+            util::Atomic<uintptr_t> m_tag;
         public:
-            constexpr KLightLock() : m_tag(0) { /* ... */ }
+            constexpr ALWAYS_INLINE KLightLock() : m_tag(0) { /* ... */ }
 
             void Lock() {
                 MESOSPHERE_ASSERT_THIS();
 
                 const uintptr_t cur_thread = reinterpret_cast<uintptr_t>(GetCurrentThreadPointer());
-                const uintptr_t cur_thread_tag = (cur_thread | 1);
 
                 while (true) {
-                    uintptr_t old_tag = m_tag.load(std::memory_order_relaxed);
+                    uintptr_t old_tag = m_tag.Load<std::memory_order_relaxed>();
 
-                    while (!m_tag.compare_exchange_weak(old_tag, (old_tag == 0) ? cur_thread : old_tag | 1, std::memory_order_acquire)) {
-                        if ((old_tag | 1) == cur_thread_tag) {
-                            return;
-                        }
+                    while (!m_tag.CompareExchangeWeak<std::memory_order_acquire>(old_tag, (old_tag == 0) ? cur_thread : (old_tag | 1))) {
+                        /* ... */
                     }
 
-                    if ((old_tag == 0) || ((old_tag | 1) == cur_thread_tag)) {
+                    if (old_tag == 0 || this->LockSlowPath(old_tag | 1, cur_thread)) {
                         break;
                     }
-
-                    this->LockSlowPath(old_tag | 1, cur_thread);
                 }
             }
 
@@ -54,19 +49,18 @@ namespace ams::kern {
                 MESOSPHERE_ASSERT_THIS();
 
                 const uintptr_t cur_thread = reinterpret_cast<uintptr_t>(GetCurrentThreadPointer());
+
                 uintptr_t expected = cur_thread;
-                do {
-                    if (expected != cur_thread) {
-                        return this->UnlockSlowPath(cur_thread);
-                    }
-                } while (!m_tag.compare_exchange_weak(expected, 0, std::memory_order_release));
+                if (!m_tag.CompareExchangeStrong<std::memory_order_release>(expected, 0)) {
+                    this->UnlockSlowPath(cur_thread);
+                }
             }
 
-            void LockSlowPath(uintptr_t owner, uintptr_t cur_thread);
+            bool LockSlowPath(uintptr_t owner, uintptr_t cur_thread);
             void UnlockSlowPath(uintptr_t cur_thread);
 
-            ALWAYS_INLINE bool IsLocked() const { return m_tag.load() != 0; }
-            ALWAYS_INLINE bool IsLockedByCurrentThread() const { return (m_tag.load() | 0x1ul) == (reinterpret_cast<uintptr_t>(GetCurrentThreadPointer()) | 0x1ul); }
+            ALWAYS_INLINE bool IsLocked() const { return m_tag.Load() != 0; }
+            ALWAYS_INLINE bool IsLockedByCurrentThread() const { return (m_tag.Load() | 0x1ul) == (reinterpret_cast<uintptr_t>(GetCurrentThreadPointer()) | 0x1ul); }
     };
 
     using KScopedLightLock = KScopedLock<KLightLock>;

libraries/libmesosphere/include/mesosphere/kern_k_light_server_session.hpp

@@ -32,7 +32,7 @@ namespace ams::kern {
             u64 m_server_thread_id;
             KThread *m_server_thread;
         public:
-            constexpr KLightServerSession() : m_parent(), m_request_list(), m_current_request(), m_server_thread_id(), m_server_thread() { /* ... */ }
+            explicit KLightServerSession() : m_current_request(nullptr), m_server_thread_id(std::numeric_limits<u64>::max()), m_server_thread() { /* ... */ }
 
             void Initialize(KLightSession *parent) {
                 /* Set member variables. */
@@ -40,7 +40,6 @@ namespace ams::kern {
             }
 
             virtual void Destroy() override;
-            static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             constexpr const KLightSession *GetParent() const { return m_parent; }
 

libraries/libmesosphere/include/mesosphere/kern_k_light_session.hpp

@@ -46,17 +46,13 @@ namespace ams::kern {
             KProcess *m_process;
             bool m_initialized;
         public:
-            constexpr KLightSession()
-                : m_server(), m_client(), m_state(State::Invalid), m_port(), m_name(), m_process(), m_initialized()
-            {
-                /* ... */
-            }
+            explicit KLightSession() : m_state(State::Invalid), m_process(), m_initialized() { /* ... */ }
 
             void Initialize(KClientPort *client_port, uintptr_t name);
-            virtual void Finalize() override;
+            void Finalize();
 
-            virtual bool IsInitialized() const override { return m_initialized; }
-            virtual uintptr_t GetPostDestroyArgument() const override { return reinterpret_cast<uintptr_t>(m_process); }
+            bool IsInitialized() const { return m_initialized; }
+            uintptr_t GetPostDestroyArgument() const { return reinterpret_cast<uintptr_t>(m_process); }
 
             static void PostDestroy(uintptr_t arg);
 

libraries/libmesosphere/include/mesosphere/kern_k_memory_block.hpp

@@ -349,28 +349,30 @@ namespace ams::kern {
                 };
             }
         public:
-            constexpr KMemoryBlock()
-                : m_device_disable_merge_left_count(), m_device_disable_merge_right_count(), m_address(), m_num_pages(), m_memory_state(KMemoryState_None), m_ipc_lock_count(), m_device_use_count(), m_ipc_disable_merge_count(), m_permission(), m_original_permission(), m_attribute(), m_disable_merge_attribute()
-            {
-                /* ... */
-            }
+            explicit KMemoryBlock() { /* ... */ }
 
-            constexpr KMemoryBlock(KProcessAddress addr, size_t np, KMemoryState ms, KMemoryPermission p, KMemoryAttribute attr)
-                : m_device_disable_merge_left_count(), m_device_disable_merge_right_count(), m_address(addr), m_num_pages(np), m_memory_state(ms), m_ipc_lock_count(0), m_device_use_count(0), m_ipc_disable_merge_count(), m_permission(p), m_original_permission(KMemoryPermission_None), m_attribute(attr), m_disable_merge_attribute()
+            constexpr KMemoryBlock(util::ConstantInitializeTag, KProcessAddress addr, size_t np, KMemoryState ms, KMemoryPermission p, KMemoryAttribute attr)
+                : util::IntrusiveRedBlackTreeBaseNode<KMemoryBlock>(util::ConstantInitialize), m_device_disable_merge_left_count(),
+                  m_device_disable_merge_right_count(), m_address(addr), m_num_pages(np), m_memory_state(ms), m_ipc_lock_count(0),
+                  m_device_use_count(0), m_ipc_disable_merge_count(), m_permission(p), m_original_permission(KMemoryPermission_None),
+                  m_attribute(attr), m_disable_merge_attribute()
             {
                 /* ... */
             }
 
             constexpr void Initialize(KProcessAddress addr, size_t np, KMemoryState ms, KMemoryPermission p, KMemoryAttribute attr) {
                 MESOSPHERE_ASSERT_THIS();
-                m_address             = addr;
-                m_num_pages           = np;
-                m_memory_state        = ms;
-                m_ipc_lock_count      = 0;
-                m_device_use_count    = 0;
-                m_permission          = p;
-                m_original_permission = KMemoryPermission_None;
-                m_attribute           = attr;
+                m_device_disable_merge_left_count  = 0;
+                m_device_disable_merge_right_count = 0;
+                m_address                          = addr;
+                m_num_pages                        = np;
+                m_memory_state                     = ms;
+                m_ipc_lock_count                   = 0;
+                m_device_use_count                 = 0;
+                m_permission                       = p;
+                m_original_permission              = KMemoryPermission_None;
+                m_attribute                        = attr;
+                m_disable_merge_attribute          = KMemoryBlockDisableMergeAttribute_None;
             }
 
             constexpr bool HasProperties(KMemoryState s, KMemoryPermission p, KMemoryAttribute a) const {

libraries/libmesosphere/include/mesosphere/kern_k_memory_block_manager.hpp

@@ -88,7 +88,9 @@ namespace ams::kern {
         private:
             void CoalesceForUpdate(KMemoryBlockManagerUpdateAllocator *allocator, KProcessAddress address, size_t num_pages);
         public:
-            constexpr KMemoryBlockManager() : m_memory_block_tree(), m_start_address(), m_end_address() { /* ... */ }
+            constexpr explicit KMemoryBlockManager(util::ConstantInitializeTag) : m_memory_block_tree(), m_start_address(Null<KProcessAddress>), m_end_address(Null<KProcessAddress>) { /* ... */ }
+
+            explicit KMemoryBlockManager() { /* ... */ }
 
             iterator end() { return m_memory_block_tree.end(); }
             const_iterator end() const { return m_memory_block_tree.end(); }
@@ -105,7 +107,7 @@ namespace ams::kern {
             void UpdateIfMatch(KMemoryBlockManagerUpdateAllocator *allocator, KProcessAddress address, size_t num_pages, KMemoryState test_state, KMemoryPermission test_perm, KMemoryAttribute test_attr, KMemoryState state, KMemoryPermission perm, KMemoryAttribute attr);
 
             iterator FindIterator(KProcessAddress address) const {
-                return m_memory_block_tree.find(KMemoryBlock(address, 1, KMemoryState_Free, KMemoryPermission_None, KMemoryAttribute_None));
+                return m_memory_block_tree.find(KMemoryBlock(util::ConstantInitialize, address, 1, KMemoryState_Free, KMemoryPermission_None, KMemoryAttribute_None));
             }
 
             const KMemoryBlock *FindBlock(KProcessAddress address) const {

libraries/libmesosphere/include/mesosphere/kern_k_memory_layout.hpp

@@ -20,6 +20,8 @@
 
 #if defined(ATMOSPHERE_BOARD_NINTENDO_NX)
     #include <mesosphere/board/nintendo/nx/kern_k_memory_layout.hpp>
+#elif defined(ATMOSPHERE_BOARD_QEMU_VIRT)
+    #include <mesosphere/board/qemu/virt/kern_k_memory_layout.hpp>
 #else
     #error "Unknown board for KMemoryLayout"
 #endif
@@ -54,12 +56,12 @@ namespace ams::kern {
 
     class KMemoryLayout {
         private:
-            static /* constinit */ inline uintptr_t s_linear_phys_to_virt_diff;
-            static /* constinit */ inline uintptr_t s_linear_virt_to_phys_diff;
-            static /* constinit */ inline KMemoryRegionTree s_virtual_tree;
-            static /* constinit */ inline KMemoryRegionTree s_physical_tree;
-            static /* constinit */ inline KMemoryRegionTree s_virtual_linear_tree;
-            static /* constinit */ inline KMemoryRegionTree s_physical_linear_tree;
+            static constinit inline uintptr_t s_linear_phys_to_virt_diff;
+            static constinit inline uintptr_t s_linear_virt_to_phys_diff;
+            static constinit inline KMemoryRegionTree s_virtual_tree;
+            static constinit inline KMemoryRegionTree s_physical_tree;
+            static constinit inline KMemoryRegionTree s_virtual_linear_tree;
+            static constinit inline KMemoryRegionTree s_physical_linear_tree;
         private:
             template<typename AddressType> requires IsKTypedAddress<AddressType>
             static ALWAYS_INLINE bool IsTypedAddress(const KMemoryRegion *&region, AddressType address, KMemoryRegionTree &tree, KMemoryRegionType type) {
@@ -210,13 +212,17 @@ namespace ams::kern {
             static NOINLINE auto GetKernelPageTableHeapRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramKernelPtHeap); }
             static NOINLINE auto GetKernelInitPageTableRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramKernelInitPt); }
 
-            static NOINLINE auto GetKernelPoolManagementRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramPoolManagement); }
             static NOINLINE auto GetKernelPoolPartitionRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramPoolPartition); }
+            static NOINLINE auto GetKernelPoolManagementRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramPoolManagement); }
             static NOINLINE auto GetKernelSystemPoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramSystemPool); }
             static NOINLINE auto GetKernelSystemNonSecurePoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramSystemNonSecurePool); }
             static NOINLINE auto GetKernelAppletPoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramAppletPool); }
             static NOINLINE auto GetKernelApplicationPoolRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_DramApplicationPool); }
 
+            static NOINLINE bool HasKernelSystemNonSecurePoolRegion() { return GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramSystemNonSecurePool) != nullptr; }
+            static NOINLINE bool HasKernelAppletPoolRegion() { return GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramAppletPool) != nullptr; }
+            static NOINLINE bool HasKernelApplicationPoolRegion() { return GetPhysicalMemoryRegionTree().FindFirstDerived(KMemoryRegionType_DramApplicationPool) != nullptr; }
+
             static NOINLINE auto GetKernelTraceBufferRegionPhysicalExtents() { return GetPhysicalMemoryRegionTree().GetDerivedRegionExtents(KMemoryRegionType_KernelTraceBuffer); }
     };
 

libraries/libmesosphere/include/mesosphere/kern_k_memory_manager.hpp

@@ -68,7 +68,7 @@ namespace ams::kern {
                     Impl *m_next;
                     Impl *m_prev;
                 public:
-                    Impl() : m_heap(), m_page_reference_counts(), m_management_region(), m_pool(), m_next(), m_prev() { /* ... */ }
+                    Impl() : m_heap(), m_page_reference_counts(), m_management_region(Null<KVirtualAddress>), m_pool(), m_next(), m_prev() { /* ... */ }
 
                     size_t Initialize(KPhysicalAddress address, size_t size, KVirtualAddress management, KVirtualAddress management_end, Pool p);
 

libraries/libmesosphere/include/mesosphere/kern_k_memory_region.hpp

@@ -43,13 +43,14 @@ namespace ams::kern {
                 }
             }
         public:
-            constexpr ALWAYS_INLINE KMemoryRegion() : m_address(0), m_pair_address(0), m_last_address(0), m_attributes(0), m_type_id(0) { /* ... */ }
-            constexpr ALWAYS_INLINE KMemoryRegion(uintptr_t a, size_t la, uintptr_t p, u32 r, u32 t) :
+            constexpr ALWAYS_INLINE KMemoryRegion() : util::IntrusiveRedBlackTreeBaseNode<KMemoryRegion>(util::ConstantInitialize), m_address(0), m_pair_address(0), m_last_address(0), m_attributes(0), m_type_id(0) { /* ... */ }
+
+            ALWAYS_INLINE KMemoryRegion(uintptr_t a, size_t la, uintptr_t p, u32 r, u32 t) :
                 m_address(a), m_pair_address(p), m_last_address(la), m_attributes(r), m_type_id(t)
             {
                 /* ... */
             }
-            constexpr ALWAYS_INLINE KMemoryRegion(uintptr_t a, size_t la, u32 r, u32 t) : KMemoryRegion(a, la, std::numeric_limits<uintptr_t>::max(), r, t) { /* ... */ }
+            ALWAYS_INLINE KMemoryRegion(uintptr_t a, size_t la, u32 r, u32 t) : KMemoryRegion(a, la, std::numeric_limits<uintptr_t>::max(), r, t) { /* ... */ }
         private:
             constexpr ALWAYS_INLINE void Reset(uintptr_t a, uintptr_t la, uintptr_t p, u32 r, u32 t) {
                 m_address      = a;

libraries/libmesosphere/include/mesosphere/kern_k_object_name.hpp

@@ -29,8 +29,6 @@ namespace ams::kern {
         private:
             char m_name[NameLengthMax];
             KAutoObject *m_object;
-        public:
-            constexpr KObjectName() : m_name(), m_object() { /* ... */ }
         public:
             static Result NewFromName(KAutoObject *obj, const char *name);
             static Result Delete(KAutoObject *obj, const char *name);

libraries/libmesosphere/include/mesosphere/kern_k_page_bitmap.hpp

@@ -43,44 +43,25 @@ namespace ams::kern {
                         return rnd_bit;
                     }
                 public:
-                    RandomBitGenerator() : m_rng(), m_entropy(), m_bits_available() {
+                    RandomBitGenerator() : m_entropy(), m_bits_available() {
                         m_rng.Initialize(static_cast<u32>(KSystemControl::GenerateRandomU64()));
                     }
 
                     size_t SelectRandomBit(u64 bitmap) {
-                        u64 selected     = 0;
+                        u64 selected = 0;
 
-                        u64 cur_num_bits = BITSIZEOF(bitmap) / 2;
-                        u64 cur_mask     = (1ull << cur_num_bits) - 1;
+                        for (size_t cur_num_bits = BITSIZEOF(bitmap) / 2; cur_num_bits != 0; cur_num_bits /= 2) {
+                            const u64 high = (bitmap >> cur_num_bits);
+                            const u64 low  = (bitmap & (~(UINT64_C(0xFFFFFFFFFFFFFFFF) << cur_num_bits)));
 
-                        while (cur_num_bits) {
-                            const u64 low  = (bitmap >> 0) & cur_mask;
-                            const u64 high = (bitmap >> cur_num_bits) & cur_mask;
-
-                            bool choose_low;
-                            if (high == 0) {
-                                /* If only low val is set, choose low. */
-                                choose_low = true;
-                            } else if (low == 0) {
-                                /* If only high val is set, choose high. */
-                                choose_low = false;
-                            } else {
-                                /* If both are set, choose random. */
-                                choose_low = this->GenerateRandomBit();
-                            }
-
-                            /* If we chose low, proceed with low. */
-                            if (choose_low) {
-                                bitmap    = low;
-                                selected += 0;
-                            } else {
+                            /* Choose high if we have high and (don't have low or select high randomly). */
+                            if (high && (low == 0 || this->GenerateRandomBit())) {
                                 bitmap    = high;
                                 selected += cur_num_bits;
+                            } else {
+                                bitmap    = low;
+                                selected += 0;
                             }
-
-                            /* Proceed. */
-                            cur_num_bits /= 2;
-                            cur_mask >>= cur_num_bits;
                         }
 
                         return selected;

libraries/libmesosphere/include/mesosphere/kern_k_page_group.hpp

@@ -28,11 +28,11 @@ namespace ams::kern {
         private:
             friend class KPageGroup;
         private:
-            KBlockInfo *m_next{};
-            u32 m_page_index{};
-            u32 m_num_pages{};
+            KBlockInfo *m_next;
+            u32 m_page_index;
+            u32 m_num_pages;
         public:
-            constexpr KBlockInfo() = default;
+            KBlockInfo() : m_next(nullptr) { /* ... */ }
 
             constexpr ALWAYS_INLINE void Initialize(KPhysicalAddress addr, size_t np) {
                 MESOSPHERE_ASSERT(util::IsAligned(GetInteger(addr), PageSize));

libraries/libmesosphere/include/mesosphere/kern_k_page_heap.hpp

@@ -59,7 +59,7 @@ namespace ams::kern {
                     size_t m_block_shift;
                     size_t m_next_block_shift;
                 public:
-                    Block() : m_bitmap(), m_heap_address(), m_end_offset(), m_block_shift(), m_next_block_shift() { /* ... */ }
+                    Block() : m_bitmap(), m_heap_address(Null<KPhysicalAddress>), m_end_offset(), m_block_shift(), m_next_block_shift() { /* ... */ }
 
                     constexpr size_t GetShift() const { return m_block_shift; }
                     constexpr size_t GetNextShift() const { return m_next_block_shift; }
@@ -134,7 +134,7 @@ namespace ams::kern {
 
             void FreeBlock(KPhysicalAddress block, s32 index);
         public:
-            KPageHeap() : m_heap_address(), m_heap_size(), m_initial_used_size(), m_num_blocks(), m_blocks() { /* ... */ }
+            KPageHeap() : m_heap_address(Null<KPhysicalAddress>), m_heap_size(), m_initial_used_size(), m_num_blocks(), m_blocks() { /* ... */ }
 
             constexpr KPhysicalAddress GetAddress() const { return m_heap_address; }
             constexpr size_t GetSize() const { return m_heap_size; }

libraries/libmesosphere/include/mesosphere/kern_k_page_table_base.hpp

@@ -144,45 +144,60 @@ namespace ams::kern {
                     PageLinkedList *GetPageList() { return std::addressof(m_ll); }
             };
         private:
-            KProcessAddress m_address_space_start{};
-            KProcessAddress m_address_space_end{};
-            KProcessAddress m_heap_region_start{};
-            KProcessAddress m_heap_region_end{};
-            KProcessAddress m_current_heap_end{};
-            KProcessAddress m_alias_region_start{};
-            KProcessAddress m_alias_region_end{};
-            KProcessAddress m_stack_region_start{};
-            KProcessAddress m_stack_region_end{};
-            KProcessAddress m_kernel_map_region_start{};
-            KProcessAddress m_kernel_map_region_end{};
-            KProcessAddress m_alias_code_region_start{};
-            KProcessAddress m_alias_code_region_end{};
-            KProcessAddress m_code_region_start{};
-            KProcessAddress m_code_region_end{};
-            size_t m_max_heap_size{};
-            size_t m_mapped_physical_memory_size{};
-            size_t m_mapped_unsafe_physical_memory{};
-            size_t m_mapped_ipc_server_memory{};
-            mutable KLightLock m_general_lock{};
-            mutable KLightLock m_map_physical_memory_lock{};
-            KLightLock m_device_map_lock{};
-            KPageTableImpl m_impl{};
-            KMemoryBlockManager m_memory_block_manager{};
-            u32 m_allocate_option{};
-            u32 m_address_space_width{};
-            bool m_is_kernel{};
-            bool m_enable_aslr{};
-            bool m_enable_device_address_space_merge{};
-            KMemoryBlockSlabManager *m_memory_block_slab_manager{};
-            KBlockInfoManager *m_block_info_manager{};
-            KResourceLimit *m_resource_limit{};
-            const KMemoryRegion *m_cached_physical_linear_region{};
-            const KMemoryRegion *m_cached_physical_heap_region{};
-            MemoryFillValue m_heap_fill_value{};
-            MemoryFillValue m_ipc_fill_value{};
-            MemoryFillValue m_stack_fill_value{};
+            KProcessAddress m_address_space_start;
+            KProcessAddress m_address_space_end;
+            KProcessAddress m_heap_region_start;
+            KProcessAddress m_heap_region_end;
+            KProcessAddress m_current_heap_end;
+            KProcessAddress m_alias_region_start;
+            KProcessAddress m_alias_region_end;
+            KProcessAddress m_stack_region_start;
+            KProcessAddress m_stack_region_end;
+            KProcessAddress m_kernel_map_region_start;
+            KProcessAddress m_kernel_map_region_end;
+            KProcessAddress m_alias_code_region_start;
+            KProcessAddress m_alias_code_region_end;
+            KProcessAddress m_code_region_start;
+            KProcessAddress m_code_region_end;
+            size_t m_max_heap_size;
+            size_t m_mapped_physical_memory_size;
+            size_t m_mapped_unsafe_physical_memory;
+            size_t m_mapped_ipc_server_memory;
+            mutable KLightLock m_general_lock;
+            mutable KLightLock m_map_physical_memory_lock;
+            KLightLock m_device_map_lock;
+            KPageTableImpl m_impl;
+            KMemoryBlockManager m_memory_block_manager;
+            u32 m_allocate_option;
+            u32 m_address_space_width;
+            bool m_is_kernel;
+            bool m_enable_aslr;
+            bool m_enable_device_address_space_merge;
+            KMemoryBlockSlabManager *m_memory_block_slab_manager;
+            KBlockInfoManager *m_block_info_manager;
+            KResourceLimit *m_resource_limit;
+            const KMemoryRegion *m_cached_physical_linear_region;
+            const KMemoryRegion *m_cached_physical_heap_region;
+            MemoryFillValue m_heap_fill_value;
+            MemoryFillValue m_ipc_fill_value;
+            MemoryFillValue m_stack_fill_value;
         public:
-            constexpr KPageTableBase() { /* ... */ }
+            constexpr explicit KPageTableBase(util::ConstantInitializeTag)
+                : m_address_space_start(Null<KProcessAddress>), m_address_space_end(Null<KProcessAddress>), m_heap_region_start(Null<KProcessAddress>),
+                  m_heap_region_end(Null<KProcessAddress>), m_current_heap_end(Null<KProcessAddress>), m_alias_region_start(Null<KProcessAddress>),
+                  m_alias_region_end(Null<KProcessAddress>), m_stack_region_start(Null<KProcessAddress>), m_stack_region_end(Null<KProcessAddress>),
+                  m_kernel_map_region_start(Null<KProcessAddress>), m_kernel_map_region_end(Null<KProcessAddress>), m_alias_code_region_start(Null<KProcessAddress>),
+                  m_alias_code_region_end(Null<KProcessAddress>), m_code_region_start(Null<KProcessAddress>), m_code_region_end(Null<KProcessAddress>),
+                  m_max_heap_size(), m_mapped_physical_memory_size(), m_mapped_unsafe_physical_memory(), m_mapped_ipc_server_memory(), m_general_lock(),
+                  m_map_physical_memory_lock(), m_device_map_lock(), m_impl(util::ConstantInitialize), m_memory_block_manager(util::ConstantInitialize),
+                  m_allocate_option(), m_address_space_width(), m_is_kernel(), m_enable_aslr(), m_enable_device_address_space_merge(),
+                  m_memory_block_slab_manager(), m_block_info_manager(), m_resource_limit(), m_cached_physical_linear_region(), m_cached_physical_heap_region(),
+                  m_heap_fill_value(), m_ipc_fill_value(), m_stack_fill_value()
+            {
+                /* ... */
+            }
+
+            explicit KPageTableBase() { /* ... */ }
 
             NOINLINE Result InitializeForKernel(bool is_64_bit, void *table, KVirtualAddress start, KVirtualAddress end);
             NOINLINE Result InitializeForProcess(ams::svc::CreateProcessFlag as_type, bool enable_aslr, bool enable_device_address_space_merge, bool from_back, KMemoryManager::Pool pool, void *table, KProcessAddress start, KProcessAddress end, KProcessAddress code_address, size_t code_size, KMemoryBlockSlabManager *mem_block_slab_manager, KBlockInfoManager *block_info_manager, KResourceLimit *resource_limit);

libraries/libmesosphere/include/mesosphere/kern_k_page_table_manager.hpp

@@ -27,9 +27,10 @@ namespace ams::kern {
         private:
             using BaseHeap = KDynamicResourceManager<impl::PageTablePage, true>;
         private:
-            KPageTableSlabHeap *m_pt_heap{};
+            KPageTableSlabHeap *m_pt_heap;
         public:
-            constexpr KPageTableManager() = default;
+            constexpr explicit KPageTableManager(util::ConstantInitializeTag) : m_pt_heap() { /* ... */ }
+            explicit KPageTableManager() { /* ... */ }
 
             ALWAYS_INLINE void Initialize(KDynamicPageManager *page_allocator, KPageTableSlabHeap *pt_heap) {
                 m_pt_heap = pt_heap;

libraries/libmesosphere/include/mesosphere/kern_k_port.hpp

@@ -41,11 +41,13 @@ namespace ams::kern {
             State m_state;
             bool m_is_light;
         public:
-            constexpr KPort() : m_server(), m_client(), m_name(), m_state(State::Invalid), m_is_light() { /* ... */ }
+            explicit KPort() : m_state(State::Invalid), m_is_light() { /* ... */ }
 
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
             void Initialize(s32 max_sessions, bool is_light, uintptr_t name);
+            void Finalize() { /* ... */ }
+
             void OnClientClosed();
             void OnServerClosed();
 

libraries/libmesosphere/include/mesosphere/kern_k_priority_queue.hpp

@@ -235,7 +235,7 @@ namespace ams::kern {
             KPriorityQueueImpl m_suggested_queue;
         private:
             constexpr ALWAYS_INLINE void ClearAffinityBit(u64 &affinity, s32 core) {
-                affinity &= ~(u64(1ul) << core);
+                affinity &= ~(UINT64_C(1) << core);
             }
 
             constexpr ALWAYS_INLINE s32 GetNextCore(u64 &affinity) {

libraries/libmesosphere/include/mesosphere/kern_k_process.hpp

@@ -34,7 +34,7 @@
 
 namespace ams::kern {
 
-    class KProcess final : public KAutoObjectWithSlabHeapAndContainer<KProcess, KSynchronizationObject>, public KWorkerTask {
+    class KProcess final : public KAutoObjectWithSlabHeapAndContainer<KProcess, KWorkerTask> {
         MESOSPHERE_AUTOOBJECT_TRAITS(KProcess, KSynchronizationObject);
         public:
             enum State {
@@ -57,73 +57,73 @@ namespace ams::kern {
             using TLPTree = util::IntrusiveRedBlackTreeBaseTraits<KThreadLocalPage>::TreeType<KThreadLocalPage>;
             using TLPIterator = TLPTree::iterator;
         private:
-            KProcessPageTable           m_page_table{};
-            std::atomic<size_t>         m_used_kernel_memory_size{};
-            TLPTree                     m_fully_used_tlp_tree{};
-            TLPTree                     m_partially_used_tlp_tree{};
-            s32                         m_ideal_core_id{};
-            void                       *m_attached_object{};
-            KResourceLimit             *m_resource_limit{};
-            KVirtualAddress             m_system_resource_address{};
-            size_t                      m_system_resource_num_pages{};
-            size_t                      m_memory_release_hint{};
-            State                       m_state{};
-            KLightLock                  m_state_lock{};
-            KLightLock                  m_list_lock{};
-            KConditionVariable          m_cond_var{};
-            KAddressArbiter             m_address_arbiter{};
-            u64                         m_entropy[4]{};
-            bool                        m_is_signaled{};
-            bool                        m_is_initialized{};
-            bool                        m_is_application{};
-            char                        m_name[13]{};
-            std::atomic<u16>            m_num_running_threads{};
-            u32                         m_flags{};
-            KMemoryManager::Pool        m_memory_pool{};
-            s64                         m_schedule_count{};
-            KCapabilities               m_capabilities{};
-            ams::svc::ProgramId         m_program_id{};
-            u64                         m_process_id{};
-            s64                         m_creation_time{};
-            KProcessAddress             m_code_address{};
-            size_t                      m_code_size{};
-            size_t                      m_main_thread_stack_size{};
-            size_t                      m_max_process_memory{};
-            u32                         m_version{};
-            KHandleTable                m_handle_table{};
-            KProcessAddress             m_plr_address{};
-            void                       *m_plr_heap_address{};
-            KThread                    *m_exception_thread{};
-            ThreadList                  m_thread_list{};
-            SharedMemoryInfoList        m_shared_memory_list{};
-            IoRegionList                m_io_region_list{};
-            bool                        m_is_suspended{};
-            bool                        m_is_immortal{};
-            bool                        m_is_jit_debug{};
-            bool                        m_is_handle_table_initialized{};
-            ams::svc::DebugEvent        m_jit_debug_event_type{};
-            ams::svc::DebugException    m_jit_debug_exception_type{};
-            uintptr_t                   m_jit_debug_params[4]{};
-            u64                         m_jit_debug_thread_id{};
-            KWaitObject                 m_wait_object{};
-            KThread                    *m_running_threads[cpu::NumCores]{};
-            u64                         m_running_thread_idle_counts[cpu::NumCores]{};
-            KThread                    *m_pinned_threads[cpu::NumCores]{};
-            std::atomic<s64>            m_cpu_time{};
-            std::atomic<s64>            m_num_process_switches{};
-            std::atomic<s64>            m_num_thread_switches{};
-            std::atomic<s64>            m_num_fpu_switches{};
-            std::atomic<s64>            m_num_supervisor_calls{};
-            std::atomic<s64>            m_num_ipc_messages{};
-            std::atomic<s64>            m_num_ipc_replies{};
-            std::atomic<s64>            m_num_ipc_receives{};
-            KDynamicPageManager         m_dynamic_page_manager{};
-            KMemoryBlockSlabManager     m_memory_block_slab_manager{};
-            KBlockInfoManager           m_block_info_manager{};
-            KPageTableManager           m_page_table_manager{};
-            KMemoryBlockSlabHeap        m_memory_block_heap{};
-            KBlockInfoSlabHeap          m_block_info_heap{};
-            KPageTableSlabHeap          m_page_table_heap{};
+            KProcessPageTable           m_page_table;
+            util::Atomic<size_t>        m_used_kernel_memory_size;
+            TLPTree                     m_fully_used_tlp_tree;
+            TLPTree                     m_partially_used_tlp_tree;
+            s32                         m_ideal_core_id;
+            void                       *m_attached_object;
+            KResourceLimit             *m_resource_limit;
+            KVirtualAddress             m_system_resource_address;
+            size_t                      m_system_resource_num_pages;
+            size_t                      m_memory_release_hint;
+            State                       m_state;
+            KLightLock                  m_state_lock;
+            KLightLock                  m_list_lock;
+            KConditionVariable          m_cond_var;
+            KAddressArbiter             m_address_arbiter;
+            u64                         m_entropy[4];
+            bool                        m_is_signaled;
+            bool                        m_is_initialized;
+            bool                        m_is_application;
+            char                        m_name[13];
+            util::Atomic<u16>           m_num_running_threads;
+            u32                         m_flags;
+            KMemoryManager::Pool        m_memory_pool;
+            s64                         m_schedule_count;
+            KCapabilities               m_capabilities;
+            ams::svc::ProgramId         m_program_id;
+            u64                         m_process_id;
+            s64                         m_creation_time;
+            KProcessAddress             m_code_address;
+            size_t                      m_code_size;
+            size_t                      m_main_thread_stack_size;
+            size_t                      m_max_process_memory;
+            u32                         m_version;
+            KHandleTable                m_handle_table;
+            KProcessAddress             m_plr_address;
+            void                       *m_plr_heap_address;
+            KThread                    *m_exception_thread;
+            ThreadList                  m_thread_list;
+            SharedMemoryInfoList        m_shared_memory_list;
+            IoRegionList                m_io_region_list;
+            bool                        m_is_suspended;
+            bool                        m_is_immortal;
+            bool                        m_is_jit_debug;
+            bool                        m_is_handle_table_initialized;
+            ams::svc::DebugEvent        m_jit_debug_event_type;
+            ams::svc::DebugException    m_jit_debug_exception_type;
+            uintptr_t                   m_jit_debug_params[4];
+            u64                         m_jit_debug_thread_id;
+            KWaitObject                 m_wait_object;
+            KThread                    *m_running_threads[cpu::NumCores];
+            u64                         m_running_thread_idle_counts[cpu::NumCores];
+            KThread                    *m_pinned_threads[cpu::NumCores];
+            util::Atomic<s64>           m_cpu_time;
+            util::Atomic<s64>           m_num_process_switches;
+            util::Atomic<s64>           m_num_thread_switches;
+            util::Atomic<s64>           m_num_fpu_switches;
+            util::Atomic<s64>           m_num_supervisor_calls;
+            util::Atomic<s64>           m_num_ipc_messages;
+            util::Atomic<s64>           m_num_ipc_replies;
+            util::Atomic<s64>           m_num_ipc_receives;
+            KDynamicPageManager         m_dynamic_page_manager;
+            KMemoryBlockSlabManager     m_memory_block_slab_manager;
+            KBlockInfoManager           m_block_info_manager;
+            KPageTableManager           m_page_table_manager;
+            KMemoryBlockSlabHeap        m_memory_block_heap;
+            KBlockInfoSlabHeap          m_block_info_heap;
+            KPageTableSlabHeap          m_page_table_heap;
         private:
             Result Initialize(const ams::svc::CreateProcessParameter &params);
 
@@ -145,7 +145,7 @@ namespace ams::kern {
                 m_pinned_threads[core_id] = nullptr;
             }
         public:
-            KProcess() { /* ... */ }
+            explicit KProcess() : m_is_initialized(false) { /* ... */ }
 
             Result Initialize(const ams::svc::CreateProcessParameter &params, const KPageGroup &pg, const u32 *caps, s32 num_caps, KResourceLimit *res_limit, KMemoryManager::Pool pool, bool immortal);
             Result Initialize(const ams::svc::CreateProcessParameter &params, svc::KUserPointer<const u32 *> caps, s32 num_caps, KResourceLimit *res_limit, KMemoryManager::Pool pool);
@@ -284,8 +284,12 @@ namespace ams::kern {
 
             constexpr KProcessAddress GetProcessLocalRegionAddress() const { return m_plr_address; }
 
+            constexpr void *GetProcessLocalRegionHeapAddress() const { return m_plr_heap_address; }
+
+            KThread *GetExceptionThread() const { return m_exception_thread; }
+
             void AddCpuTime(s64 diff) { m_cpu_time += diff; }
-            s64 GetCpuTime() { return m_cpu_time; }
+            s64 GetCpuTime() { return m_cpu_time.Load(); }
 
             constexpr s64 GetScheduledCount() const { return m_schedule_count; }
             void IncrementScheduledCount() { ++m_schedule_count; }
@@ -380,13 +384,14 @@ namespace ams::kern {
             }
         public:
             /* Overridden parent functions. */
-            virtual bool IsInitialized() const override { return m_is_initialized; }
+            bool IsInitialized() const { return m_is_initialized; }
 
             static void PostDestroy(uintptr_t arg) { MESOSPHERE_UNUSED(arg); /* ... */ }
 
-            virtual void Finalize() override;
+            void Finalize();
 
-            virtual u64 GetId() const override final { return this->GetProcessId(); }
+            ALWAYS_INLINE u64 GetIdImpl() const { return this->GetProcessId(); }
+            ALWAYS_INLINE u64 GetId() const { return this->GetIdImpl(); }
 
             virtual bool IsSignaled() const override {
                 MESOSPHERE_ASSERT_THIS();
@@ -394,7 +399,7 @@ namespace ams::kern {
                 return m_is_signaled;
             }
 
-            virtual void DoWorkerTask() override;
+            void DoWorkerTaskImpl();
         private:
             void ChangeState(State new_state) {
                 if (m_state != new_state) {