support booting with more than 4G of memory#2361
support booting with more than 4G of memory#2361mazunki wants to merge 5 commits intoincludeos:mainfrom
Conversation
`Expects()` doesn't actually shut down the service, causing an infinite loop of strace errors
The multiboot specification explicitly states that `mem_upper` is not guaranteed to be the correct value for mem_upper. Additionally, placing the heap in-between `mem_lower` and `mem_upper` only really makes sense on 32-bit systems. By using memory mapped pages we can pick the largest consecutive region of memory available. Selecting 1_000_000 as the minimum address is a temporary hack since it's not immediately obvious how to get the correct address for virtual mappings that are still not initialized (such as LiveUpdate and SystemLog). The correct solution here would be to make these regions dynamic in size, initialize them earlier so the regions are already claimed, or re-initialize the heap. This is only necessary for systems that have less than 4G of memory. See-Also: https://www.gnu.org/software/grub/manual/multiboot/multiboot.html See-Also: https://wiki.osdev.org/Multiboot
This will permit repurposing an already assigned mapping, allowing multiboot to preemptively assigning "free memory" regions, resizing ranges by deleting and recreating them, or making temporary regions. Could also be useful for NUMA.
|
Note that this solution isn't perfect: LiveUpdate and other components should be made dynamic and cooperate somehow with selecting the heap's size. The preemptive support of unassigning memory map ranges aims to facilitate this transition. |
|
Integration tests are currently unable to run Pending introducing integration tests for differently sized |
MagnusS
left a comment
There was a problem hiding this comment.
Looks good, but some smaller comments inline :-) Thanks!
| @@ -0,0 +1,3 @@ | |||
| { | |||
| "mem": 16384 | |||
There was a problem hiding this comment.
This is a lot - could we make it smaller?
|
|
||
| void allow_executable() | ||
| { | ||
| // FIXME: this seems weird. is this only used for tests? |
There was a problem hiding this comment.
This is used for the executable part of the IncludeOS service binary
| m.flags = os::mem::Access::execute | os::mem::Access::read; | ||
|
|
||
| os::mem::map(m, "ELF .text"); | ||
| os::mem::map(m, "Executable"); |
There was a problem hiding this comment.
Perhaps the text could be "ELF executable"
| inline auto strace(Fn func, [[maybe_unused]]const char* name, Args&&... args) { | ||
| if (!kernel::state().allow_syscalls) { | ||
| fprintf(stderr, "Syscalls not allowed here. Unexpected call to %s - terminating\n", name); | ||
| os::halt(); |
There was a problem hiding this comment.
Instead of halt here it would better to fix the Expect-call so we can get stack traces when this happens after boot (and heap is available)
| } | ||
| uintptr_t end = heap_map.addr + heap_map.len - 1; | ||
|
|
||
| if (heap_map.addr < 0x1'000'000) { |
There was a problem hiding this comment.
Could this be solved by adding the LiveUpdate and Syslog ranges to the map statically? Should at least be commented here why there's a check
This PR adds support for booting IncludeOS with more than 4 GB of physical memory, updating the example unikernel's message to show these mappings. Before this PR, the max size of the heap would be ~2.2GiB.
The old example unikernel wasn't terribly useful: maybe we could extend the example unikernel to be informative of more aspects of the kernel's state after it has been booted?
It additionally adds preemptive supports to unassigning virtual mappings, and shuts down the unikernel if a strace condition has been hit.
See full commit messages for specific details.