- KIP's pointer to UTCB seems to work with existing l4lib ipc functions.
- Works up to clone()
- In clone we mmap() the same UTCB on each new thread - excessive.
- Generally during page fault handling, cloned threads may fault on the same page
multiple times even though a single handling would be enough for all of them.
Need to detect and handle this.
Added setting of utcb address to l4_thread_control.
This is going to be moved to exchange_registers() since we need to pass
both the utcb physical and virtual address and exregs fits such context
modification better than thread_control.
- Now libl4 has no references to utcb page or shmat etc.
- Pager does not deal with special case utcb page allocation.
It instead allocates a shared page from shm memory pool.
- All tasks working to original standard.
Next:
- Add per-thread utcb allocation from the kernel
- Add larger register file for standard ipc
- Add long ipc (up to 1Kb)
Previously a so-called utcb shared page was used for transfering
data between posix services. This was a special shmat/get/dt case
allocating from its own virtual pool. Now the term utcb is renamed
as a shared page and integrated with the shm* handling routines.
Generic l4 threads will use long-ipc and not this method. Posix
services will continue to communicate on a shared page for now.
modified: tasks/libl4/include/l4lib/ipcdefs.h
modified: tasks/libl4/src/init.c
new file: tasks/libposix/init.c
modified: tasks/mm0/include/shm.h
modified: tasks/mm0/include/task.h
deleted: tasks/mm0/include/utcb.h
modified: tasks/mm0/main.c
modified: tasks/mm0/src/boot.c
modified: tasks/mm0/src/clone.c
modified: tasks/mm0/src/execve.c
modified: tasks/mm0/src/exit.c
modified: tasks/mm0/src/init.c
modified: tasks/mm0/src/shm.c
modified: tasks/mm0/src/task.c
deleted: tasks/mm0/src/utcb.c
deleted: tools/l4-qemu
- Directory creation, file read/write is OK.
- Cannot reuse old task's fds. They are not recycled for some reason.
- Problems with fork/clone/exit. They fail for a reason.
Increased inode block pointers to 40. The current maximum allowed (and checked).
Updates to file size after every file write ensures subsequent writes can
correctly operate using updated file size information (i.e. not try to add
more pages that are already present). We cannot do this inside write() because
directory writes rely on byte-granularity updates on file buffers, whereas
file updates are by page-granularity (currently).
It turned out we used one version of kmalloc for malloc() and another for kfree()!
Now fixed.
Added parent-child relationship to tasks. Need to polish handling CLONE_PARENT and THREAD.
l4_unmap now returns -1 if given range was only partially unmapped.
do_munmap() now only unmaps address ranges that have correspondence in
the unmapped vmas. Trying to unmap regions with no correspondent vmas
causes problems in corner cases, e.g. mm0 that tries to mmap its own
address space during initialisation would unmap its whole address space
and fail to execute.
When sys_munmap() splits a vma, the new vma had no copy of the objects
in the original vma. Now we fixed that using a vma_copy_links() function
which can also be used as part of fork().
Still testing sys_munmap(). It now correctly spots and unmaps the overlapping vma.
The issue now is that if a split occurs, we forgot to add same objects to new vma.
do_munmap currently shrinks, splits, destroys vmas and unmaps the given
virtual address range from the task. Unmapped pages may go completely unused
but page reclamation will be done in another part of the pager rather than
directly on the munmap instance.
- Fixed an important bug with shadow object handling.
When a shadow is dropped, if there are references left
to it, both the object in front and dropped object becomes
a shadow of the original object underneath. We had thought
of this case but had not increase the shadow count.
- Added a test mechanism that tests the number of objects,
vmfiles, shadows etc. by first counting them and trying to
reach the same number by other means, i.e. per-object-shadow counts.
It discovered a plethora of bugs.
- Added new set of functions to register objects, files and tasks
globally with the pager, these functions introduce a refcount as
well as adding structures to linked lists.
- fork/exit now seems to work stably i.e. no negative shadow counts etc.
- Added cleaner allocation of shm addresses by moving the allocation to do_mmap().
- Added deletion routine for all objects: shadow, vm_file of type vfs_file, shm_file, etc.
- Need to make sure objects get deleted properly after exit().
- Currently we allow a single, unique virtual address for each shm segment.
- Updated sleeping paths such that a task is atomically put into
a runqueue and made RUNNABLE, or removed from a runqueue and made SLEEPING.
- Modified vma dropping sources to handle both copy_on_write() and exit() cases
in a common function.
- Added the first infrastructure to have a pager to suspend a task and wait for
suspend completion from the scheduler.
Now all system calls can simply return their final values and they
will be sent to client parties from a single location. Should have had this
simple cleanup a long time ago.
fs0 used to receive open() requests and notify pager about them via a syscall ipc.
This caused deadlocks because normally request flow is mm0 -> fs0 on all other calls.
The solution was to have mm0 ask and validate file descriptors from fs0 on the first
request instance that involved that file descriptor. By this method we delay the
validation of the fd until its first use, and avoid deadlock. It also fits well with
the lazy request handling design philosophy.
- Fixed do_mmap() so that it returns mapped address, and various bugs.
- A child seems to fork with new setup, but with incorrect return value.
Need to use and test exregs() for fork + clone.
- Shmat searches an unmapped area if input arg is invalid, do_mmap()
should do this.
- Added mutex_trylock()
- Implemented most of exchange_registers()
- thread_control() now needs a lock for operations that can modify thread context.
- thread_start() does not initialise scheduler flags, now done in thread_create.
TODO:
- Fork/clone'ed threads should retain their context in tcb, not syscall stack.
- exchange_registers() calls in userspace need cleaning up.
For clone, file descriptor and vm area structures need to be
separate from the tcb and reached via a pointer so that they
can be shared among multiple tcbs.
- Added automatic utcb map/prefaulting of forked tasks for fs0
so that it does not need to explicitly request those tasks from mm0.
Eliminating fs0 requests to mm0 reduce deadlock possibilities.
- Replaced kmalloc with a public malloc implementation because of a bug in kmalloc.
- Fixed a kfree bug. default_release_pages was trying to free page_array pages.
- Adding prefaulting of fs0 to avoid page fault deadlocks.
- Fixed a bug that a vmo page_cache equivalence would simply drop a link to
an original vmo, even if the vmo could have more pages outside the page cache,
or if the vmo was not a shadow vmo.
- Fixed a bug with page allocator where recursion would corrupt global variables.
- Now going to fix or re-write a simpler page allocator that works.
Added a list of links for vm objects so they can follow
the links that point at them.
More succinct handling of the case where a vm object
is dropped. Now depending on the object's number of link
references and shadow references, upon a drop it could
either be merged, deleted or kept.
Added opener reference count for vm files. Now files
have opener count, objects have shadow and link count.
Link count is also meaningful for how many tasks have
mmap'ed that object.
Bahadir Balban