nortti
/
sortix-mirror
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Mirror of most important branches of the main Sortix repository
574 Commits 2 Branches 0 Tags 24 MiB
C 62.3%
C++ 24.8%
Roff 8.8%
Assembly 1.5%
Shell 1.4%
Other 1.2%
Go to file
Jonas 'Sortie' Termansen 51e3de971c Multithreaded kernel and improvement of signal handling. 
Pardon the big ass-commit, this took months to develop and debug and the
refactoring got so far that a clean merge became impossible. The good news
is that this commit does quite a bit of cleaning up and generally improves
the kernel quality.

This makes the kernel fully pre-emptive and multithreaded. This was done
by rewriting the interrupt code, the scheduler, introducing new threading
primitives, and rewriting large parts of the kernel. During the past few
commits the kernel has had its device drivers thread secured; this commit
thread secures large parts of the core kernel. There still remains some
parts of the kernel that is _not_ thread secured, but this is not a problem
at this point. Each user-space thread has an associated kernel stack that
it uses when it goes into kernel mode. This stack is by default 8 KiB since
that value works for me and is also used by Linux. Strange things tends to
happen on x86 in case of a stack overflow - there is no ideal way to catch
such a situation right now.

The system call conventions were changed, too. The %edx register is now
used to provide the errno value of the call, instead of the kernel writing
it into a registered global variable. The system call code has also been
updated to better reflect the native calling conventions: not all registers
have to be preserved. This makes system calls faster and simplifies the
assembly. In the kernel, there is no longer the event.h header or the hacky
method of 'resuming system calls' that closely resembles cooperative
multitasking. If a system call wants to block, it should just block.

The signal handling was also improved significantly. At this point, signals
cannot interrupt kernel threads (but can always interrupt user-space threads
if enabled), which introduces some problems with how a SIGINT could
interrupt a blocking read, for instance. This commit introduces and uses a
number of new primitives such as kthread_lock_mutex_signal() that attempts
to get the lock but fails if a signal is pending. In this manner, the kernel
is safer as kernel threads cannot be shut down inconveniently, but in return
for complexity as blocking operations must check they if they should fail.

Process exiting has also been refactored significantly. The _exit(2) system
call sets the exit code and sends SIGKILL to all the threads in the process.
Once all the threads have cleaned themselves up and exited, a worker thread
calls the process's LastPrayer() method that unmaps memory, deletes the
address space, notifies the parent, etc. This provides a very robust way to
terminate processes as even half-constructed processes (during a failing fork
for instance) can be gracefully terminated.

I have introduced a number of kernel threads to help avoid threading problems
and simplify kernel design. For instance, there is now a functional generic
kernel worker thread that any kernel thread can schedule jobs for. Interrupt
handlers run with interrupts off (hence they cannot call kthread_ functions
as it may deadlock the system if another thread holds the lock) therefore
they cannot use the standard kernel worker threads. Instead, they use a
special purpose interrupt worker thread that works much like the generic one
expect that interrupt handlers can safely queue work with interrupts off.
Note that this also means that interrupt handlers cannot allocate memory or
print to the kernel log/screen as such mechanisms uses locks. I'll introduce
a lock free algorithm for such cases later on.

The boot process has also changed. The original kernel init thread in
kernel.cpp creates a new bootstrap thread and becomes the system idle thread.
Note that pid=0 now means the kernel, as there is no longer a system idle
process. The bootstrap thread launches all the kernel worker threads and then
creates a new process and loads /bin/init into it and then creates a thread
in pid=1, which starts the system. The bootstrap thread then quietly waits
for pid=1 to exit after which it shuts down/reboots/panics the system.

In general, the introduction of race conditions and dead locks have forced me
to revise a lot of the design and make sure it was thread secure. Since early
parts of the kernel was quite hacky, I had to refactor such code. So it seems
that the risk of dead locks forces me to write better code.

Note that a real preemptive multithreaded kernel simplifies the construction
of blocking system calls. My hope is that this will trigger a clean up of
the filesystem code that current is almost beyond repair.

Almost all of the kernel was modified during this refactoring. To the extent
possible, these changes have been backported to older non-multithreaded
kernel, but many changes were tightly coupled and went into this commit.

Of interest is the implementation of the kthread_ api based on the design
of pthreads; this library allows easy synchronization mechanisms and
includes C++-style scoped locks. This commit also introduces new worker
threads and tested mechanisms for interrupt handlers to schedule work in a
kernel worker thread.

A lot of code have been rewritten from scratch and has become a lot more
stable and correct.

Share and enjoy!
 		2012-09-08 18:45:41 +02:00
bench Added copyright headers to benchmarks, games and utilities. 2012-03-11 15:57:13 +01:00
debsrc Refactored all the sortix headers into a include directory. 2012-03-22 00:52:29 +01:00
games Added {,p}{read,write}{all,least}(3). 2012-03-24 15:34:30 +01:00
initrd The Sortix programs are now compiled into initrd/. 2011-08-28 12:42:06 +02:00
isosrc/boot/grub Initial version of Sortix. 2011-08-05 14:25:00 +02:00
libmaxsi Multithreaded kernel and improvement of signal handling. 2012-09-08 18:45:41 +02:00
mkinitrd Implemented a new initrd format with better inode support. 2012-07-02 17:40:52 +02:00
mxmpp Removed references to mksound from mxmpp. 2011-12-04 21:33:23 +01:00
sortix Multithreaded kernel and improvement of signal handling. 2012-09-08 18:45:41 +02:00
utils pager(1) now detects the terminal resolution. 2012-07-31 16:48:56 +02:00
.gitignore Initial signal support. Please squash improvements into this commit. 2011-11-23 00:19:09 +01:00
Makefile Implemented a new initrd format with better inode support. 2012-07-02 17:40:52 +02:00
README Refactored all the sortix headers into a include directory. 2012-03-22 00:52:29 +01:00
crosscompilemakefile.mak Refactored all the sortix headers into a include directory. 2012-03-22 00:52:29 +01:00
gpl.html Refactored all the sortix headers into a include directory. 2012-03-22 00:52:29 +01:00
lgpl.html Refactored all the sortix headers into a include directory. 2012-03-22 00:52:29 +01:00

README 

The Sortix Operating System
===========================
Sortix is a hobby operating system. It was originally created as a tool to learn
more about kernel and operating system design and implementation. Today is
transforming into a real operating system. The standard library and kernel is
rich enough that some third party software can and has been ported to Sortix.
However, the system remains quite limited as of this writing. Many features are
missing such as proper filesystem support, bitmap graphics, and networking.
Proper filesystem support is currently being added.

The system aims to be an Unix-clone and is heavily based on POSIX. However, I've
drawn much inspiration from systems such as Plan 9, GNU/Hurd and MINIX. Indeed,
I plan to construct a micro-kernel with user-space filesystems, per-process
namespaces, replacing many system calls with filesystem nodes, and other
exciting features. This design will make it safe to let normal users perform
operations such as mounting and create their own "sub-operating-system"
environment where they are the root.

System Requirements
-------------------
Sortix has very low system requirements. It also works well under virtual
machines such as VirtualBox and Qemu.

* A 32-bit x86 or 64-bit x86_64 CPU.
* A dozen megabytes of RAM.
* A harddisk or cdrom drive or support for booting from USB.
* A multiboot compliant bootloader if booting from harddisk.
* A Parallel ATA harddisk, if you wish to access it from Sortix. SATA is not
  supported yet.

Features
--------
The current development version of Sortix offers a traditional multi-process
protected environment with round-robin scheduling. A quick and dirty shell is
able to execute programs in foreground or background mode, handle IO redirection
and piping the standard output of a process into the standard input of another.
A real shell will be added as the system matures and I get around to finish the
work in progress shell.

A number of standard utilities are present such as cat, head, tail, clear, cp,
column, kill, ls, rm, pwd, uname, echo, and uptime. There is even a number of
non-standard utilities such as calc, help, init, kernelinfo, memstat, and pager.
This collection of utilities will continue to grow as it matures and third party
software is ported. I've currently had some luck partially porting binutils,
ocaml, and gzip, but the system isn't fully ready for such software yet.

A number of small games is present and uses the VGA textmode to render ASCII
graphics. Notably you can play two-player Pong, or single-player Snake, or the
nice and Turing-complete Conway's Game of Life. These are probably the main
attraction of the system for non-technical people.

The Sortix kernel has very basic filesystem support. The root filesystem / is
simply a single-directory RAM filesystem. The init ramdisk is mounted read-only
on /bin and various devices are accessible through the /dev filesystem. Work is
underway to create an ext2 filesystem server, but it won't be of much use until
the kernel virtual filesystem is completed in the 0.7dev development cycle.

There currently is no concept of users in the system (only the root user exists
I decided to delay making a multi-user system until the base system is in place.
Note that there is only a single terminal - even though the system is a
multi-process system, there is only a single /dev/vga and there is no framework
in place for sharing it.

Improvements in Sortix 0.6
--------------------------
The 0.6 release improves greatly upon Sortix 0.5.

* Implemented an ATA driver.
* User-space programs can now use dynamic allocation.
* Standard Error support.
* Added a new terminal driver allowing standard input from keyboard.
* Shell understands pipes, IO redirection and background processes.
* COM ports are available as /dev/comN.
* ls now formats files nicely in columns.
* Added a paging program for displaying text-files page by page.
* Support for call-traces in the kernel.
* Fixes to avoid conflicts with GRUB.
* Countless bug-fixes and small improvements.
* Massive enhancements to the standard library.

The 0.6 release drops support for the javascript virtual machine used to power
amazing things such as jslinux. This is largely because Sortix has outgrown it
and needs a less limited environment, in particular the lack of a VGA and that
the keyboard events are truncated to that of serial connections is annoying and
make it possible to keep the games compatible without severe performance
penalties.

Known bugs
----------
There is currently a lot of bugs and small quirks in Sortix. They are caused by
the system being young and incomplete. They will gradually be fixed as the base
system improves. They are tolerated in the releases because they are not
critical and improvement situation from the last release.

Job control and Unix signals is not fully or correctly implemented. This means
that sequences such as Ctrl-C (SIGINT) not always works correctly. This will
be implemented soon enough (depends partially on VFS; see above). Whenever you
hit Ctrl-C the kernel attempts to terminate the currently running process (even
if it isn't the foreground process). In addition, the kernel can only send
signals to running processes. This means that that programs that read a line of
input at a time from the terminal will not be terminated until you press enter.
This will be fixed when proper support for kernel threads is added.

The shell only looks at whitespace when parsing input lines. This means that
operators such as & ; and | must be surrounded by whitespace on both sides.
Support for quotes and escape characters is also not implemented yet. The shell
does not accept very long lines yet.

Transferring large binary streams over pipes may cause instability in the kernel
due to a bug that wasn't fixed in time for the 0.6 release. In particular, the
command $ cat /bin/cat | cat seems to trigger the issue occasionally. This bug
will likely go away when I rewrite the kernel IO framework.

Some programs change the format of the terminal standard input delivered to
them. For instance, the pong game needs to capture every keyboard event. However
running such programs at the same time as programs that need proper line-
buffered input (such as the shell) causes conflicts and neither program gets
what it needs causing programs to exit or crash. Note that if the shell fails
to read in its expected format, it exits with an error code. When it does this
the init process does too, and the kernel panics. This issue is caused by the
design of the terminal system and can't be fixed until it is rewritten.

The filesystem is currently hacked together in the kernel. When things go wrong
you may experience the utilities returning the wrong error codes. This is
harmless but confusing. All the current kernel filesystem code will be replaced
a user-space filesystem framework and will be present in the next release.

Improvements scheduled for Sortix 0.7
-------------------------------------
* Proper kernel threads.
* Kernel virtual filesystem.
* User-space filesystem framework.
* User-space ext2 filesystem driver.
* Refactor and extend the standard library.
* Mount-points.
* Environmental variables.
* Improved terminal framework.
* New and improved shell.
* Copy-on-write in fork(2) and mmap(2).

Technical details
----------------
The system is mostly coded in C++, but also contains a few files in C. However,
the user-land experiences a normal C programming interface as per POSIX.
Executable files natively uses the ELF format used on GNU/Linux and other
systems. There is no shared library support yet, but it'll be possible when I
get around to implement copy-on-write memory, mmap(2) and swapping to disk.

Building
--------
To build the Sortix source code you need to install a few dependencies. First of
all you need the GNU Compiler Collection (C and C++), GNU Make, and GNU
Binutils. You then need to build and install the included macro preprocessor
(mxmpp) somewhere in your PATH such as /usr/bin. If you wish to build the 32-bit
version of Sortix, you need the Netwide Assembler (nasm) as parts of it hasn't
been ported to the GNU assembler yet. You need a GNU/Linux build system to build
Sortix, although, it wouldn't be difficult to port the build system to other
platforms. You can then build the Sortix kernel and user-space utilities by
running make in the Sortix root source directory. By default it will build to
your CPU architecture (64-bit on 64-bit systems, 32-bit otherwise). Use
CPU=x86 or CPU=x64 as arguments to make to control which target is built.

To build a bootable ISO you need GNU GRUB 2, as that is used by "make iso" to
generate the iso. In turn, GNU GRUB relies on xorriso to create the iso file.

You can burn the ISO to a cdrom or dvd, or even dd(1) it onto a USB memory stick
and boot from if it your BIOS supports it. You can also provide it to a virtual
machine. Alternatively you can install the kernel binary and initrd in your
/boot directory and configure GRUB to boot Sortix.

Links
-----
You can visit the official website at http://www.maxsi.org/software/sortix/ for
more information and news. You can also download the newest release and cutting
edge nightly builds.

You can retrieve the current git master from our gitorious project page from
https://gitorious.org/sortix/.

License
-------

Copyright(C) Jonas 'Sortie' Termansen <sortie@maxsi.org> and contributors 2011,
2012.

The Sortix kernel, the filesystem servers, the initrd tools, the utilities, the
games, and the benchmark programs are licensed under the GNU General Public
License, either version 3 or (at your option) any later version.

The libmaxsi standard library is licensed under the GNU Lesser General Public
License, either version 3 or (at your option) any later version.

Any experimental repositories and branches on Gitorious related to Sortix but
which contains no copyright statements are also released under the GNU General
Public License, either version 3 or (at your option) any later version. These
things are so experimental that I didn't add copyright statements yet.

Sortix is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the gpl.html and lgpl.html files for more information.