What is needed to create an operating system. Let's write a kernel! We create the simplest working operating system kernel. Windows maintenance, updates

What you need to know to write an operating system

Creating an operating system is one of the most difficult tasks in programming because it requires extensive and complex knowledge about how a computer works. Which ones? Let's find out below.

What is OS

Operating system (OS) is software, which works with computer hardware and its resources and is a bridge between the hardware and software of the computer.

First generation computers did not have operating systems. Programs on the first computers included code for the direct operation of the system, communication with peripheral devices and calculations for which this program was written. Because of this situation, even programs that were simple in logic were difficult to implement in software.

As computers became more diverse and complex, writing programs that worked as both an OS and an application became simply inconvenient. Therefore, to make programs easier to write, computer owners began to develop software. This is how operating systems appeared.

The OS provides everything necessary for user programs to run. Their appearance meant that programs no longer needed to control the entire volume of work on the computer (this is an excellent example of encapsulation). Now programs needed to work specifically with the operating system, and the system itself took care of resources and working with peripherals (keyboard, printer).

Brief history of operating systems

C language

As mentioned above, there are several high-level programming languages for writing an OS. However, the most popular of them is C.

You can start learning this language from here. This resource will introduce you to basic concepts and prepares you for more complex tasks.

“Learn C the Hard Way” is the title of another book. In addition to the usual theory, it contains many practical solutions. This tutorial will cover all aspects of the language.

Or you can choose one of these books:

"The C Programming Language" by Kernighan and Ritchie;
"C Programming Absolute Beginner's Guide" by Parry and Miller.

OS development

After mastering everything you need to know regarding computer science, assembly language and C, you should read at least one or two books about actual OS development. Here are some resources for this:

"Linux From Scratch". The operating room assembly process is discussed here. Linux systems(the textbook has been translated into many languages, including Russian). Here, as in other textbooks, you will be provided with all the necessary basic knowledge. Relying on them, you can try your hand at creating an OS. To make the software part of the OS more professional, there are additions to the textbook: “

The book "The Operating System from 0 to 1" is published on GitHub and has over 2,000 stars and 100 forks. As the title suggests, after reading it, you can create your own operating system - and, perhaps, few things in the world of programmers could be cooler.

With this book you will learn the following:

You will learn how to create an operating system based on hardware technical documentation. This is how it works in the real world, you won't be able to use Google for quick answers.
You will understand how computer components interact with each other, from software to hardware.
Learn to write code yourself. Blindly copying code is not learning, you will actually learn how to solve problems. By the way, blind copying is also dangerous.
Master the familiar tools for low-level development.
Get familiar with assembly language.
Find out what programs consist of and how operating system launches them. We gave a short overview of this topic for the curious in .
You will figure out how to debug a program directly on hardware with GDB and QEMU.

The C programming language. You can quickly master it by following.
Basic knowledge of Linux. Just study on our website.
Basic knowledge in physics: atoms, electrons, protons, neutrons, voltage.

I’ll say right away, don’t close the article with the thoughts “Damn, another Popov.” He just has a polished Ubuntu, while I have everything from scratch, including the kernel and applications. So, continuation under the cut.

OS group: Here.
First I'll give you one screenshot.

There are no more of them, and now let’s talk in more detail about why I am writing it.

It was a warm April evening, Thursday. Since childhood, I dreamed of writing an OS, when I suddenly thought: “Now I know the advantages and asm, why not make my dream come true?” I googled sites on this topic and found an article from Habr: “How to start and not quit writing an OS.” Thanks to its author for the link to the OSDev Wiki below. I went there and started working. There was all the data about the minimal OS in one article. I started building cross-gcc and binutils, and then rewrote everything from there. You should have seen my joy when I saw the inscription “Hello, kernel World!” I jumped right out of my chair and realized that I would not give up. I wrote "console" (in quotes; I didn't have access to a keyboard), but then decided to write a window system. In the end it worked, but I did not have access to the keyboard. And then I decided to come up with a name based on the X Window System. I googled Y Window System - it exists. As a result, I named Z Window System 0.1, included in OS365 pre-alpha 0.1. And yes, no one saw her except myself. Then I figured out how to implement keyboard support. Screenshot of the very first version, when there was nothing yet, not even a window system:

The text cursor didn't even move, as you can see. Then I wrote a couple simple applications based on Z. And here is the release 1.0.0 alpha. There were a lot of things there, even system menus. A file manager and the calculator just didn't work.

I was directly terrorized by a friend who cares only about beauty (Mitrofan, sorry). He said: “Wash down the VBE mode 1024*768*32, wash it down, wash it down! Well, let’s drink it up!” Well, I was already tired of listening to him and still cut him down. About the implementation below.

I made everything with my bootloader, namely GRUB. With its help, you can set the graphical mode without complications by adding a few magic lines to the Multiboot header.

Set ALIGN, 1<<0 .set MEMINFO, 1<<1 .set GRAPH, 1<<2 .set FLAGS, ALIGN | MEMINFO | GRAPH .set MAGIC, 0x1BADB002 .set CHECKSUM, -(MAGIC + FLAGS) .align 4 .long MAGIC .long FLAGS .long CHECKSUM .long 0, 0, 0, 0, 0 .long 0 # 0 = set graphics mode .long 1024, 768, 32 # Width, height, depth
And then from the Multiboot information structure I take the framebuffer address and screen resolution and write pixels there. VESA did everything very confusingly - RGB colors must be entered in reverse order (not R G B, but B G R). For several days I didn’t understand why the pixels weren’t displayed!? In the end, I realized that I forgot to change the values of 16 color constants from 0...15 to their RGB equivalents. As a result, I released it and at the same time cut down the gradient background. Then I made a console, 2 applications and released 1.2. Oh yes, I almost forgot - you can download the OS at

Reading Habr over the past two years, I have seen only a few attempts to develop an OS (specifically: from users and (postponed indefinitely) and (not abandoned, but for now it looks more like a description of the operation of the protected mode of x86-compatible processors, which is undoubtedly also you need to know to write an OS for x86); and a description of the finished system from (though not from scratch, although there is nothing wrong with that, maybe even the opposite)). For some reason, I think that almost all system (and some application) programmers have at least once thought about writing their own operating system. In this connection, 3 OS from the large community of this resource seems a ridiculous number. Apparently, the majority of those who think about their own OS don’t go anywhere beyond the idea, a small part stops after writing the bootloader, a few write pieces of the kernel, and only the hopelessly stubborn create something vaguely reminiscent of an OS (if compared with something like Windows/Linux) . There are many reasons for this, but the main one in my opinion is that people give up development (some before even starting) due to the small number of descriptions of the process of writing and debugging an OS, which is quite different from what happens during development application software.

With this short note I would like to show that if you start correctly, then there is nothing particularly difficult in developing your own OS. Below the cut is a short and fairly general guide to writing an OS from scratch.

How No need start off

Please do not take the following text as explicit criticism of anyone's articles or guides on writing an OS. It’s just that too often in such articles under loud headlines the emphasis is placed on the implementation of some minimal workpiece, and it is presented as a prototype of the kernel. In fact, you should think about the structure of the kernel and the interaction of parts of the OS as a whole, and consider that prototype as a standard “Hello, World!” application in the world of application software. As a slight justification for these comments, it should be said that below there is a subsection “Hello, World!”, which in this case is given exactly as much attention as necessary, and no more.

There is no need to write a bootloader. Smart people came up with the Multiboot Specification, implemented it and described in detail what it is and how to use it. I don’t want to repeat myself, I’ll just say that it works, it makes life easier, and it should be used. By the way, it is better to read the specification in full, it is small and even contains examples.

There is no need to write the OS entirely in assembler. This is not so bad, rather the opposite - fast and small programs will always be held in high esteem. It’s just that since this language requires significantly more effort to develop, using assembler will only lead to a decrease in enthusiasm and, as a result, to throwing the OS sources into a long box.

There is no need to load a custom font into video memory and display anything in Russian. This makes no sense. It is much easier and more versatile to use English, and leave changing the font for later, loading it from the hard drive through the file system driver (at the same time there will be an additional incentive to do more than just start).

Preparation

To begin with, as always, you should familiarize yourself with the general theory in order to have some idea about the upcoming scope of work. Good sources on the issue under consideration are the books of E. Tanenbaum, which have already been mentioned in other articles about writing an OS on Habré. There are also articles describing existing systems, and there are various guides/mailings/articles/examples/sites with an emphasis on OS development, links to some of which are given at the end of the article.

After the initial educational program, you need to decide on the main questions:

target architecture - x86 (real/protected/long mode), PowerPC, ARM, ...
kernel/OS architecture - monolith, modular monolith, microkernel, exokernel, various hybrids
language and its compiler - C, C++, ...
Kernel file format - elf, a.out, coff, binary, ...
development environment (yes, this also plays an important role) - IDE, vim, emacs, ...

Next, you should deepen your knowledge according to the chosen one and in the following areas:

video memory and working with it - output as proof of operation is necessary from the very beginning
HAL (Hardware Abstraction layer) - even if there is support for several hardware architectures and there are no plans to properly separate the lowest-level parts of the kernel from the implementation of such abstract things as processes, semaphores, and so on, it will not be superfluous
memory management - physical and virtual
execution management - processes and threads, their planning
device management - drivers
virtual file systems - to provide a single interface to the contents of various file systems
API (Application Programming Interface) - how exactly applications will access the kernel
IPC (Interprocess Communication) - sooner or later processes will have to communicate

Tools

Taking into account the chosen language and development tools, you should select a set of utilities and their settings that in the future will, by writing scripts, make it as easy and quick as possible to build, prepare an image and launch a virtual machine with a project. Let's take a closer look at each of these points:

any standard tools are suitable for assembly, such as make, cmake,... Here scripts for the linker and (specially written) utilities for adding a Multiboot header, checksums, or for any other purposes can be used.
By preparing an image we mean mounting it and copying files. Accordingly, the image file format must be selected so that it is supported by both the mounting/copying utility and the virtual machine. Naturally, no one forbids performing the actions from this point either as the final part of the assembly, or as preparation for launching the emulator. It all depends on the specific means and the chosen options for their use.
starting a virtual machine is not difficult, but you need to remember to first unmount the image (unmounting at this point, since there is no real point in this operation before starting the virtual machine). It would also be useful to have a script to launch the emulator in debug mode (if available).

If all the previous steps have been completed, you should write a minimal program that will boot as a kernel and print something to the screen. If inconveniences or shortcomings of the chosen means are discovered, it is necessary to eliminate them (the shortcomings), or, in the worst case, take them for granted.

At this step, you need to check as many features as possible of the development tools that you plan to use in the future. For example, loading modules into GRUB or using a physical disk/partition/flash drive in a virtual machine instead of an image.

After this stage is successful, the real development begins.

Providing run-time support

Since it is proposed to write in high-level languages, care should be taken to provide support for some of the language features that are usually implemented by the authors of the compiler package. For example, for C++, these include:

function for dynamically allocating a block of data on the stack
working with heap
data block copy function (memcpy)
function is the entry point to the program
calls to constructors and destructors of global objects
a number of functions for working with exceptions
stub for unrealized pure virtual functions

When writing "Hello, World!" The absence of these functions may not make itself felt, but as code is added, the linker will begin to complain about unmet dependencies.

Naturally, we should also mention the standard library. A full implementation is not necessary, but a core subset of the functionality is worth implementing. Then writing code will be much more familiar and faster.

Debugging

Don't look at what is said about debugging towards the end of the article. In fact, this is a very serious and difficult issue in OS development, since conventional tools are not applicable here (with some exceptions).

We can recommend the following:

goes without saying, debug output
assert with immediate exit to the “debugger” (see next paragraph)
some semblance of a console debugger
check if the emulator allows you to connect a debugger, symbol tables or something else

Without a debugger built into the kernel, searching for errors has a very real chance of turning into a nightmare. So there is simply no escape from writing it at some stage of development. And since this is inevitable, it is better to start writing it in advance and thus greatly facilitate the development and save quite a lot of time. It is important to be able to implement a debugger in a kernel-independent manner so that debugging has minimal impact on the normal operation of the system. Here are a few types of commands that may be useful:

part of standard debugging operations: breakpoints, call stack, printing values, printing dump, ...
commands to display various useful information, such as the scheduler execution queue or various statistics (it is not as useless as it might seem at first)
commands for checking the consistency of the state of various structures: lists of free/used memory, heap or message queue

Development

Next, it is necessary to write and debug the main elements of the OS, which at the moment should ensure its stable operation, and in the future - easy extensibility and flexibility. In addition to memory/process/(whatever) managers, the interface of drivers and file systems is very important. Their design should be approached with special care, taking into account the variety of device types/FS. Over time, of course, they can be changed, but this is a very painful and error-prone process (and debugging the kernel is not an easy task), so just remember - think about these interfaces at least ten times before you start implementing them.

Similar SDK

As the project develops, new drivers and programs should be added to it. Most likely, already on the second driver (possibly of a certain type)/program some common features will be noticeable (directory structure, assembly control files, specification of dependencies between modules, repeated code in main or in system request handlers (for example, if the drivers themselves check their compatibility with the device )). If this is the case, then this is a sign of the need to develop templates for various types of programs for your OS.

There is no need for documentation describing the process of writing this or that type of program. But it’s worth making a blank from standard elements. This will not only make it easier to add programs (which can be done by copying existing programs and then changing them, but this will take more time), but will also make them easier to update when interfaces, formats, or anything else changes. It is clear that such changes ideally should not occur, but since OS development is an atypical thing, there are quite a lot of places for potentially wrong decisions. But the understanding of the fallacy of the decisions made, as always, will come some time after their implementation.

Further actions

In short, then: read about operating systems (and, first of all, about their design), develop your system (pace is not really important, the main thing is not to stop completely and return to the project from time to time with new strength and ideas) and It’s natural to correct errors in it (to find which you sometimes need to start the system and “play” with it). Over time, the development process will become easier and easier, errors will occur less frequently, and you will be included in the list of “hopelessly stubborn”, those few who, despite the certain absurdity of the idea of developing their own OS, still did it.

Author Vulf gamer asked a question in the section Other languages and technologies

How to create your own OS? and got the best answer

Answer from Alexander Bagrov[guru]
The idea is laudable.
First of all, you need to know the command system of the machine for which you intend to write an OS.
The command system is directly reflected in assembly language.
Therefore, first of all, you need to come up with your own assembly language and write a program for it (assembler) that translates alphanumeric symbols into machine language
code.
If you are interested, you can see what requirements a new (ideal) OS should have.
Some such features are listed here:
.ru/D_OS/OS-PolyM.html#IdealOS
It is necessary to study materials from microprocessor developer sites. For example, Intel and AMD.
You might find the video lecture course on OS useful, which is presented here:
.ru/D_OS/OS_General.html
PS: Don't listen to the pessimists. Based on the ideology of a rooster chasing a chicken:
“I won’t catch up, at least I’ll warm up.”
Source: Website "Use your PC correctly!"

Answer from 2 answers[guru]

Hello! Here is a selection of topics with answers to your question: How to create your own OS?

Answer from Vadim Khprlamov[newbie]
Of course) There are only gates sitting here) Ask Microsoft)

Answer from Irina Starodubtseva[newbie]
take all the operating systems and cram them into one

Answer from Alexander Tuntsov[guru]
Get acquainted with the Linux OS, learn programming and go

Answer from ~In Flames~[guru]
Teach programming at the highest level, gather a whole crowd of the same computer geniuses, and then you can do it.

Answer from Rasul Magomedov[guru]
Start by creating a fun wallpaper

Answer from Captain Google[guru]
Don’t listen to “10 years to learn the basics”; Torvalds wrote the first version of Linux at the age of 22, and he got a computer at 12. As you understand, he not only studied the basics.
Start by studying what already exists - on the one hand, “Modern Operating Systems” by Tannenbaum, on the other hand, build Linux From Scratch, on the third, learn Assembly, C, C++. Everything about everything can be done in 3-4 years. After that, you can start developing your system... if you still want.

Answer from Yoanya Semenov[guru]
do you know what Gates did? try the same, they say it turns out profitable..
when his parents punished him, he, having nothing better to do, began jumping butt on the keyboard, then sold it, calling what he got "windows"
ps and if it’s real, first write “Hello World” in C++ and you’ll immediately understand that the idea is paranoid

Answer from Kostafey[guru]
What for? What fundamentally dissatisfies the existing ones? Is there really not one that at least partially satisfies your OS requirements? Maybe it's better to join the development team? There will be 100500 times more sense.
And then, you will abandon this idea for another 0.(0)1% of its implementation.

Answer from Evgeniy Lomega[guru]
E. Tanenbaum "Operating systems: development and implementation"
Good luck
PS Unfortunately, you are unlikely to succeed as Bill Gates did. His mother is a cool banker, is yours?

Answer from Krab Bark[guru]
You can write a simple OS yourself, but it will never be able to compete with an OS like Windows, MAC OS or Linux, on which hundreds or thousands of programmers have worked for at least ten years. Besides, the OS is only the foundation. It is necessary that hardware developers write their own drivers for this OS, application program developers write editors, players, browsers, games for it, the devil in a mortar... And without this, the OS will remain a useless foundation for a house that no one will build.

Answer from Vadim Stakhanov[active]
It would be better if I went to study as a philologist. And then he would shout “Free cash register!”

Answer from =Serge=[guru]
Hooray! Finally, question 58 on the site about creating “your own” OS))
Here are the questions about “writing your own OS” - there are only 34 of them))
Read...92 questions*10 answers = approximately 920 answers))
At the same time, you will probably understand what “not boring wallpaper” means).

Answer from Irreproducible[guru]
another Denis Popov with another BolgenOS?

Answer from Ivan Tatarchuk[newbie]
launch notepad, download the toadscript compiler and start jumping butt on the keyboard
compile in 60 minutes and that’s it
your oska is ready

Answer from Mila the Sheep[newbie]
OS? Which exactly? OS-original Chester (original character (translated))
She needs milking images of herself in cartoons or films.
1. Think about what kind of cartoon/movie you want for OS
2. Consider the cartoon/film style
3. Who will your character be (fairy, pony, magician, robot, etc.)
4. Describe it mentally, then on paper
5. Come up with a design
6. Come up with a name and bio
7. Draw a character!
8. Now let's get down to business with Paint Tooth Sai